1. Background
In daily development, a lot of INFO-level logs are usually printed for the convenience of debugging and checking problems.
With the increasing number of visits, if you are not careful, the size of a log file in one day will be larger than a certain threshold (such as 5G). Therefore, you will receive an alarm to optimize the log size, and will report back to your supervisor if the log size is not optimized within a certain period of time. ,Oops...
If the log is too large, it is easy to cause some operation and maintenance operations to consume machine performance, such as log file retrieval, data collection, and disk cleaning.
So, what are the common ideas for log slimming? This article is based on a specific case to talk about my views.
2. Diary slimming methodology
2.1 Print only necessary logs
Sometimes, for the convenience of testing, a lot of INFO level logs are temporarily printed. For such logs, unnecessary logs can be deleted or adjusted to DEBUG level before the project goes live.
However, in some scenarios, some logs can be printed as DEBUG or INFO. Printing at INFO level takes up space, and printing at DEBUG level needs to be used when checking problems online. What should I do?
We can transform the log tool class to support a certain switch in context transfer (normal calls do not have this switch, but pass through the company's Tracer or RPC context), and can temporarily upgrade the DEBUG log to the INFO level. The pseudo code is as follows:
if(log.isDebugEnable()){
log.debug(xxx);
}else if(TracerUtils.openDebug2Info()){
log.info("【debug2info】"+xxx);
}
In this way, some logs that are confused about whether to print as INFO log can be printed as DEBUG level, and automatically upgraded to INFO log when the problem is checked. In order to avoid misunderstandings, distinguish between DEBUG boost INFO logs and normal INFO logs, and add a log prefix similar to [debug2info].
Of course, you can also engage in some other salacious operations, here is just an example, please make inferences by yourself.
2.2 Combined printing
Some logs that can be merged can be considered for merging.
If the INFO log is printed before and after the same method:
INFO [64-bit traceId] XXXService size =10 before execution INFO [64-bit traceId] XXXService size =4 after execution
Can be combined into one:
INFO [64-bit traceId] XXXService size =10 before execution and size =4 after execution
2.3 Simplification & Abbreviation & Compression
某个日志非常有必要，但是打印的对象有些大，如果可以满足问题排查需求的情况下，我们可以：
（1）选择只打印其 ID
（2）创建一个只保留关键字段的日志专用对象，转化为日志专用对象，再打印。
（3）可以用缩写，如 write 简化为 w, read 简化为 r, execute 简化为e 等；比如 pipeline 中有 20个核心 bean ，打印日志时可以使用不同的编号替代 bean 全称，如 S1,S2 ，虽然没那么直观，但既可以查问题，又降低了日志量。
三、优化案例
3.1 场景描述
一个业务场景涉及很多 bean， 为了复用一些通用逻辑，这些 bean 都继承自某个抽象类。 在抽象类中，定义了执行 bean 前后的一些通用逻辑，如执行前后打印当前 pipeline 中 item 的数量。 最后一个 bean 执行完结果转换后需要打印出结果。
3.2 优化分析
3.2.1 只打印必要日志
（1）由于当前 bean 执行前 相当于前一个 bean 执行后，因此只打印执行后的日志就可以，执行前的INFO 日志可以删除或者改为 DEBUG （只打印必要日志）
（2）通常问题只出现在执行前后 size 不一致的情况下，因此执行后打印日志前可以加个判断，如果执行前后 size 相同则不打印。（只打印必要日志） 伪代码如下：
if(sizeBefore != sizeAfter){
log.info("service:{}, 前size:{},后size:{}", getName(),sizeBefore, sizeAfter)
}
这招效果很明显，因为大多数 bean 的执行前后 size 是相同的，就不会打印这条日志。 而假设之前有 20 个，这条日志就需要打印 20次，改进后可能只需要打印 2-3 次。
3.2.2 日志合并
（2）为了方便查问题还需要打印执行前的 size ，那么将执行前的 size 记录在内存中，打印执行后日志时多打印出执行前的 size。（合并打印） 伪代码如下：
log.info("service:{}, 执行前size:{}", getName(),sizeBefore)
log.info("service:{}, 执行后size:{}", getName(),sizeBefore, sizeAfter)
合并后
log.info("service:{}, 前size:{},后size:{}", getName(),sizeBefore, sizeAfter)
3.2.3 日志精简
对于最终结果，将结果对象（如 XXDTO）转化为只包括关键信息，如 id, title 的日志对象（XXSimpleLogDTO），转化为日志对象后再打印。
log.info("resultId:{}",result.getId());
或者
log.info("result:{}",toSimpleLog(result));
3.3 效果评估
The log generates about 5G a day, and about 80% of it is the size before and after the print execution, about 10% is the final result of printing, and some other logs. After the above method is optimized, the daily log volume is less than 1G.
There is a trade-off between meeting the needs of troubleshooting and realizing log thinning.
4. Summary
Log thinning requires a trade-off, keeping the logs necessary to troubleshoot problems as thin as possible.
It can be optimized by deleting unnecessary logs, merging logs, and simplifying logs.
We can also perform some tricky operations and support online DEBUG to temporarily increase INFO (of course, arthas can also be used) to assist us in checking problems.
What other good journaling tips do you have? Feel free to comment and communicate with me at the end of the article.
It is not easy to create. If this article is helpful to you, please like, favorite and follow. Your support and encouragement are the biggest motivation for my creation.