foreword
Recently, a friend always complained to me: the company system log is really bad, there is very little useful information and a lot of useless. Even that useful information is a pain to track down among so many node logs.
I asked him if the company did not collect logs. It is better to collect logs than to view the logs of each node. He said that the company has access to a paid third-party log product and collected it. However, it is only convenient for unified viewing and searching, but the log of the system itself lacks some key elements. It is messy, and it is difficult to locate when viewing call logs between many microservices.
The problems can be summed up as follows:
- Not all logs have key information, such as order number and SKU information. Some logs have it, and some logs don't. As a result, although some logs display the processing step log, it is not known which object is processed.
- There is no uniform standard for the log, which makes it look very disorganized
- It is more painful to trace the call chain of the same request between microservices. It is often only possible to search according to the timestamp. When the concurrency is small, it can be found by the timestamp. If the concurrency is large, the cost of checking the problem is too high.
I recommended him some distributed tracing frameworks, skywalking, pinpoint. After reading it, he said that it is very complete, but the cost of construction and implementation is a bit high. There are also storage costs involved, and the company has purchased a third-party log service. It's still a bit of a hassle to connect. I'm afraid the above does not agree with such a toss.
Recently, I saw such an open source framework in the open source community. It claims to be a lightweight log tracking artifact. It can be accessed in 10 minutes, so I recommended it to a friend. A few days later, he told me that this thing is very suitable for his current pain point, and it has been put into production now. Judging from the preliminary results, he is still very satisfied. It can reduce the time cost for their log location.
Project Features
Invited, so I plan to analyze this framework. Give everyone an intuitive understanding.
This framework is called TLog, and the project is hosted on Gitee
https://gitee.com/dromara/TLog
The homepage looks like this, with a strong darkness. . .
When I use it, to put it bluntly, TLog automatically prefixes each line of logs, which is the so-called 标签. Labels are divided into system-level labels and business-type labels, among which business-type labels can be customized by developers. Draw a picture for easy understanding:
Each line of log that TLog finally renders, as shown in the figure above. Among them, there are currently 5 pieces of information that can be displayed in the system log. The upstream information can let you know who called your system. The link TraceId is the unique ID of the global link across microservices. You can get all the information by searching for an ID. A log of the same request in the system. This is still delicious!
Regarding SpanId, the explanation on the official website is that this represents the position of the call in the entire call chain tree. The specific demonstration borrows the diagram from the official website, and the explanation is fairly clear:
Personal understanding of SpanId is that this thing can let you know the level of the system in a certain call chain. If you collect it, you can generate a call chain tree through spanId. In fact, I really hope that TLog can realize the display of this tree, but unfortunately it is not supported yet.
"The positioning of TLog is to provide the easiest way to solve the problem of log tracking. It does not collect logs and does not require additional storage space. It just automatically tags your business logs and automatically generates TraceId throughout you. A whole link of microservices. And provides upstream and downstream node information. It is suitable for small and medium-sized enterprises and corporate projects that want to quickly solve log tracking problems."
This is a repeat of the official website. In fact, when I was testing, the logs provided by TLog were the logs themselves. In multi-node microservices, the logs are still scattered. It just logically concatenates the logs to a certain extent. But at the same time, the TLog documentation also recommends that other solutions be used for log collection. For example, ELK, Alibaba Cloud Log, other paid log products, etc. TLog just modifies the log and does not generate a new log. Therefore, there is no problem in connecting with other log collection solutions. For companies that have already connected with log collection solutions, there is no need to modify anything.
Supported logging frameworks
Every company uses a variety of log frames. TLog claims to support the three major logging frameworks: log4j, log4j2, logback
In the actual test, in these three frameworks, TLog can also print out labels normally. It's just that during the access process, there are three official access methods.
After testing, the javaagent method is indeed unstable for some projects, and some complex projects will be invalid. Regarding the claim to be the most stable log adaptation method, I tested the company's project, and it was indeed able to access it smoothly.
Access method, follow the document step by step.
Supported RPC frameworks
Since log tracking is performed across microservices, commonly used RPCs should also be supported in terms of implementation. TLog supports three common RPCs: Dubbo, SpringCloud, and Dubbox, which is good.
In the actual test, in the Spring cloud, TLog also supports the Gateway of Spring Cloud.
During the access process, no matter which RPC framework it is, TLog can also be automatically adapted in the springboot environment, and it can be automatically assembled by introducing one. No additional configuration is required. This is very convenient.
However, in the native spring environment (non-springboot), additional configuration of xml is still required, but it is relatively simple, and the documentation also has special instructions.
business label
In addition to the tags given by the system, I found that another major feature of TLog is that it allows developers to customize tags. Access is also very simple, for example:
@TLogAspect({"str","user.name","user.userDetail.company","user.dddd"})
public void test(String str, User user){
log.info("这是自定义表达标签");
log.info("这是业务日志1");
log.info("这是业务日志2");
log.info("这是业务日志3");
log.info("这是业务日志4");
log.info("这是业务日志5");
}
As long as you add a label to the method, all logs under this method, including the next N levels, will automatically add the label you defined
This function can add a lot of marks to the typesetting and searching of logs. This is awesome!
Even custom tags support logical processing of information, and custom classes can be used to process this information
@TLogAspect(convert = CustomAspectLogConvert.class)
public void demo(Person person){
log.info("自定义Convert示例");
}
This specific effect, you can try it. In short, one label handles all custom labels.
Support for other scenarios
Parameters & time-consuming printing support:
After the introduction of TLog, I found that TLog also comes with the parameter printing and time-consuming of each method under which framework, the default is off, and you only need to configure it:
tlog.enableInvokeTimePrint=true
The result is as follows:
Asynchronous thread & thread pool support
If your project has asynchronous threads, the consistency of label delivery is also automatically supported without any problems.
But for thread pool scenarios, TLog does not have native support. But provides a helper class that requires a small amount of invasive code. There is room for improvement.
MQ support
Similarly, TLog also takes into account the delivery of MQ scene tags. This also requires a small amount of hacking code. If you don't change anything, it cannot be supported in the MQ scenario.
performance
As for the performance, I did a simple test on TLog, only in the log4j2 environment, the test condition is to print out several w logs synchronously, the time-consuming in the native environment and the time-consuming after adding the TLog framework, Each group was measured 10 times, and the average value was taken.
The test code is very simple:
StopWatch stopWatch = new StopWatch();
stopWatch.start();
for (int i = 0; i < 100; i++) {
log.info("test log {}", i+1);
}
stopWatch.stop();
log.info("耗时:{}",stopWatch.getTotalTimeSeconds());
Without TLog
| Print 5w logs (seconds) | Print 10w logs (seconds) | |
|---|---|---|
| 1st | 6.496249974 | 15.595447718 |
| 2nd | 6.185712521 | 14.295489162 |
| the 3rd time | 6.116123718 | 13.559289437 |
| 4th | 6.205771261 | 12.782565374 |
| 5th | 6.727208117 | 12.884360048 |
| 6th time | 5.908489157 | 14.604699842 |
| 7th time | 6.153151066 | 13.700609245 |
| 8th time | 6.603960836 | 13.048889457 |
| 9th time | 6.139718196 | 12.584335736 |
| 10th time | 6.365920588 | 12.85222910 |
| average | 6.29 | 13.60 |
Join TLog
| Print 5w logs (seconds) | Print 10w logs (seconds) | |
|---|---|---|
| 1st | 5.997981416 | 12.878389572 |
| 2nd | 6.154590093 | 14.268328625 |
| the 3rd time | 6.228010581 | 12.385200456 |
| 4th | 6.452949788 | 15.542794904 |
| 5th | 6.156225995 | 12.350440713 |
| 6th time | 6.121611887 | 12.543883453 |
| 7th time | 6.18131273 | 12.192140225 |
| 8th time | 6.238254682 | 12.159684042 |
| 9th time | 6.403632588 | 12.308115127 |
| 10th time | 5.954781153 | 12.321667925 |
| average | 6.19 | 12.89 |
The test results are a bit unexpected. After adding TLog, the average of 10 times is faster than not adding it. But I speculate that the test environment and the number of samples are too small. It does not mean that adding is faster than not adding. Can only say that if you do 100, 1000 tests. The 2 should be about the same.
This performance test also reflects that TLog will not bring additional overhead to the system. Appreciate this too.
Summarize
In this evaluation, the open source framework TLog is generally a logging framework, but it is a very distinctive Java open source project integrating distributed tracing, log tags and other small functions. Structurally, it is very small. And the performance is also very good. In terms of practicality, it solves the pain point of quickly locating logs in small and medium-sized companies. The disadvantage is that more effective log mining cannot be done without collecting logs, but this is also the reason why TLog claims 10-minute access. From an objective point of view, this has advantages and disadvantages. Hope TLog can improve the function of this part in the future.
about me
I'm an open source author and a content creator. "Yuanren Tribe" is a technology sharing account that insists on being original. It will always share original technical articles and grow with you.
