I. Introduction
I just finished the first double 11 stress test since I was transferred, and I have gained a lot, which is hard to put into words. In this article, let’s talk about asynchronous logging first. In a system with high concurrency and high traffic, the response delay requirement is relatively small, and synchronous logging has been satisfied. There is no demand. Synchronous logging will block the thread that calls logging, and logging itself needs to be written to disk, so rt will increase. Asynchronous logging is designed to solve this problem.
Second, the log printing model
- Sync log model
As shown in the figure above, when multiple business threads print logs, they will not return until the content is written to the disk, so the rt of logging is the time-consuming of writing to the disk.
- Asynchronous logging model
As shown in the figure above, when multiple business threads print logs, they put the print task into the memory queue and then return it directly. The specific print log is a log thread with a log system that goes to the queue to get it and execute it. It can be seen that this kind of printing rt is to write Time consuming of the memory queue.
3. Some things about asynchronous logs
- Asynchronous log settings
For logback, the queue in the asynchronous log is a bounded ArrayBlockingQueue, where queueSize is the queue size, and taskLogAppender is the referenced ordinary synchronous log
discardingThreshold is a threshold, see its effect through the following code:
When the remaining capacity of the queue is less than this threshold and the current log level is TRACE, DEBUG or INFO, these logs are discarded.
During the stress test, the code configuration is as above, that is, the asynchronous log is configured, but the thread is still blocked in the place where the log is logged. After inspection, it is blocked to the put method of the log queue ArrayBlockingQueue:
It can be seen that the put method will suspend the current thread when the queue is full. So how to solve that?
The discardingThreshold is introduced above. It can be seen that setting it to 0 in this article means that the logs of level TRACE, DEBUG or INFO will never be discarded. As long as discardingThreshold>0, the logs of level TRACE, DEBUG or INFO will be discarded when the queue is almost full. This seems to be can solve the problem. But what if it prints a warn-level log? It will still block when put.
By looking at the code, I found that there is a judgment when I finally write the log:
If neverBlock=true is set, the offer method of the ArrayBlockingQueue pair will be called instead of put when writing the log queue, and the offer is non-blocking:
It can be seen that if the queue is full, it will return directly instead of suspending the current thread. So configure the asynchronous appender as follows:
4. Summary
In systems with high concurrency and low latency requirements, unimportant logs can be set to be asynchronous, and attention should be paid to setting the policy of discarding when the queue is full to prevent business threads from being suspended and affecting RT.
(Original address: http://www.importnew.com/27247.html. Respect originality, thank the author!)