Asynchronous log output mode of log4j2

background

Use log4j2's synchronization log for log output. The log output statement and the program's business logic statement will run in the same thread.
When using asynchronous logs for output, log output statements and business logic statements are not run in the same thread, but a dedicated thread is used for log output operations. The main thread that processes business logic can perform subsequent business without waiting. logic.
There are two ways to implement asynchronous logging in Log4j2: AsyncAppender and AsyncLogger.
among them:

• AsyncAppender uses ArrayBlockingQueue to save log events that need to be output asynchronously;
• AsyncLogger uses the Disruptor framework to achieve high throughput.

The first way to achieve asynchronous AsyncAppender

AsyncAppender is directly configured in the xml configuration file of log4j2, pay attention to the comment position of the following code

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn">
  <Appenders>
    <!--正常的Appender配置，此处配置的RollingFile会在下面AsyncAppender被通过name引用-->
    <RollingFile name="RollingFileError" fileName="${Log_Home}/error.${date:yyyy-MM-dd}.log" immediateFlush="true"
filePattern="${Log_Home}/$${date:yyyy-MM}/error-%d{MM-dd-yyyy}-%i.log.gz">
    	<PatternLayout pattern="%d{yyyy-MM-dd 'at' HH:mm:ss z} %-5level %logger{36} : %msg%xEx%n"/>
    	<ThresholdFilter level="error" onMatch="ACCEPT" onMismatch="DENY"/>
    	<Policies>
                <TimeBasedTriggeringPolicy modulate="true" interval="1"/>
                <SizeBasedTriggeringPolicy size="10MB"/>
    	</Policies>
   </RollingFile>
    <!--一个Appender配置完毕-->
    <!--异步AsyncAppender进行配置直接引用上面的RollingFile的name-->
    <Async name="Async">
      <AppenderRef ref="MyFile"/>
    </Async>
    <!--异步AsyncAppender配置完毕，需要几个配置几个-->
  </Appenders>
  <Loggers>
    <Root level="error">
      <!--此处如果引用异步AsyncAppender的name就是异步输出日志-->
      <!--此处如果引用Appenders标签中RollingFile的name就是同步输出日志-->
      <AppenderRef ref="Async"/>
    </Root>
  </Loggers>
</Configuration>

The key content is all in the comments of the above code. The configuration of AsyncAppender is implemented in the xml file, and there is no need to separately quote the package to support. After configuring AsyncAppender, the operation of writing log events to the file will be executed in a separate thread.
Common parameters of AsyncAppender

parameter name	Types of	Description
name	String	The name of Async Appender
AppenderRef	String	The name of the Appender called asynchronously, you can configure multiple
blocking	boolean	The default is true. If true, appender will wait until there is an idle in the queue; if false, log events will be discarded when the queue is full. (If the error appender is configured, the log events to be discarded will be processed by the error appender)
bufferSize	integer	The maximum number of log events that can be stored in the queue, the default is 128

The second way to achieve asynchronous AsyncLogger

AsyncLogger in Log4j2 uses the Disruptor framework internally.

Introduction to Disruptor

Disruptor is a high-performance queue developed by LMAX, a British foreign exchange trading company. The system based on Disruptor can support 6 million orders per second in a single thread.
At present, many well-known projects including Apache Strom and Log4j2 have applied Disruptor to obtain high performance.
The core data structure inside the Disruptor framework is RingBuffer, which is a lock-free ring queue.

Why is the Disruptor so fast?

• lock-free-uses CAS to achieve thread safety
• Use cache line filling to solve the false sharing problem

First apply the relevant package in the pom list

<dependency>
  <groupId>com.lmax</groupId>
  <artifactId>disruptor</artifactId>
  <version>3.4.2</version>
</dependency>

The second step is to configure the AsyncLogger
log4j2.xml configuration in the xml file of log4j2 as follows:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="debug" name="MyApp" packages="">
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n" />
        </Console>
        <RollingFile name="RollingFile" fileName="logs/app.log"
                     filePattern="logs/app-%d{yyyy-MM-dd HH}.log">
            <PatternLayout>
                <Pattern>%d %p %c{
    
    1.} [%t] %m%n</Pattern>
            </PatternLayout>
            <Policies>
                <SizeBasedTriggeringPolicy size="500MB"/>
            </Policies>
        </RollingFile>
        <RollingFile name="RollingFile2" fileName="logs/app2.log"
                     filePattern="logs/app2-%d{yyyy-MM-dd HH}.log">
            <PatternLayout>
                <Pattern>%d %p %c{
    
    1.} [%t] %m%n</Pattern>
            </PatternLayout>
            <Policies>
                <SizeBasedTriggeringPolicy size="500MB"/>
            </Policies>
        </RollingFile>
    </Appenders>
    <Loggers>
    <!--上面的配置都和原配置一样，就是在下方这直接定义AsyncLogger，他的name在java类中被引用即可-->
        <AsyncLogger name="com.meituan.Main" level="trace" additivity="false">
            <appender-ref ref="RollingFile"/>
        </AsyncLogger>
        <AsyncLogger name="RollingFile2" level="trace" additivity="false">
            <appender-ref ref="RollingFile2"/>
        </AsyncLogger>
        <Root level="debug">
            <AppenderRef ref="Console"/>
            <AppenderRef ref="RollingFile"/>
        </Root>
    </Loggers>
</Configuration>

The java code is as follows:

public class Main {
    
    
    public static void main(String args[]) {
    
    
        //引用com.meituan.Main日志输出器
        Logger logger = LogManager.getLogger(Main.class);
        //引用的名为RollingFile2的异步AsyncLogger
        Logger logger2 = LogManager.getLogger("RollingFile2");
        Person person = new Person("Li", "lei");
        logger.info("hello, {}", person);
        logger2.info("good bye, {}", person);
}

Two AsyncLoggers are configured in the above log4j2.xml, the names are com.meituan.Main and RollingFile2.
Also, use two loggers in the main method to output two logs.
In the startup phase of loading log4j2.xml, if AsyncRoot or AsyncLogger is detected, a disruptor instance will be started.