foreword
We often use the Log4j logging framework. Recently, I encountered a problem related to log configuration. To put it simply, on the basis of the original log configuration, the logs of the specified class are printed to the specified log file.
This may not be so easy to understand, let me start from the source of demand.
1. Sources of demand for extended configuration
Our project uses the Log4j2 logging framework, and the log configuration log4j.ymlis as follows:
Configuration:
status: warn
Appenders:
Console:
name: Console
target: SYSTEM_OUT
# 不重要
RollingFile:
- name: ROLLING_FILE
# 不重要
Loggers:
Root:
level: info
AppenderRef:
- ref: Console
- ref: ROLLING_FILE
Logger:
- name: com.myproject
level: info
Configuration is simple, just a scrolling log file and console output. Now there is such a requirement: the HTTP interface access log of the project should be printed separately into a log file logs/access.log, and whether this function is casslog.accessLogEnabledenabled or not is determined by the configuration switch.
Just do it, I immediately changed the original log4j.ymlfile to log4j_with_accesslog.yml, and added the access log Appender: ACCESS_LOG, as shown in the following configuration.
Configuration:
status: warn
Appenders:
Console:
name: Console
target: SYSTEM_OUT
# 不重要
RollingFile:
- name: ROLLING_FILE
# 不重要
### 新增的配置开始（1） ###
- name: ACCESS_LOG
fileName: logs/access.log
### 新增的配置结束（1） ###
Loggers:
Root:
level: info
AppenderRef:
- ref: Console
- ref: ROLLING_FILE
Logger:
- name: com.myproject
level: info
### 新增的配置开始（2） ###
- name: com.myproject.commons.AccessLog
level: trace
additivity: false
AppenderRef:
- ref: Console
- ref: ACCESS_LOG
### 新增的配置结束（2） ###
[New configuration start (1)] and [New configuration start (2)] in the above configuration comments are the added configuration content. The function switch is implemented as follows, and the judgment is made when the project starts.
import org.springframework.boot.logging.log4j2.Log4J2LoggingSystem;
public class MyProjectLoggingSystem extends Log4J2LoggingSystem {
static final boolean accessLogEnabled =
Boolean.parseBoolean(System.getProperty("casslog.accessLogEnabled", "true"));
@Override
protected String[] getStandardConfigLocations() {
if (accessLogEnabled) {
return new String[]{"casslog_with_accesslog.yml"};
}
return new String[]{"casslog.yml"};
}
}
In this way, the function is realized, and the program can indeed run. But it always feels not elegant enough. If there are hundreds of projects that need to add this function, the log configuration files of these projects will have to be changed, and it will crash even if you think about it.
2. Look at the implementation of the open source project Nacos
Friends who have used Nacos may know that the configuration module and service discovery module of Nacos are two functions, and the logs are also separated. It can be seen nacos-client.jarin the specific passage .nacos-log4j2.xml
Note that the Nacos source code version of this article is
nacos-client 1.4.1.
nacos-log4j2.xmlI made a simplification, the content is as follows.
<Configuration status="WARN">
<Appenders>
<RollingFile name="CONFIG_LOG_FILE" fileName="${sys:JM.LOG.PATH}/nacos/config.log"
filePattern="${sys:JM.LOG.PATH}/nacos/config.log.%d{yyyy-MM-dd}.%i">
<!-- 不重要 -->
</RollingFile>
<RollingFile name="NAMING_LOG_FILE" fileName="${sys:JM.LOG.PATH}/nacos/naming.log"
filePattern="${sys:JM.LOG.PATH}/nacos/naming.log.%d{yyyy-MM-dd}.%i">
<!-- 不重要 -->
</RollingFile>
</Appenders>
<Loggers>
<!-- 不重要 -->
<Logger name="com.alibaba.nacos.client.config" level="${sys:com.alibaba.nacos.config.log.level:-info}"
additivity="false">
<AppenderRef ref="CONFIG_LOG_FILE"/>
</Logger>
<Logger name="com.alibaba.nacos.client.naming" level="${sys:com.alibaba.nacos.naming.log.level:-info}"
additivity="false">
<AppenderRef ref="NAMING_LOG_FILE"/>
</Logger>
<!-- 不重要 -->
</Loggers>
</Configuration>
com.alibaba.nacos.client.configFrom the above log configuration, we can see that Nacos outputs the log of the class with the package name to ${sys:JM.LOG.PATH}/nacos/config.logthe file, and com.alibaba.nacos.client.namingoutputs the log of the class with the package name to ${sys:JM.LOG.PATH}/nacos/naming.logthe file. ${sys:JM.LOG.PATH}The path configured by default is the user directory.
Next, let's see how Nacos loads the log configuration into the application. (Please appreciate the implementation code yourself)
import static org.slf4j.LoggerFactory.getLogger;
public class LogUtils {
public static final Logger NAMING_LOGGER;
static {
NacosLogging.getInstance().loadConfiguration();
NAMING_LOGGER = getLogger("com.alibaba.nacos.client.naming");
}
}
public class NacosLogging {
private AbstractNacosLogging nacosLogging;
public void loadConfiguration() {
try {
nacosLogging.loadConfiguration();
}
// 省略...
}
}
public abstract class AbstractNacosLogging {
public abstract void loadConfiguration();
}
public class Log4J2NacosLogging extends AbstractNacosLogging {
private final String location = getLocation("classpath:nacos-log4j2.xml");
@Override
public void loadConfiguration() {
final LoggerContext loggerContext = (LoggerContext) LogManager.getContext(false);
final Configuration contextConfiguration = loggerContext.getConfiguration();
// load and start nacos configuration
Configuration configuration = loadConfiguration(loggerContext, location);
configuration.start();
// append loggers and appenders to contextConfiguration
Map<String, Appender> appenders = configuration.getAppenders();
for (Appender appender : appenders.values()) {
contextConfiguration.addAppender(appender);
}
Map<String, LoggerConfig> loggers = configuration.getLoggers();
for (String name : loggers.keySet()) {
if (name.startsWith(NACOS_LOGGER_PREFIX)) {
contextConfiguration.addLogger(name, loggers.get(name));
}
}
loggerContext.updateLoggers();
}
}
To sum up, it is to first nacos-log4j2.xmlconvert the extended configuration (i.e.) into LoggerConfigan object; then LoggerConfigadd the instance to the application's log configuration context contextConfiguration; and finally update the application Loggers.
3. Learn and use immediately
We regard the extended log as an object. For example, the "access log" here and the "configuration module log" in Nacos can be called extended logs. Let's start by writing the abstraction for extending the log AbstractLogExtend.
@Slf4j
public abstract class AbstractLogExtend {
public void loadConfiguration() {
final LoggerContext loggerContext = (LoggerContext) LogManager.getContext(false);
final Configuration contextConfiguration = loggerContext.getConfiguration();
// load and start casslog extend configuration
Configuration configurationExtend = loadConfiguration(loggerContext);
configurationExtend.start();
// append loggers and appenders to contextConfiguration
Map<String, Appender> appenders = configurationExtend.getAppenders();
for (Appender appender : appenders.values()) {
addAppender(contextConfiguration, appender);
}
Map<String, LoggerConfig> loggersExtend = configurationExtend.getLoggers();
loggersExtend.forEach((loggerName, loggerConfig) ->
addLogger(contextConfiguration, loggerName, loggerConfig)
);
loggerContext.updateLoggers();
}
private Configuration loadConfiguration(LoggerContext loggerContext) {
try {
URL url = ResourceUtils.getResourceUrl(logConfig());
ConfigurationSource source = getConfigurationSource(url);
// since log4j 2.7 getConfiguration(LoggerContext loggerContext, ConfigurationSource source)
return ConfigurationFactory.getInstance().getConfiguration(loggerContext, source);
} catch (Exception e) {
throw new IllegalStateException("Could not initialize Log4J2 logging from " + logConfig(), e);
}
}
/**
* 要扩展配置的文件名
*/
public abstract String logConfig();
}
AbstractLogExtendTwo methods are defined, namely:
- loadConfiguration(): load the extended log configuration;
- logConfig(): the path of the extended log configuration file;
We then load these extended logs into the application.
public class LogExtendInitializer {
private final List<AbstractLogExtend> cassLogExtends;
@PostConstruct
public void init() {
cassLogExtends.forEach(cassLogExtend -> {
try {
cassLogExtend.loadConfiguration();
}
// 省略...
});
}
}
At this point, the basic class code is written. Let's go back to the requirements at the beginning of the article to see how to achieve them.
First configure the access log accesslog-log4j.xml.
<Configuration status="WARN">
<Appenders>
<!-- 不重要 -->
<RollingFile name="ACCESS_LOG" fileName="logs/access.log"
filePattern="logs/$${date:yyyy-MM}/access-%d{yyyy-MM-dd}-%i.log.gz">
<!-- 不重要 -->
</RollingFile>
</Appenders>
<Loggers>
<Root level="INFO"/>
<Logger name="com.myproject.commons.AccessLog" level="trace" additivity="false">
<AppenderRef ref="Console"/>
<AppenderRef ref="ACCESS_LOG"/>
</Logger>
</Loggers>
</Configuration>
I will accesslog-log4j.xmlput it under the class package here.
Then there is accesslog-log4j.xmlthe path of the configured file. Here I define "access log" as an object AccessLogConfigExtend.
public class AccessLogConfigExtend extends AbstractLogExtend {
@Override
public String logConfig() {
return "classpath:com/github/open/casslog/accesslog/accesslog-log4j.xml";
}
}
In this way, the access log is configured, and the access log can also be packaged into a basic jarpackage for use by other projects, so that other projects do not need to be configured repeatedly.
For configuration switches, you can use @Conditionalto achieve, as follows.
@Configuration
@ConditionalOnProperty(value = "casslog.accessLogEnabled")
public class AccessLogAutoConfiguration {
@Bean
public AccessLogConfigExtend accessLogConfigExtend() {
return new AccessLogConfigExtend();
}
}
This implementation is indeed a lot more elegant!
summary
This case is a function implemented by the log component I was working on before, and the source code is placed on my Github: https://github.com/studeyang/casslog . At the beginning, the access log was implemented through the inelegant method mentioned in the article. Later, when monitoring message consumption, I wanted to put the consumption log separately into a new log file for ELK to collect and analyze. Therefore, the functional commonality between [Access Log] and [Consumption Monitoring] is extracted to realize the expansion of logs.
If you want to communicate with me, welcome to follow my WeChat public account [Student Yang technotes].