0, background
Recently, a message queue has been introduced to the company's project, and the writing scenario that does not require immediate return has been modified.
Because the current study of kafka is more in-depth, plus the summary of the experience of introducing kafka in previous projects, this process is relatively smooth.
The shortcoming is that because of the log monitoring and analysis level, ELK's log collection, storage, and display analysis technology stack has been applied. The need for multiple projects to write to the same log file.
The relevant environment is explained as follows:
JDK 1.8
springboot+springcloud 1.5.13.RELEASE
logback 1.1.11
Each microservice depends on a basic jar package
Deploy multiple microservices on the same machine
1. Process
During the test, it was found that the consumption of the message queue was normal. When analyzing the logs, it was found that the topic consumed by a certain service had no logs, and then a logback RollingFileAppender was located in a magical situation.
For example, today is April 9th, but it can stably record part of the log to today's log and the other part to the rolling log of April 8.
Naturally, yesterday ’s log file is not in the filebeat collection list, and behaves as if it has not been recorded.
One is normal, the other is like being squeezed into yesterday's log file.
In terms of the frequency of occurrence, there are sometimes problems when starting the service, and the problem is reproduced steadily at 0 o'clock.
The key condition: multiple projects share a written log path.
2. Solution
In the search process, I first read through many tangled logback relative path problems. .
Next see the RollingFileAppender provided by logback allows multi-process writing scheme: prudent option. However, due to the inability to configure the file option, it is necessary to involve logstash configuration changes and modification costs.
After repeatedly adjusting the search keywords, finally:
"There is no way to see in front of you, Cui Hao's poem is on top." The predecessor's statement is ready!
https://blog.csdn.net/Abysscarry/article/details/102847754
The scene of "Cui Brother" is non-stop deployment and process-level failover. For me, this is also a very good and unrealized effort.
Finally, refer to "Add the same class as the open source framework package path" in the project to solve the problem. Follow up to observe.
3. Memo
(1) Each time a log is printed, TimeBasedRollingPolicy will determine whether scrolling is required, and perform the scrolling operation when the conditions are found to be satisfied. And our first feeling is that we judge at 0.
(2) The native method called in the FileOutputStream constructor will automatically create the file.
(3) About repeated class loading
When a class is loaded, it will first determine whether the class has been loaded. The second same class is not loaded.
When the project uses Maven to manage dependencies, the class loading order is in the order used in pom.xml (from top to bottom, top priority is loaded); and the dependencies used by the parent POM will be loaded last
https://blog.csdn.net/jason20ming/article/details/7192486
Simply verify it, just create a class with the same package name in the basic jar package referenced by the project, and then keep the basic jar package referenced in the pom of the microservice project.
The spring-boot-starter as the parent, the logback module it contains will be loaded at the end. In this way, when ClassLoader finds that TimeBasedRollingPolicy has been loaded, it will no longer load the file of the same name of the logback module.
If you are still worried, exclude the logback that spring-boot-starter depends on, and then quote it separately at the bottom of the pom. In this way, the order rules from top to bottom are applied stably.