Traditional ELK icon:
Simple implementation of distributed log collection using ElK disadvantages?
1, logstash too much, not expansion.
This form is shown above a tomcat corresponds to a logstash, add a node must have the same logstash, it can be very wasteful.
2, read IO file may be generated logs are lost.
3, not a real-time
For example logstash, found by a timer after the underlying changes to database synchronization, the timing is the inevitable delay.
Well, since the emergence of these problems, what solution?
Installation kafka
kafka release is based on a subscription model, similar to producers and consumers.
Photo kafka get to know the role:
Thinking:
1. Logstash input sources are those?
Local files, kafka, database, mongdb, redis, etc.
2. Those logs will need to enter logstash
error level
How to distinguish between log index file 3. AOP exception notification service and service
service name
4. In a distributed log collection, the same service cluster is then no need to distinguish log index file.
The purpose of the unified management of the same node log my information.
6. The same service cluster, then, is to distinguish between log index file is not needed when the search log, how to locate the server node information it?
To be continued.