Because logstash jvm is running, resource consumption is relatively large, so later authors also used to write a function golang less resource consumption but also small lightweight logstash-forwarder. However, the authors just one person, after joining the company http://elastic.co, es because the company itself has acquired another open source project packetbeat, and this project is to use specialized golang, there is the whole team, so just put the company es logstash- forwarder development work also merged into the same golang carry out his team, so a new project called filebeat.
logstash and filebeat have a log collection function, filebeat lighter, less resource intensive, but logstash has a filter function to filter the log analysis. Acquisition of log structures are generally filebeat, then sent to the message queue, redis, kafaka. Then logstash to acquire, filtration analysis using a filter function, and then stored in elasticsearch