I. Overview
1. What is logstash
According to the official website :
Logstash is an open source server-side data processing pipeline capable of ingesting data from multiple sources simultaneously, transforming it, and sending it to your favorite "repository". (Our repository is of course Elasticsearch.)
//Belong to elasticsearch's product (JRuby development, the developer once said that if he knew scala, he would not use jruby..)
That is to say, it is the "back wave" of flume, which solves the problem of data loss of the "front wave" flume!
2. Infrastructure
Input: Collect data from various sources
Filtering: Analyze and transform data in real time
Output: Select repository to export data
For more detailed introduction, including specific supported input and output, please refer to : https://www.elastic.co/guide/index.html
2. Installation
logstash5.x 6.x requires JDK1.8+, if not installed, please install JDK1.8+ first
1. Download
https://www.elastic.co/downloads/past-releases
Select the appropriate version and download it
2. Unzip
[hadoop@mini1 ~]$ tar -zxvf logstash-5.6.9.tar.gz -C apps/