ELK--Logstash installation

China Mirror: https://www.newbe.pro/Mirrors/Mirrors-Logstash/

https://mirrors.huaweicloud.com/logstash/6.4.1/

1. Download the logstash installation package

wget https://mirrors.huaweicloud.com/logstash/6.4.1/logstash-6.4.1.tar.gz

2. Unzip the installation package

tar -xvf logstash-6.4.1.tar.gz

Installation package structure:

3. Configure Logstash

To configure Logstash, you need to create a configuration file that specifies the plug-ins to be used and the settings of each plug-in. When running logstash, use -f to specify the configuration file. Official website demo: https://www.elastic.co/guide/en/logstash/current/config-examples.html

3.1 Direct start

bin/logstash -e 'input { stdin {} } output { stdout{} }'

bin/logstash -e 'input { stdin {} } output { stdout{codec => ouyangcheng} }'

bin/logstash -e 'input { stdin {} } output { elasticsearch {hosts => ["127.0.0.1:9200"]} stdout{} }'

bin/logstash -e 'input { stdin {} } output { elasticsearch {hosts => ["192.168.0.101:9200", "192.168.0.102:9200"]} stdout{} }'

3.2 In the form of a configuration file

3.2.1 Edit the configuration file: vim logstash-comcat.config

input {
    file {
        type => "tomcatlog"
        path => "/home/data/logs/*/*.log"
        discover_interval => 10
    }
}
 
output {
    elasticsearch {
        index => "tomcat-%{+YYYY.MM.dd}"
        hosts => ["127.0.0.1:9200"]
    }
}

3.2.2 Start logstack

sh bin/logstash -f logstash-comcat.config

3.2.3 msyql data synchronization

(1) Edit the mysql configuration file

vim logstash-mysql.config

(2) The content of the file is:

input {
    stdin {}
    jdbc {
        type => "jdbc"
        jdbc_user => "root"
        jdbc_password => "123456"
        jdbc_driver_library => "/home/data/soft/logstash-6.4.1/mysql-connector-java-8.0.13.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        statement => "SELECT * from t_blog where update_time >= :sql_last_value"
        tracking_column => "update_time"
        clean_run => true
        schedule => "* * * * *"
        jdbc_connection_string => "jdbc:mysql://127.0.0.1:3306/oyc?characterEncoding=UTF-8"
    }
}
output {
    elasticsearch {
        hosts => ["127.0.0.1:9200"]
        index => "blog"
        document_id => "%{id}"
        document_type => "blog"
    }
    stdout {
        codec => json_lines
    }
}

(3) Start

nohup sh bin/logstash -f  logstash-mysql.config &

(4) Effect

Synchronous Data:

Use kibana to query data:

For more cases, please refer to the gitbook manual: http://doc.yonyoucloud.com/doc/logstash-best-practice-cn/index.html

https://blog.csdn.net/qq_38270106/article/details/88699334