logstash-6.2.3从kafka读取日志输出到elasticsearch-6.2.3中

下载

wget https://artifacts.elastic.co/downloads/logstash/logstash-6.2.3.tar.gz

解压

tar -zxvf logstash-6.2.3.tar.gz

配置

从kafka读取数据输出到elasticsearch中，中间少了filter的配置，仅仅是个最简单的配置

input {   
    kafka {
        bootstrap_servers => ["10.112.101.90:9092"] # 注意这里配置的kafka的broker地址不是zk的地址
        group_id => "logstash" # 自定义groupid 
        topics => ["ecplogs"]  # kafka topic 名称 
        consumer_threads => 5 
        decorate_events => true
        codec => "json"
        }
}
output {
  elasticsearch { hosts => ["10.112.101.90:9200"] }
  stdout { codec => rubydebug } 
}

运行

bin/logstash -f logstash.conf