loastash 会占用系统资源,所以用filebeat来做传输,在专门的虚机上搭logstash
1、filebeat安装
rpm -ivh https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.1.1-x86_64.rpm
cd /etc/filebeat
mv filebeat.yml filebeat.yml.bak
//
vi filebeat.yml
filebeat.prospectors:
- input_type: log
paths:- /var/log/messages.log
tail_files: true
close_inactive: 2m
scan_frequency: 1s
exclude_lines: ["^debug"]
include_lines: ["^err", "^warn"]
output.logstash:
hosts: ["172.16.54.95:5044"]
//启动
service filebeat start
2、logstash端,开放5044端口
vi messages.conf
input{
beats{
port => "5044"
}
}
output {
stdout{codec=>rubydebug}
elasticsearch {
hosts => "172.16.54.95:9200"
index => "lvsmessags"
}
}
//elasticsearch可以指定document_id:document_id => "%{uid}"
- /var/log/messages.log