filebeat distinguish log collection when logstash

filebeat distinguish log collection when logstash

 

 

1. Scene

filebeat while collecting and nginx web server log project, the need for two logs were processed in logstash

2. The difference between the versions

== == 6.x may be used prior to the prospectors filebeat configuration inside document_type type, then if [type] == "string" in which logstash to match, not here detailed records
after == == Configuration 6.x file does not support document_type, that is to say the old method is invalid, currently I'm using == == version 6.2.3 of filebeat

3. Solution

Filebeat fields in a new field inside, and record keeping of this field in distinguishing the logstahs

filebeat Configuration

filebeat.prospectors:

- type: log

  paths:
    - /Library/apache-tomcat-8.5.15/bin/logs/web.log
    #- c:\programdata\elasticsearch\logs\*

  fields: 
    document_type: weblog   #这一行的key：value都可以自己定义

 

logstash Configuration

output{
    if[fields][document_type] == "weblog" {
        stdout { codec => rubydebug }  
    }
}