filebeat + kafka

kafka appears not receive filebeat data, and finally find the version compatibility issues

filebeat replaced filebeat-7.4.2-linux-x86_64 

kafka is docker-compose start, the version is kafka_2.12-2.3.0

 

filebeat Configuration

filebeat.inputs:
  - type: log
    enabled: true
    paths:
      - /opt/logs/debug.log # log path
output.kafka:
  hosts: ["10.238.2.228:9092", "10.238.2.229:9092", "10.238.2.230:9092"]   #kafka地址
  topic: 'xxx-debug-log'
  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000
  processors:
    - drop_fields:
        fields: ["input", "ecs", "agent"]