ELK (log audit system)

ELk Introduction and workflow

　　ELK immediately (Elasticsearch + Logstash + Kibana)

Download the installation package

System Environment: Contos7.0
Java environment: Portal (this is a historical Download, mine is jdk-8u151-linux-x64.tar.gz)
Logstash / elasticsearch / Kibana / Filebeat: Portal (I always choose the version 7.0)
redis: Portal

After completing the download to the server, to extract all under the "/ etc / elk" directory Note: This is used stand-alone deployment (memory should not be less than 2G)

Java environment configuration

tar -zvxf jdk-8u151-linux-x64.tar.gz -C /data/app/
ln -s /data/app/jdk1.8.0_151 /data/app/jdk
cat <<EOF >> /etc/profile   # 追加文件
"""
export JAVA_HOME=/data/app/jdk
PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH
CLASSPATH=.$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$JAVA_HOME/lib/tools.jar
EOF
"""
source /etc/profile
ln -s /data/app/jdk/bin/java /usr/bin/java 
java -version        # 查看是否安装成功

command

elasticsearch deployment

elasticsearch installation

-ivh elasticsearch-7.0.0- RPM . x86_64.rpm 
 # edit the configuration file 
Vim / etc / elasticsearch / elasticsearch.yml
 "" " 
path.logs: / var / log / elasticsearch 
cluster.name: elk01 
node.name: Node. 1- 
Path.Data: / var / lib / elasticsearch 
path.logs: / var / log / elasticsearch 
network.host: 0.0.0.0 
http.port,: 9200 
discovery.seed_hosts: [ "10.60.53.143",] 
cluster.initial_master_nodes: [ " 10.60.53.143 ",] 
" "" 
# specific role can read English explanation of the configuration file 
systemctl restart elasticsearch   # start service

mounting step npm

logstash deployment

rpm -ivh logstash-7.0.0.rpm

updating..................

kibana department

-ivh kibana-7.0.0- RPM x86_64.rpm
 # edit the configuration file 
vim / etc / kibana / kibana.yml
 "" " 
server.port: 5601 
server.host:" 0.0.0.0 " 
elasticsearch.hosts: [" HTTP: //10.60.53.143:9200 "] 
" "" 
# start 
systemctl start kibana 
systemctl enable kibana

filebeat deployment

installation

rpm -ivh filebeat-7.0.0-x86_64.rpm

Filebeat modify the configuration file "filebeat.yml" and Redis configuration file "6379.conf"

filebeat not running log, run directly view the system log messages to.
After configuring filebeat must restart.
After rebooting has a value redis view, there is a normal value.

# Cancellation bind field, protected-mode set to NO 
# bind 127.0.0.1 
protected-mode NO

/etc/redis/6379.conf

filebeat.inputs:
- type: log
  paths:
    - /root/channelHandle-out-2.log
  fields:
    log_file: xsj_channelhandle_out_2
    log_type: a-out-log
  fields_under_root: true
  encoding: utf-8
processors:
  - drop_event:
      when.not.contains:
        message: "收到"
output.redis:
  hosts: ["10.60.53.143:6379"]
  db: 0
  # password: "[email protected]"
  key: "%{[log_file]:xsj}"
  timeout: 5

etc/filebeat/filebaet.yml