ZERO
Continuously updated , please visit: https://zorkelvll.cn/blogs/zorkelvll/articles/2019/03/20/1553062484473
background
Due to recent project requirements, the need to establish a system log aggregation, classic ELK (logstash = "Elasticsearch =" kibana) log system architecture, this is a record of the installation process associated elasticsearch \ logstash \ kibana \ filebeat such as the environment!
Description:
- Data flow to: filebeat => logstash => Elasticsearch => kibana
- Components boot sequence: Elasticsearch => logstash => kibana (source ELK name?) => Filebeat
- elasticsearch-6.6.1 logstash-6.6.2 kibana-6.6.2 filebeat-6.6.2
download
In elk official website to download the installation package logstash \ elasticsearch \ kibana and filebeats, which Elasticsearch the download and installation process can be found ElasticSearch environment to build
cd /usr/local #root用户
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.6.2.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.6.2-linux-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.6.2-linux-x86_64.tar.gz
tar -zxvf logstash-6.6.2.tar.gz
chown es:es /usr/local/logstash-6.6.2/ -R
tar -zxvf kibana-6.6.2-linux-x86_64.tar.gz
chown es:es /usr/local/kibana-6.6.2-linux-x86_64/ -R
tar -zxvf filebeat-6.6.2-linux-x86_64.tar.gz
chown es:es /usr/local/filebeat-6.6.2-linux-x86_64/ -R
Configuration
elasticserach
See ElasticSearch environment to build
logstash
cd /usr/local/logstash-6.6.2 #es用户 vim default.conf #创建default.conf文件并添加内容: # 监听5044端口作为输入 input { beats { port => "5044" } } # 数据过滤 filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}" } } geoip { source => "clientip" } } # 输出配置为本机的9200端口,这是ElasticSerach服务的监听端口 output { elasticsearch { hosts => ["127.0.0.1:9200"] } }
kibana
cd /usr/local/kibana-6.6.2-linux-x86_64 #es用户 vim config/kibana.yml # 将server.host配置项取消注释并修改(若修改为局域网ip地址则内网访问,此处设为0位可外网访问) 为server.host:"0.0.0.0"
filebeat
cd /usr/local/filebeat-6.6.2-linux-x86_64 #es用户 vim filebeat.yml #配置日志以及日志文件路径(配置如截图,且需要保证es用户具有访问该log目录的权限),如nginx日志为例 vim filebeat.yml #配置elasticsearch日志输出地址或者logstash输出地址,在这里我们将采用filebeat先收集日志到logstash中,然后由logstash再到elasticsearch中,因此注释掉默认的elasticsearch地址并取消默认注释的logstash地址(配置如截图)
start up
elasticserach
See ElasticSearch environment to build
logstash
cd /usr/local/logstash-6.6.2 #es用户 nohup bin/logstash -f default.conf --config.reload.automatic & #以后台进程形式启动logstash服务 tail -f nohup.out #查看启动日志(或者先以bin/logstash -f default.conf -config.reload.automatic命令启动在控台打印启动日志确定可以成功关闭后再以后台形式启动)
kibana
cd /usr/local/kibana-6.6.2-linux-x86_64 #es用户 nohup bin/kibana & #以后台进程形式启动kibana服务 tail -f nohup.out #查看启动日志(或者先以bin/kibana命令启动在控台打印启动日志确定可以成功关闭后再以后台形式启动) 浏览器中访问 http://ip:5601 #可以访问页面成功,则kibana启动成功
shut down
kibana
filebeat
cd /usr/local/filebeat-6.6.2-linux-x86_64 #es用户 nohup ./filebeat -e -c filebeat.yml -d "publish" & #以后台进程形式启动filebeat服务 tail -f nohup.out #查看启动日志(或者先以./filebeat -e -c filebeat.yml -d "publish"命令启动在控台打印启动日志确定可以成功关闭后再以后台形式启动)
Nginx logs confirm the presence or absence of kibana
Browser to access http: // ip: whether there exist 5601 # nginx log in to view kibana
ALL:
- Kibana interface localization