ELK installation configuration, monitoring nginx logs, White learned, only to make a record. What principle is unclear. Through the process! To facilitate future review it!
1, ready to work
a) turn off the firewall
Turn off the firewall: service iptables stop
Permanently turn off the firewall: chkconfig iptables off
Check firewall status: service iptables status
b) Close SELinux
Permanent: Modify / etc / sysconfig / selinux
The text SELINUX = enforcing, to SELINUX = disabled. Then restart
Effective immediately: setenforce 0
View Status: getenforce
Both machines
The machine 1, IP = 192.168.10.128 installation deployment elk
Machine 2, IP = 192.168.10.129 installation and deployment ngix filebeat
Mounting the deployment machine 1, (ELK)
For installation package
mkdir / Elk; cd / Elk
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.2.3.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/ -6.2.3-Linux-kibana x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.3.tar.gz
And extract copied to the / usr / local / directory
CD / Elk
the tar-6.2.3.tar.gz XF elasticsearch
the tar-6.2.3.tar.gz -xf logstash
the tar--xf kibana the x86_64-6.2.3-Linux. tar.gz
cp -a elasticsearch-6.2.3 /usr/local/
cp -a logstash-6.2.3 /usr/local/
cp -a kibana-6.2.3-linux-x86_64 /usr/local/
yum -y install java-1.8 * # elasticsearch based on the latest version jdk1.8, you need to install jdk [source yum local configuration there can be used directly]
[Configuration] elasticsearch
useradd elasticsearch # elasticsearch create user
chown -R elasticsearch.elasticsearch /usr/local/elasticsearch-6.2.3/
su - elasticsearch # elasticsearch users need to switch to start the service
cd /usr/local/elasticsearch-6.2.3/
./bin/elasticsearch -d # to start the service
Check whether the success of the process (need to wait for it)
netstat -antp
9200 port
If an error occurs, you can view the log
cat /usr/local/elasticsearch-6.2.3/logs/elasticsearch.log
Whether the test can access
curl localhost: 9200
[Configuration] logstash
Note Cut back to the root user
vim /usr/local/logstash-6.2.3/vendor/bundle/jruby/2.3.0/gems/logstash-patterns-core-4.1.2/patterns/grok-patterns # Add filtering configuration nginx , grok the use of filter plug-in log analysis (written here is a regular, direct copy, and no specific studies)
#Nginx log
WZ([^]*)
NGINXACCESS %{IP:remote_jp} \- \- \[%{HTTPDATE:timestamp}\] "%{WORD:method}% {WZ:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:status} %{NUMBER:bytes} %{QS:referer} %{QS:agent} %{QS:xforward}
Creating logstash configuration file
[root @ server local] # vim /usr/local/logstash-6.2.3/default.conf # here need to focus on the next path, back to start the service need to call this path
{INPUT
Beats {
Port => "5044"
}
}
# filtering data
filter {
Grok {
match => { "Message" => "% NGINXACCESS {}"}
}
GeoIP {
# Nginx client IP
Source => "192.168.10.129 "
}
}
# output port configured in native 9200, which is listening service port elasticSearch
output {
elasticsearch {
the hosts => [" 127.0.0.1:9200 "]
}
}
Start logstash service
cd /usr/local/logstash-6.2.3/
nohup ./bin/logstash -f default.conf & # calls default.conf configuration file, when the path is not set, the default path in which the command execution, otherwise it will reported the following error
[2019-06-14T15:51:21,543][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/usr/local/logstash-6.2.3/bin/default.conf"}
Check the service is started
netstat -natp | grep 5044
[Kibana arrangement]
Vim /Usr/local/kibana-6.2.3-Linux-x86_64/config/kibana.Yml
Cd /Usr/local/kibana-6.2.3-Linux-x86_64/
Nohup Bin / kibana Ando
netstat -natp | grep 5601
kibana page
Machine 2, IP = 192.168.10.129 installation and deployment ngix filebeat
[Client Configuration] nginx
yum -y install nginx
nginx # yum own source, can not download
You can use the following configuration
cd /etc/yum.repos.d/
vim nginx.repo
netstat -natp | grep 5601
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1
yum install nginx -y
Start nginx, and set the boot from the start
service nginx start
chkconfig nginx on
chkconfig --list nginx
nginx的页面
【filebeat】下载filebeat并解压到/usr/local路径下
wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.2.3-linux-x86_64.tar.gz
tar -xf ./filebeat-6.2.3-linux-x86_64.tar.gz -C /usr/local/
配置filebeat
vim /usr/local/filebeat-6.2.3-linux-x86_64/filebeat.yml
24 enabled: false #修改为true
27 paths:
28 - /var/log/*.log #修改为/var/log/nginx/*.log
143 #output.elasticsearch:
144 # Array of hosts to connect to. #注释掉，关闭elasticsearch收集
145 # hosts: ["localhost:9200"]
153 #output.logstash:
154 # The Logstash hosts
155 #hosts: ["127.0.0.1:5044"] #取消注释，将logstash开启收集，并将IP修改为ELK服务器的地址
启动filebeat
cd /usr/local/filebeat-6.2.3-linux-x86_64
nohup ./filebeat -e -c filebeat.yml & #注意路径，这里选用相对路径，并且注意调用的配置文件路径(filebeat.yml)
以上步骤完成后，多刷新几次nginx页面，稍后在kibana页面上就可以看到对应的日志信息
目前，仅限安装了，其他深入的东东，还没有了解。先记录下，以便以后使用吧！