foreword
As an operation and maintenance, you need to use the monitoring system to check the server status and website traffic indicators, and use the data of the monitoring system to understand the results of the online release and the health status of the website. With an excellent monitoring software, we can:
- Browse all server status of the entire website through a friendly interface;
- The monitoring data can be easily viewed on the web front end;
- It is possible to look back to find the system problems and alarm conditions when the accident occurred.
Table of contents
2. Zabbix monitoring principle
1. Use official or developed templates
(1) Monitor the number of client host logins
(2) Monitor nginx client status information
2. Specify the alarm medium in the user settings
3. Specify the monitoring trigger for the alarm
Six, configure zabbix server automatic discovery and automatic registration
1. Overview of zabbix
1. What is zabbix?
Zabbix is an enterprise-level open source solution that provides distributed system monitoring and network monitoring functions based on a web interface.
Zabbix can monitor various network parameters, ensure the safe operation of the server system, and provide a flexible notification mechanism to allow system administrators to quickly locate and solve various problems.
zabbix consists of two parts, zabbix server and optional component zabbix agent. Collect data through the c/s mode, and display and configure it on the web side through the b/s mode.
zabbix server can provide remote server/network status monitoring, data collection and other functions through SNMP, zabbix agent, ping, port monitoring and other methods, and it can run on platforms such as Linux.
The zabbix agent needs to be installed on the monitored target server. It mainly completes the collection of hardware information or memory, CPU and other information related to the operating system.
2. Zabbix monitoring principle
The zabix agent is installed on the monitored host. The zabiboix agent is responsible for collecting the local data of the client regularly and sending it to the zabbix server. After the zabbix server receives the data, it stores the data in the database, and the user can see it based on Zabbix WEB Data presents images on the front end. When zabbix monitors a specific project, the project will set a trigger threshold. When the monitored indicator exceeds the threshold set by the trigger, some necessary actions will be taken, including sending information (email, WeChat, SMS) , Send commands (shell commands, reboot, restart, install, etc.).
3. Monitoring object
Zabbix monitoring is deployed in the system, including five common programs: zabbix_server, zabix_agent, zabbix proxy, zabbix_get, zabix_sender, etc.
（1）zabbix server
The zabbix server daemon process, in which the data of zabbix_agent, zabbix_get, zabbix_sender, and zabbix _proxy are finally submitted to the zabbix server;
（2）zabbix agent
The client daemon process is responsible for collecting client data, for example: collecting CPU load, memory, hard disk usage, etc.;
（3）zabbix proxy
The zabbix distributed agent daemon process, usually larger than 500 hosts, requires distributed monitoring architecture deployment;
（4）zabbix get
zabbix data receiving tool, a command used alone, usually executes the command to obtain remote client information on the server or proxy side;
（5）zabbix sender
Zabbix data sending tool, the user sends data to the server or proxy end, usually the user takes a long time to check.
2. Deploy the zabbix server
The zabbix server zabbix_server uses port 10051 by default
zabbix_server: 192.168.116.60 (at least 2G memory, 4G recommended)
1. Turn off the firewall and selinux, set the server name, and synchronize the time
systemctl disable --now firewalld
setenforce 0
hostnamectl set-hostname zbx-server
su
ntpdate ntp.aliyun.com2. Download zabbix and related software
#获取 zabbix 的下载源
rpm -ivh https://mirrors.aliyun.com/zabbix/zabbix/5.0/rhel/7/x86_64/zabbix-release-5.0-1.el7.noarch.rpm
#更换 zabbix.repo 为阿里源
cd /etc/yum.repos.d
sed -i 's#http://repo.zabbix.com#https://mirrors.aliyun.com/zabbix#' zabbix.repo
yum clean all && yum makecache
yum install -y zabbix-server-mysql zabbix-agent
#安装SCL(Software Collections)，便于后续安装高版本的 php，默认 yum 安装的 php 版本为 5.4，版本过低，zabbix 5.0 版本对 php 版本最低要 7.2.0 版本。SCL 可以使得在同一台机器上使用多个版本的软件，而又不会影响整个系统的依赖环境。软件包会安装在 /etc/opt/rh/ 目录下。
yum install -y centos-release-scl You also need to modify the zabbix source and start the front-end environment download
#安装前端环境
yum install -y zabbix-web-mysql-scl zabbix-apache-conf-scl
#安装 zabbix 所需的数据库
yum install -y mariadb-server mariadb
systemctl enable --now mariadb
#初始化数据库
mysql_secure_installation
#导入数据库信息
zcat /usr/share/doc/zabbix-server-mysql-5.0.15/create.sql.gz | mysql -uroot -pabc123 zabbix
3. Modify the zabbix server configuration file and modify the database password
vim /etc/zabbix/zabbix_server.conf
......
DBPassword=zabbix #124行，指定 zabbix 数据库的密码4. Modify the php configuration file of zabbix
vim /etc/opt/rh/rh-php72/php-fpm.d/zabbix.conf
......
php_value[date.timezone] = Asia/Shanghai #24行，取消注释，修改时区5. Start zabbix related services
systemctl restart zabbix-server zabbix-agent httpd rh-php72-php-fpm
systemctl enable zabbix-server zabbix-agent httpd rh-php72-php-fpm6. Browser access to zabbix: http://192.168.116.60/zabbix
You can click on the host to view the graphical data, but there will be Chinese garbled characters, and you need to import the character set
#解决 zabbix-server Web页面中文乱码问题
yum install -y wqy-microhei-fonts
\cp -f /usr/share/fonts/wqy-microhei/wqy-microhei.ttc /usr/share/fonts/dejavu/DejaVuSans.ttf
3. Deploy the zabbix client
The zabbix 5.0 version uses a new version of the client agent2 developed in the golang language.
The client zabbix_agent2 uses port 10050 by default.zabbix_agent01：192.168.116.70
1. Turn off the firewall and selinux, set the server name, and synchronize the time
systemctl disable --now firewalld
setenforce 0
hostnamectl set-hostname zbx-agent01
su
ntpdate ntp.aliyun.com2. Download the zabbix client
#设置 zabbix 的下载源，安装 zabbix-agent2
rpm -ivh https://mirrors.aliyun.com/zabbix/zabbix/5.0/rhel/7/x86_64/zabbix-release-5.0-1.el7.noarch.rpm
cd /etc/yum.repos.d
sed -i 's#http://repo.zabbix.com#https://mirrors.aliyun.com/zabbix#' /etc/yum.repos.d/zabbix.repo
yum install -y zabbix-agent2
3. Modify the client configuration file and open it
#修改 agent2 配置文件
vim /etc/zabbix/zabbix_agent2.conf
......
Server=192.168.116.60 #80行，指定 zabbix 服务端的 IP 地址
ServerActive=192.168.116.60 #120行，指定 zabbix 服务端的 IP 地址
Hostname=zbx-agent01 #131行，指定当前 zabbix 客户端的主机名
#开启服务
systemctl enable --now zabbix-agent2.service
4. Verify the connectivity of zabbix-agent2 on the server side
#安装 zabbix 主动获取数据的命令
yum install -y zabbix-get
#测试
zabbix_get -s '192.168.116.70' -p 10050 -k 'agent.ping'
zabbix_get -s '192.168.116.70' -p 10050 -k 'system.hostname'
5. Create a new client
4. Monitoring template
1. Use official or developed templates
Browse Zabbix / Zabbix - ZABBIX GIT
Zabbix searcher - 900+ Zabbix projects: templates, scripts, integrations, ...
2. Custom monitoring template
(1) Monitor the number of client host logins
First, customize the key on the client (here is used to monitor the number of host logins, the threshold is 3, and the alarm is greater than 3)
Create a new .conf file under /etc/zabbix/zabbix_agent2.d/, customize the key in it, and the format is
UserParameter=<key>,<commend>
Then restart the service to read the file configuration systemctl restart zabbix-agent2.service
You can use the command to test on the server side (the current login number is 4)
After the test is correct, you can add a custom template
Create a template, find and configure it
Create configuration application set
Create configuration triggers
Create configuration graphics
Add the template to the client
Check the graph (the number of logins is monitored)
(2) Monitor nginx client status information
The client installs nginx and simulates monitoring of the nginx server
vim /etc/yum.repos.d/nginx.repo
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0
enabled=1
yum install nginx -y
Modify nginx configuration file /etc/nginx/conf.d/default.conf, open status statistics
Restart the service and view the status statistics page
systemctl start nginx
curl -s 127.0.0.1/status
Write a script to monitor nginx status information, obtain the above values, and give execution permission
#/bin/bash
#Description:Automated monitoring nginx performance and process nginx_status scripts
HOST="localhost"
PORT=80
URL="status"
NGINX_COMMAND=$1
#检查nginx状态
nginx_check() {
if [ -f /sbin/pidof ]; then
/sbin/pidof nginx | wc -w
else
ps aux | grep -v "grep" | grep -c "nginx:"
fi
}
#nginx状态相关信息的值
nginx_active(){
/usr/bin/curl -s "http://$HOST:$PORT/${URL}/" 2> /dev/null | awk '/Active/ {print $NF}'
}
nginx_reading(){
/usr/bin/curl -s "http://$HOST:$PORT/${URL}/" 2> /dev/null | awk '/Reading/ {print $2}'
}
nginx_writing(){
/usr/bin/curl -s "http://$HOST:$PORT/${URL}/" 2> /dev/null | awk '/Writing/ {print $4}'
}
nginx_waiting(){
/usr/bin/curl -s "http://$HOST:$PORT/${URL}/" 2> /dev/null | awk '/Waiting/ {print $6}'
}
nginx_accepts(){
/usr/bin/curl -s "http://$HOST:$PORT/${URL}/" 2> /dev/null | awk 'NR==3 {print $1}'
}
nginx_handled(){
/usr/bin/curl -s "http://$HOST:$PORT/${URL}/" 2> /dev/null | awk 'NR==3 {print $2}'
}
nginx_requests(){
/usr/bin/curl -s "http://$HOST:$PORT/${URL}/" 2> /dev/null | awk 'NR==3 {print $3}'
}
#命令获取对应值
case $NGINX_COMMAND in
check)
nginx_check
;;
active)
nginx_active
;;
reading)
nginx_reading
;;
writing)
nginx_writing
;;
waiting)
nginx_waiting
;;
accepts)
nginx_accepts
;;
handled)
nginx_handled
;;
requests)
nginx_requests
;;
*)
echo $"USAGE:$0 {check|active|reading|writing|waiting|accepts|handled|requests}"
esac
Script test is correct
Custom monitoring variables for use in templates
UserParameter=nginx.status[*],/etc/zabbix/zabbix_shell/zabbix_nginx.sh $1
#[*] 代表自定义变量nginx.status传入的值
#$1 代表前面变量传入的值作为此脚本执行的参数，获取对应状态值Restart the service systemctl restart zabbix-agent2.service to test on the server side
The test is correct, go to the front end to customize the template
Five, configure zabbix alarm
1. Set the alarm medium
can be tested
2. Specify the alarm medium in the user settings
3. Specify the monitoring trigger for the alarm
4. Test alarm
One of the alarm triggers we set before is that when the nginx process is 0, an alarm is triggered, so we can simulate shutting down nginx to see if we can receive an alarm from zabbix through the set alarm medium.
When we restore the problem, the problem will disappear
Six, configure zabbix server automatic discovery and automatic registration
1. Automatic discovery
zabbix automatic discovery (passive mode for agent2) is that zabbix server actively discovers all clients, and then registers the client information on the server. The disadvantage is that if the number of hosts in the defined network segment is large, the registration of zabbix server will take a long time and the pressure will be high.
First add a client, install the zabbix-agent2 service, modify the configuration file to specify the server address and client name, and then start the service
Both the server and the client need to configure each other's domain name resolution
2. Automatic registration
zabbix automatically registers (active mode for agent2), and zabbix agent2 will actively report its own information and send it to zabbix server. The disadvantage is that zabbix agent2 may not be able to find the zabbix server due to incorrect configuration of the configuration file or network failure.
Disable automatic discovery first, delete client 02, and do automatic registration
Turn on auto-registration in the client configuration file, then restart the service
Add auto-registration action
7. Deploy zabbix proxy server
Configure the zabbix proxy to share the centralized pressure of the server and solve the network delay problem between multiple computer rooms.
First turn off the firewall, modify the host name, and install the proxy
systemctl disable --now firewalld
setenforce 0
hostnamectl set-hostname zbx-proxy
rpm -ivh https://mirrors.aliyun.com/zabbix/zabbix/5.0/rhel/7/x86_64/zabbix-release-5.0-1.el7.noarch.rpm
cd /etc/yum.repos.d
sed -i 's#http://repo.zabbix.com#https://mirrors.aliyun.com/zabbix#' /etc/yum.repos.d/zabbix.repo
yum install -y zabbix-proxy-mysql zabbix-getInstall the database required by the agent, do initialization, and set the password to 123123
yum install -y mariadb-server mariadb
systemctl enable --now mariadb
mysql_secure_installationAdd database and user, authorize, import database table information
mysql -uroot -p123123
CREATE DATABASE zabbix_proxy character set utf8 collate utf8_bin;
GRANT all ON zabbix_proxy.* TO 'zabbix'@'%' IDENTIFIED BY 'zabbix';
flush privileges;
quit
#退出后导入表数据
zcat /usr/share/doc/zabbix-proxy-mysql-5.0.29/schema.sql.gz | mysql -uroot -p123123 zabbix_proxy
Modify the proxy configuration file and start the service
vim /etc/zabbix/zabbix_proxy.conf
Server=192.168.116.60 #30行，指定 zabbix 服务端的 IP 地址
Hostname=zbx-proxy #49行，指定当前 zabbix 代理服务器的主机名
DBPassword=zabbix #196行，指定当前数据库 zabbix 用户的密码
systemctl start zabbix-proxy
Modify the agent to point to the address of the agent
vim /etc/zabbix/zabbix_agent2.conf
......
Server=192.168.116.40 #80行，指定 zabbix 代理服务器的 IP 地址
ServerActive=192.168.116.40 #120行，指定 zabbix 代理服务器的 IP 地址
It is also necessary to add domain name resolution to each host
Then go to the server to specify the proxy
Add proxy when creating client
Then restart the server, proxy, and client services in turn
Wait for all clients to light up
Eight, SNMP monitoring
Use SNMP simple network management protocol to manage and monitor resources in the network in real time.
First install snmp on the routers and switches that need to be monitored (here take the server as an example)
yum install -y net-snmp net-snmp-utils.x86_64
systemctl start snmpd.service
Then go to the front end to modify and use snmp
