Da's house zabbix server has been operating normally, one day suddenly found zabbix-server service will not start, and did not move any configuration, logs the problem of the investigation process
1、pid file zabbix_server.pid not readable (yet?) after start
Symptoms:
By looking at the system command systemctl status zabbix-server and journalctl -u zabbix-server -f discovery has prompted the
Error :: pid file /run/zabbix/zabbix_server.pid not readable ( yet?) After start
problem solved:
By /etc/systemd/system/zabbix-server.service deleted file pid this line, to solve this problem (the configuration file points to /var/run/zabbix_server.pid, the actual running / etc / zabbix / zabbix-server .conf file inconsistencies, but the problem did not appear before)
After resolving a complete discovery, or get up, but this error changed out
2、 cannot create semaphore set: [28] No space left on device
Symptoms:
[root@mafei zabbix]# tail -n 10 /var/log/zabbix/zabbix_server.log
zabbix_server [12330]: cannot open log: cannot create semaphore set: [28] No space left on device
zabbix_server [12939]: cannot open log: cannot create semaphore set: [28] No space left on device
zabbix_server [13662]: cannot open log: cannot create semaphore set: [28] No space left on device
problem solved
Guess the real question should be monitored because of the increased entry, the system limit is too low cause of where the problem began to change the
CAT / proc / SYS / Kernel / sem
250 32000 32 128
Meaning kernel.sem 4 argument
250 SEMMSL max semaphores per array signal sets the maximum number of signals received
32000 SEMMNS max semaphores system wide maximum number of all signals
32 SEMOPM max ops per semop call signal calling single maximum number of signals focused
maximum 128 SEMMNI max number of arrays set of signals
And then attempt to set parameters kernel.sem bigger
/etc/sysctl.conf Vim
fs.file-max = 65535
kernel.sem = 64 256 500 64000 //kernel.sem this according to the needs in a suitable adjustment larger
After modifying still try to restart zabbix-server, nothing else really is get up, this time an error has changed. . . .
3、zbx_mem_malloc(): out of memory
Symptoms:
/Var/log/zabbix/zabbix-server.log appeared in print in a large number of out of memory and other memory usage:
zbx_mem_malloc (): out of memory
### to solve the problem
here is in fact made a second error is resolved to limit the operating system level, but zabbix itself is limited, the default cache is 8M, monitored item more than one, this value is not enough
being given the already very clearly suggests Remedy: please increase CacheSize configuration parameter
so we went zabbix_server.conf find CacheSize field, the default is 8M
/etc/zabbix/zabbix_server.conf
# CacheSize = 8M
I am here to change the 2G
CacheSize = 2048M
Zabbix-server restart after saving, the perfect solution