table of Contents
fork consuming delay results in a high concurrent request
RDB and RDB AOF when there will be a snapshot generation, AOF rewrite, consuming disk IO process
When the main course fork child process, the child process is the need to copy space memory page tables of the parent process, it will also spend some time
In general, if the parent process a G data memory, then the fork may consume around 20ms, if 10G ~ 30G, it will cost 20 * 10, * 30 and even 20, which is a few hundred milliseconds
info stats in latest_fork_usec, you can see when the last fork length
redis QPS usually in the tens of thousands of single, fork may suddenly slow down request will be tens of thousands of a long operation, from a few milliseconds becomes 1 second
Optimization idea : fork main memory with the time-consuming process of redis a relationship, the general control within redis memory of 10GB; otherwise the slave -> master may be some problems in the full amount of replication time
AOF blocking issues
AOF redis write data to the buffer, a separate thread to do the opening operation fsync, once per second
But redis main thread checks twice fsync time, if more than two seconds from the last time fsync, then the write request will block
everysec, the loss of up to 2 seconds of data
Once fsync delay more than 2 seconds, the whole was slow redis
Optimization idea : to optimize hard disk write speed, recommended SSD, do not use ordinary mechanical hard drives, SSD greatly enhance the speed of disk reads and writes
Master copy from latency problems
Master-slave replication may time out serious, this time the need for good monitoring and alarm system
In the info replication, you can see offset master and slave replication, to make a difference you can see the delay amount, if the delay is too much, then the alarm (you can write a shell script to monitor)
Master-slave replication storm problem
If all of a sudden to have multiple slave from master to perform the full amount of copy, a large rdb simultaneously sent to multiple slave, can lead to serious network bandwidth is occupied
If you really want to mount a master multiple slave, then try to use the tree, do not use the star structure
Tree, meaning to say, let a slave node following not too much, you can go to configure the way through replication
If there should not have this problem in redis cluster in
vm.overcommit_memory
This information is redis warning messages when activated, to optimize the performance of these warnings can be achieved by adjusting the linux kernel configuration
[root@eshop-cache03 ~]# cat /var/log/redis/7008.log
1418:M 24 Mar 13:10:59.513 * Increased maximum number of open files to 10032 (it was originally set to 1024).
1418:M 24 Mar 13:10:59.513 # Warning: 32 bit instance detected but no memory limit set. Setting 3 GB maxmemory limit with 'noeviction' policy now.
1418:M 24 Mar 13:10:59.513 * No cluster configuration found, I'm 728e473d6e5e36ddb051c600c7708f23733c46f7
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.8 (00000000/0) 32 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in cluster mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 7008
| `-._ `._ / _.-' | PID: 1418
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
1418:M 24 Mar 13:10:59.626 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1418:M 24 Mar 13:10:59.626 # Server started, Redis version 3.2.8
1418:M 24 Mar 13:10:59.626 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
- 0: Check that there is not enough memory, if not for memory failure
- 1: Allows memory until used up
- 2: memory address space can not exceed swap + 50%
If it is 0, it may lead to failure of the operation performs a similar fork, etc., are not eligible enough memory space
The following command in the log which has prompted out
cat /proc/sys/vm/overcommit_memory
echo "vm.overcommit_memory=1" >> /etc/sysctl.conf
sysctl vm.overcommit_memory=1swapiness
# 查看 linux 内核版本
cat /proc/versionIf the linux kernel version <3.5, then swapiness set to 0, so that the system would rather swap will not oom killer (kill the process)
If the linux kernel version> = 3.5, then swapiness set to 1, so the system will not oom killer would rather swap
Redis guarantee will not be killed
echo 0 > /proc/sys/vm/swappiness
echo vm.swapiness=0 >> /etc/sysctl.confThe maximum open file handle
Increased maximum number of open files to 10032 (it was originally set to 1024).# 如果该命令不可用，可以去百度搜索不同的版本命令
ulimit -n 10032 10032Internet search about themselves, the way different operating systems, versions are not the same set of
tcp backlog
WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.cat /proc/sys/net/core/somaxconn
echo 511 > /proc/sys/net/core/somaxconnreference
- Chinese Shi Shan: one hundred million electricity supplier details page flow combat system (Second Edition): + high availability cache architecture service architecture + micro Services Architecture
- Mrcode notebook