redis performance monitoring and troubleshooting

redis bottleneck problems, and now some experience in the investigation of record for future reference herein, this just thinking of finishing, do not involve specific use.

The general idea is as follows:

1. Review by slow log

Reference http://www.cnblogs.com/onmyway20xx/p/5486604.html

　　Is there a more obvious slow query to view the next? Generally considered to occur slow query words, redis performance bottlenecks have been more obvious

2. Look through the info;

　　more info inside information, usually following a few

　　　Memory #　　　　
　　　　used_memory_human: 795.13K #redis now occupied by the memory, possibly including SWAP virtual memory.
　　　　used_memory_rss: 18259968 # redis system memory allocated to　　
　　　　used_memory_peak_human: 9.51M # Redis所用内存的峰值　
　　　　mem_fragmentation_ratio: 22.43 # used_memory_rss / used_memory, when mem_fragmentation_ratio <1 described used_memory> used_memory_rss,

　　　　Then Redis is already in use SWAP, operating performance will be greatly affected.

3. Performance testing in the current server by Benchmark;

4. The number of times a request operation by redis MONITOR calculation;

1. INFO

info command returns information server, comprising:

server: General information about the Redis server
clients: Client connections section
memory: Memory consumption related information
persistence: RDB and AOF related information
stats: General statistics
replication: Master/slave replication information
cpu: CPU consumption statistics
commandstats: Redis command statistics
cluster: Redis Cluster section
keyspace: Database related statistics

Back to list their own customized support:

[root@~]# redis-cli info
[root@~]# redis-cli info default [root@~]# redis-cli info all

For details, see: http://www.redis.cn/commands/info.html

2. MONITOR

MONITOR is a debugging command, each command returns processed by the server, it can help us understand what is happening operations on the database. There are three methods of operation:

[root@~]# redis-cli monitor
OK
1417532512.619715 [0 127.0.0.1:55043] "REPLCONF" "ACK" "6623624" [root@~]# telnet 127.0.0.1 6379 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. monitor +OK +1417532567.733458 [0 127.0.0.1:55043] "REPLCONF" "ACK" "6623708" +1417532568.735936 [0 127.0.0.1:55043] "REPLCONF" "ACK" "6623708" quit +OK Connection closed by foreign host. [root@~]# redis-cli 127.0.0.1:6379> monitor OK 1417532590.785487 [0 127.0.0.1:55043] "REPLCONF" "ACK" "6623736"

Because all of the MONITOR command returns the command server processing, so there will be some consumption on performance. Use the official pressure test tools test results are as follows

Without running MONITOR command, benchmark test results:

[root@~/software/redis-2.8.17]# src/redis-benchmark -c 10 -n 100000 -q PING_INLINE: 51020.41 requests per second PING_BULK: 50607.29 requests per second SET: 37257.82 requests per second GET: 49800.80 requests per second INCR: 38699.69 requests per second LPUSH: 38910.51 requests per second LPOP: 39277.30 requests per second SADD: 54614.96 requests per second SPOP: 51948.05 requests per second LPUSH (needed to benchmark LRANGE): 38819.88 requests per second LRANGE_100 (first 100 elements): 20112.63 requests per second LRANGE_300 (first 300 elements): 9025.27 requests per second LRANGE_500 (first 450 elements): 6836.67 requests per second LRANGE_600 (first 600 elements): 5406.28 requests per second MSET (10 keys): 19394.88 requests per second

In the case where the MONITOR command is run, the benchmark test results: (redis-cli monitor> / dev / null):

[root@~/software/redis-2.8.17]# src/redis-benchmark -c 10 -n 100000 -q PING_INLINE: 42211.91 requests per second PING_BULK: 42936.88 requests per second SET: 26143.79 requests per second GET: 33990.48 requests per second INCR: 26553.37 requests per second LPUSH: 27337.34 requests per second LPOP: 27225.70 requests per second SADD: 30459.95 requests per second SPOP: 39494.47 requests per second LPUSH (needed to benchmark LRANGE): 26315.79 requests per second LRANGE_100 (first 100 elements): 22055.58 requests per second LRANGE_300 (first 300 elements): 8104.38 requests per second LRANGE_500 (first 450 elements): 6371.05 requests per second LRANGE_600 (first 600 elements): 5031.95 requests per second MSET (10 keys): 14861.05 requests per second

We can see the basic indicators have declined.

For details, see: http://www.redis.cn/commands/monitor.html

3. SLOWLOG

Slow query log can be read by SLOWLOG.

Use SLOWLOG LEN can obtain the current slow log length.

[root@~/software/redis-2.8.17]# redis-cli 127.0.0.1:6379> slowlog len (integer) 28

Use SLOWLOG GET slow to get all the logs.

127.0.0.1:6379> slowlog get 1) 1) (integer) 27 2) (integer) 1417531320 3) (integer) 24623 4) 1) "info"

Among them, the indicators said:

A unique progressive identifier for every slow log entry.
The unix timestamp at which the logged command was processed.
The amount of time needed for its execution, in microseconds (note, translated into microseconds microseconds rather than milliseconds).
The array composing the arguments of the command.

Use SLOWLOG GET N can get the last N bars slow logs.

127.0.0.1:6379> slowlog get 2 1) 1) (integer) 27  2) (integer) 1417531320  3) (integer) 24623  4) 1) "info" 2) 1) (integer) 26  2) (integer) 1417528379  3) (integer) 21363  4) 1) "get"   2) "user:score"

Use slow SLOWLOG RESET command to reset the log. Once executed, the slow log all the previous loss.

127.0.0.1: 6379 > Re slowlog the SET 
3. Redis latency investigation

Recently more and more amount of data, concurrent write a lot of cases, the case of slow response Redis appear;

Redis commands can be used to test the response speed of redis:

 
   redis-cli --latency -h  
   xxx  
   -p  
   xxxx 
  

This command inserts the sample data to be checked Redis average delay. Ctrl + C to end the test at any time;

Let's look at the column may delay occur:

Hardware, system: hardware problems are all issues the bottom of the problem, if the hardware is slow, for example, low CPU clock speed, small memory, disk IO slow, which will make all the running in the above system response slow; In addition, the use of virtual opportunities the performance of the system is running too decline; of course, the money, this problem is easily solved; systems, resource scheduling system Linux itself will have some delay. These are generally not large, can be ignored;
Network: If the client and redis on the same server, using a socket connection is established will be much more than listening TCP / IP port fast;
Redis commands: Some time high complexity of commands, such as lrem, sort, sunion commands will take a very long time; in addition, a large number of repeated connections can also cause delay, reuse connection is a good quality; if there is a large number of write operations, use the pipeline pipeline manner (similar to mysql affairs), submit a one-time, so that less amount of data, the number of connections is also less, do not always return the data, the speed will naturally be much faster;
Persistence: Redis persistence need to fork out a process lasting operation, which in itself will lead to delay, if the data changes in large, RDB configuration time is short, and that this price is still getting bigger; plus, this thing HDD really a bit tricky, if a virtual hard disk or virtual machine, or if the NFS shared directory, that this delay will make you crash. So, if the system does not require persistence, shut it.
Related reference Address: https://blog.csdn.net/li123128/article/details/85531376
https://blog.csdn.net/xuyunti/article/details/84766936