Redis high performance reason analysis-blocking operation inside Redis and countermeasures

A very important reason why Redis is widely used is its high performance. Therefore, we must pay attention to all the factors, mechanisms and countermeasures that may affect Redis performance. The five potential factors that affect Redis performance are:

• Blocking operations inside Redis
• The impact of CPU core and NUMA architecture
• Redis key system configuration
• Redis memory fragmentation
• Redis buffer

In this article, we mainly understand the blocking operations inside Redis and how to deal with it.

What are the blocking points of Redis instances

Before analyzing the blocking point, let's take a look at the objects and operations that interact with the Redis instance:

• Client : network IO, key-value pair addition, deletion, modification, query operation, database operation;
• Disk : Generate RDB snapshots, record AOF logs, and rewrite AOF logs;
• Master-slave nodes : The master library generates and transmits RDB files, receives RDB files from the library, clears the database, and loads RDB files;
• Slice cluster instance : Transmit hash slot information to other instances and migrate data.

These interactive operations may create potential blocking points.

Choke point

• Set full query and aggregation operations : Pay attention to operations with O(n) complexity.
• Bigkey delete : The essence of the delete operation is to release the memory space occupied by the key-value pair. A large amount of memory is released at once, which will block the Redis main thread.
• Empty the database : Frequent deletion of key-value pairs is a potential risk, and emptying the database is also a potential risk.
• AOF log synchronous write : A synchronous disk write operation takes about 1~2ms. If a large number of write operations need to be recorded to the AOF log and written back synchronously, the main thread will be blocked.
• Load RDB files from the library : The larger the RDB file, the longer the blockage.

Which blocking operations can be performed asynchronously

Asynchronous execution requirements for operations: If an operation is executed asynchronously, it means that it is not an operation on the critical path of the Redis main thread .

Among the five choke points above, the critical path operations are:

• Collect all query and aggregation operations;
• Load RDB files from the library.

Asynchronous execution mechanism

Let's take a look at the asynchronous execution mechanism. The main thread interacts with the child threads through a task queue in the form of a linked list , as shown in the following figure:

After the main Redis thread is started, three child threads will be created to be responsible for the asynchronous execution of AOF log write operations, key-value pair deletion, and file closing.

Asynchronous key-value pair deletion and database emptying operations are functions provided after Redis 4.0. Redis also provides new commands to perform these two operations:

• Key-value pair deletion: When you have a large number of elements in your collection type (for example, millions or tens of millions of elements) need to be deleted, I suggest you use the UNLINK command.
• Empty the database: You can add the ASYNC option after the FLUSHDB and FLUSHALL commands, so that the background sub-thread can empty the database asynchronously. For example: FLUSHDB ASYNCand FLUSHALL AYSNC.

Suggest

• Collect full query and aggregation operations: you can use the SCAN command to read data in batches, and then perform aggregation calculations on the client;
• Load RDB files from the library: Control the data size of the main library to about 2~4GB to ensure that RDB files can be loaded at a faster speed.

---

This is the end of this article. If someone reads it, take time to write the next four articles.

Maybe the dry text seems to be boring to write. If the text alone is not easy to digest, you can join the group 973961276 to communicate and learn with everyone. There are also many video materials and technical experts in the group, and understanding with the article should make it easier to understand You have a good harvest.

Recommend a good c/c++ beginner course . This is different from what I have seen in the past that only talk about theory. This course starts with six enterprise-level projects that can be written on resumes and leads everyone to learn c/c++. Friends who are studying can find out.