As a memory-based caching system, Redis has always been known for its high performance. Because there is no context switching and lock-free operation, even in the case of single-threaded processing, the read speed can still reach 110,000 times/s, and the write speed can reach 81,000 times/ s. However, the single-threaded design also brings some problems to Redis:
-
Only one CPU core can be used;
-
If the deleted key is too large (for example, there are millions of objects in the Set type), it will cause the server to block for several seconds;
-
QPS is difficult to improve.
For the above problem, Redis in version 4.0 and version 6.0 are introduced Lazy Freeas well as 多线程IOa gradual transition to a multi-threaded, the following will be described in detail.
Single thread principle
It is said that Redis is single-threaded, so how does single-threading reflect? How to support concurrent client requests? In order to clarify these issues, let's first understand how Redis works.
The Redis server is an event-driven program, and the server needs to handle the following two types of events:
-
文件事件: The Redis server connects with the client (or other Redis server) through a socket, and the file event is the abstraction of the server's socket operation; the communication between the server and the client will generate the corresponding file event, and the server will monitor and connect processing these events to complete a series of network communication operations, such as connectingaccept,read,write,closeand the like; -
时间事件: Some operations in the Redis server (such as the serverCron function) need to be executed at a given point in time, and the time event is the server's abstraction of such timing operations, such as expired key cleaning, service status statistics, etc.
Event scheduling
As shown in the figure above, Redis abstracts file events and time events. The time trainer will monitor the I/O event table. Once a file event is ready, Redis will give priority to the file event and then the time event. In all the above event processing, Redis is 单线程processed in form, so Redis is single-threaded. In addition, as shown in the figure below, Redis has developed its own I/O event processor based on the Reactor model, that is, the file event processor. Redis uses I/O multiplexing technology for I/O event processing, and monitors multiple channels at the same time. There are two sockets, and different event processing functions are associated with the sockets, and the concurrent processing of multiple clients is realized through one thread.
Multiplexer
Because of this design, the locking operation is avoided in data processing, which makes the implementation simple enough and guarantees its high performance. Of course, Redis single-thread only refers to its event processing. In fact, Redis is not single-threaded. For example, when an RDB file is generated, it will fork a child process to achieve it. Of course, this is not the content of this article.
Lazy Free mechanism
As known above, Redis runs in a single-threaded manner when processing client commands, and the processing speed is very fast, during which time it will not respond to other client requests, but if the client sends a long command to Redis, such as delete For a Set key containing millions of objects, or to perform flushdb or flushall operations, the Redis server needs to reclaim a large amount of memory space, causing the server to be stuck for several seconds, which will be a disaster for a cache system with a high load. In order to solve this problem, Redis 4.0 was introduced to Lazy Freemake it 慢操作asynchronous, which is also a step towards multithreading in event processing.
As the author stated in his blog, to solve the problem 慢操作, you can use progressive processing, that is, add a time event, for example, when deleting a Set key with millions of objects, only part of the data in the big key is deleted each time. Finally realize the deletion of the big key. However, this solution may cause the recovery speed to not keep up with the creation speed, and eventually lead to memory exhaustion. Therefore, the final implementation of Redis is to asynchronousize the delete operation of the big key, using non-blocking delete (corresponding command UNLINK), and the space recovery of the big key is implemented by a separate thread. The main thread only releases the relationship and can quickly return to continue processing other Event, to avoid server blocking for a long time.
Take delete ( DELcommand) as an example to see how Redis is implemented. The following is the entry of the delete function, among which lazyfree_lazy_user_delis whether to modify DELthe default behavior of the command. Once it is turned on, DELit will UNLINKbe executed in a form when it is executed.
void delCommand(client *c) {
delGenericCommand(c,server.lazyfree_lazy_user_del);
}
/* This command implements DEL and LAZYDEL. */
void delGenericCommand(client *c, int lazy) {
int numdel = 0, j;
for (j = 1; j < c->argc; j++) {
expireIfNeeded(c->db,c->argv[j]);
// 根据配置确定DEL在执行时是否以lazy形式执行
int deleted = lazy ? dbAsyncDelete(c->db,c->argv[j]) :
dbSyncDelete(c->db,c->argv[j]);
if (deleted) {
signalModifiedKey(c,c->db,c->argv[j]);
notifyKeyspaceEvent(NOTIFY_GENERIC,
"del",c->argv[j],c->db->id);
server.dirty++;
numdel++;
}
}
addReplyLongLong(c,numdel);
}
Synchronous deletion is very simple, as long as the key and value are deleted, if there is an inner reference, it will be deleted recursively, which will not be introduced here. Let's look at asynchronous deletion. When Redis reclaims objects, it will first calculate the recycling revenue. Only when the recycling revenue exceeds a certain value, it will be encapsulated as a Job and added to the asynchronous processing queue, otherwise it will be directly recycled synchronously, which is more efficient. The calculation of recovery income is also very simple. For example, for Stringtype, the value of recovery income is 1, and for Settype, recovery income is the number of elements in the set.
/* Delete a key, value, and associated expiration entry if any, from the DB.
* If there are enough allocations to free the value object may be put into
* a lazy free list instead of being freed synchronously. The lazy free list
* will be reclaimed in a different bio.c thread. */
#define LAZYFREE_THRESHOLD 64
int dbAsyncDelete(redisDb *db, robj *key) {
/* Deleting an entry from the expires dict will not free the sds of
* the key, because it is shared with the main dictionary. */
if (dictSize(db->expires) > 0) dictDelete(db->expires,key->ptr);
/* If the value is composed of a few allocations, to free in a lazy way
* is actually just slower... So under a certain limit we just free
* the object synchronously. */
dictEntry *de = dictUnlink(db->dict,key->ptr);
if (de) {
robj *val = dictGetVal(de);
// 计算value的回收收益
size_t free_effort = lazyfreeGetFreeEffort(val);
/* If releasing the object is too much work, do it in the background
* by adding the object to the lazy free list.
* Note that if the object is shared, to reclaim it now it is not
* possible. This rarely happens, however sometimes the implementation
* of parts of the Redis core may call incrRefCount() to protect
* objects, and then call dbDelete(). In this case we'll fall
* through and reach the dictFreeUnlinkedEntry() call, that will be
* equivalent to just calling decrRefCount(). */
// 只有回收收益超过一定值，才会执行异步删除，否则还是会退化到同步删除
if (free_effort > LAZYFREE_THRESHOLD && val->refcount == 1) {
atomicIncr(lazyfree_objects,1);
bioCreateBackgroundJob(BIO_LAZY_FREE,val,NULL,NULL);
dictSetVal(db->dict,de,NULL);
}
}
/* Release the key-val pair, or just the key if we set the val
* field to NULL in order to lazy free it later. */
if (de) {
dictFreeUnlinkedEntry(db->dict,de);
if (server.cluster_enabled) slotToKeyDel(key->ptr);
return 1;
} else {
return 0;
}
}
Through the introduction a threaded lazy free, Redis realizes the Slow Operationcorresponding Lazyoperation, avoiding the server blocking when the big key is deleted, FLUSHALL, FLUSHDB. Of course, when implementing this function, not only the lazy freethread is introduced , but also the storage structure of the Redis aggregation type is improved. Because Redis uses many shared objects internally, such as client output cache. Of course, Redis does not use locking to avoid thread conflicts. Lock competition will cause performance degradation. Instead, shared objects are removed and data copy is used directly. As follows, ZSetdifferent implementations of node value in 3.x and 6.x.
// 3.2.5版本ZSet节点实现，value定义robj *obj
/* ZSETs use a specialized version of Skiplists */
typedef struct zskiplistNode {
robj *obj;
double score;
struct zskiplistNode *backward;
struct zskiplistLevel {
struct zskiplistNode *forward;
unsigned int span;
} level[];
} zskiplistNode;
// 6.0.10版本ZSet节点实现，value定义为sds ele
/* ZSETs use a specialized version of Skiplists */
typedef struct zskiplistNode {
sds ele;
double score;
struct zskiplistNode *backward;
struct zskiplistLevel {
struct zskiplistNode *forward;
unsigned long span;
} level[];
} zskiplistNode;
The removal of shared objects not only realizes the lazy freefunction, but also brings the possibility for Redis to move into multithreading, as the author said:
Now that values of aggregated data types are fully unshared, and client output buffers don’t contain shared objects as well, there is a lot to exploit. For example it is finally possible to implement threaded I/O in Redis, so that different clients are served by different threads. This means that we’ll have a global lock only when accessing the database, but the clients read/write syscalls and even the parsing of the command the client is sending, can happen in different threads.
Multithreaded I/O and its limitations
Redis was introduced in version 4.0 Lazy Free. Since then, Redis has a Lazy Freethread dedicated to the recovery of large keys, and at the same time, it has also removed the aggregate type of shared objects, which brings the possibility of multi-threading. Redis does not live up to expectations. It has been implemented in version 6.0. 多线程I/O.
Realization principle
As the official replies in the past, the performance bottleneck of Redis is not on the CPU, but on the memory and the network. Therefore, the multithreading released in 6.0 does not change the event processing to multithreading, but on I/O. In addition, if the event processing is changed to multithreading, not only will it lead to lock competition, but also frequent context switching, namely Using segmented locks to reduce competition will also have major changes to the Redis kernel, and performance may not necessarily be significantly improved.
Multi-threaded IO implementation
The red part in the above figure is the multi-threaded part implemented by Redis, which uses multiple cores to share the I/O read and write load. In 事件处理线程each time to get readable event, we will be ready for all events assigned to read I/O线程, and wait at all I/O线程after the completion of the read operation, 事件处理线程started tasking, after the end of treatment, the same event is assigned to write I/O线程, wait for all I/OThe thread completes the write operation.
Taking read event processing as an example, look at the 事件处理线程task allocation process:
int handleClientsWithPendingReadsUsingThreads(void) {
...
/* Distribute the clients across N different lists. */
listIter li;
listNode *ln;
listRewind(server.clients_pending_read,&li);
int item_id = 0;
// 将等待处理的客户端分配给I/O线程
while((ln = listNext(&li))) {
client *c = listNodeValue(ln);
int target_id = item_id % server.io_threads_num;
listAddNodeTail(io_threads_list[target_id],c);
item_id++;
}
...
/* Wait for all the other threads to end their work. */
// 轮训等待所有I/O线程处理完
while(1) {
unsigned long pending = 0;
for (int j = 1; j < server.io_threads_num; j++)
pending += io_threads_pending[j];
if (pending == 0) break;
}
...
return processed;
}
I/O线程Processing flow:
void *IOThreadMain(void *myid) {
...
while(1) {
...
// I/O线程执行读写操作
while((ln = listNext(&li))) {
client *c = listNodeValue(ln);
// io_threads_op判断是读还是写事件
if (io_threads_op == IO_THREADS_OP_WRITE) {
writeToClient(c,0);
} else if (io_threads_op == IO_THREADS_OP_READ) {
readQueryFromClient(c->conn);
} else {
serverPanic("io_threads_op value is unknown");
}
}
listEmpty(io_threads_list[id]);
io_threads_pending[id] = 0;
if (tio_debug) printf("[%ld] Done\n", id);
}
}
limitation
From the above implementation point of view, the 6.0 version of multithreading is not a complete multithreading. It I/O线程can only perform read or write operations at the same time. It is 事件处理线程always in a waiting state during this period . It is not a pipeline model and has a lot of waiting overhead for round-robin training.
Tair multithreading realization principle
Compared with the 6.0 version of multithreading, Tair's multithreading implementation is more elegant. As shown in the figure below, Tair is Main Threadresponsible for client connection establishment, etc., IO Threadrequest reading, response sending, command parsing, etc. The Worker Threadthread is dedicated to event processing. IO ThreadRead the user's request and analyze it, and then put the result of the analysis in the form of a command in the queue and send it to the Worker Threadprocessing. Worker ThreadAfter the command is processed, a response is generated and sent to it through another queue IO Thread. In order to improve the parallelism of threads, IO Threadand Worker Threaduse lock-free queues and pipes for data exchange between them, the overall performance will be better.
summary
Redis 4.0 introduces Lazy Freethreads, which solves the problem of server congestion caused by deletion of large keys. In version 6.0 I/O Thread, threads are introduced , which formally realizes multi-threading, but compared with Tair, it is not very elegant, and the performance improvement is not much. Look, the performance of the multi-threaded version is twice that of the single-threaded version, and the Tair multi-threaded version is 3 times that of the single-threaded version. In the author's opinion, Redis multithreading is nothing more than two ideas, I/O threadingand Slow commands threading, as the author said in his blog:
I/O threading is not going to happen in Redis AFAIK, because after much consideration I think it’s a lot of complexity without a good reason. Many Redis setups are network or memory bound actually. Additionally I really believe in a share-nothing setup, so the way I want to scale Redis is by improving the support for multiple Redis instances to be executed in the same host, especially via Redis Cluster.
What instead I really want a lot is slow operations threading, and with the Redis modules system we already are in the right direction. However in the future (not sure if in Redis 6 or 7) we’ll get key-level locking in the module system so that threads can completely acquire control of a key to process slow operations. Now modules can implement commands and can create a reply for the client in a completely separated way, but still to access the shared data set a global lock is needed: this will go away.
Redis authors prefer to use clusters to solve this problem I/O threading, especially in the context of the native Redis Cluster Proxy released in version 6.0, which makes clusters easier to use. In addition, the author is more inclined slow operations threading(such as version 4.0 released Lazy Free) to solve the multi-threading problem. In subsequent versions, whether the IO Threadimplementation will be more complete and the use of Module to optimize slow operations is really worth looking forward to.