redis的事件主要由两种fileEvent 也就是各种socked的异步操作,通过epoll网络模型监听。另外一种就是timeEvent事件,redis就有一个timeEvent事件,可以理解为定时器。
注册timeEvent事件时,事件函数是int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) ,此事件函数主要有以下操作:
在此函数中主要做一下工作:
(1)过期key的收集工作,收集的方式和rehash方式一样,分为active和lazy collection.
(2)软件狗–这地方还没明白意思,后面再看
(3) update 一些静态数据
(4)rehash 哈希表,也就是上面介绍的
(5)触发BGSAVE / AOF读写,以及处理中断的子进程,BGSAVE / AOF进程主要是数据持久化的操作,后面针对这两个再分别写一篇文章
(6) 处理不同类型的客户端超时操作
(7) Replication reconnection 应该是和集群操作有关,后面会专门看redis的集群操作
此事件函数中所有的操作一秒钟执行server.hz次,同时为了调节操作的执行频次,使用run_with_period(milliseconds)函数。
void databasesCron(void) {
/* Expire keys by random sampling. Not required for slaves
* as master will synthesize DELs for us. */
if (server.active_expire_enabled && server.masterhost == NULL)
activeExpireCycle();
在客户端进行写操作时,如果带有过期时间的参数,那么server端会在db->expires哈希表中,存储此key的过期时间,同时在主字典中保存value,但是key是公用一块内存。节省了内存消耗。
重要的操作在activeExpireCycle函数,此函数回收过时key,算法是动态的,如果有较多的过期key的话,会消耗cpu执行事件。
为了动态的执行此操作,定义了一下static变量,用来终止或者继续过期key的回收操作。如果有太多的过期key的话,一直回收也会带来客户端的阻塞,所以redis将其做了动态处理,比如说每次处理几个数据库,每次占用多少cpu时间。
见下面的代码:
void activeExpireCycle(void) {
/* This function has some global state in order to continue the work
* incrementally across calls. */
static unsigned int current_db = 0; /* Last DB tested. */
static int timelimit_exit = 0; /* Time limit hit in previous call? */
unsigned int j, iteration = 0;
unsigned int dbs_per_call = REDIS_DBCRON_DBS_PER_CALL;
long long start = ustime(), timelimit;
/* We usually should test REDIS_DBCRON_DBS_PER_CALL per iteration, with
* two exceptions:
*
* 1) Don't test more DBs than we have.
* 2) If last time we hit the time limit, we want to scan all DBs
* in this iteration, as there is work to do in some DB and we don't want
* expired keys to use memory for too much time. */
if (dbs_per_call > (unsigned)server.dbnum || timelimit_exit)
dbs_per_call = (unsigned)server.dbnum;
下面的一小段代码,减小回收的频次,也就是一次回收只执行timelimit时间,如果不进行此操作,1ms内会执行server.hz次。同时限制了每次调用此函数回收的数据库的数量dbs_per_call。
/* We can use at max REDIS_EXPIRELOOKUPS_TIME_PERC percentage of CPU time
* per iteration. Since this function gets called with a frequency of
* server.hz times per second, the following is the max amount of
* microseconds we can spend in this function. */
timelimit = 1000000*REDIS_EXPIRELOOKUPS_TIME_PERC/server.hz/100;
timelimit_exit = 0;
if (timelimit <= 0) timelimit = 1;
for (j = 0; j < dbs_per_call; j++) {
int expired;
redisDb *db = server.db+(current_db % server.dbnum);
/* Increment the DB now so we are sure if we run out of time
* in the current DB we'll restart from the next. This allows to
* distribute the time evenly across DBs. */
//下次执行此函数时从此current_db 数据库开始执行
current_db++;
/* Continue to expire if at the end of the cycle more than 25%
* of the keys were expired. */
do {
unsigned long num, slots;
long long now;
如果此数据库中过期的hash表中size为0,那么回收下一个数据库
/* If there is nothing to expire try next DB ASAP. */
if ((num = (unsigned long)dictSize(db->expires)) == 0) break;
slots = (unsigned long)dictSlots(db->expires);
now = mstime();
如果hash表中元素/槽的比例小于0.1 ,如果还回收 从所有的key中随机选key会消耗代价较大,跳出此数据库,等待何时时机再回收。
/* When there are less than 1% filled slots getting random
* keys is expensive, so stop here waiting for better times...
* The dictionary will be resized asap. */
if (num && slots > DICT_HT_INITIAL_SIZE &&
(num*100/slots < 1)) break;
回收的主循环处,从所有的过期key中随机抽样key,进行回收
/* The main collection cycle. Sample random keys among keys
* with an expire set, checking for expired ones. */
expired = 0;
if (num > REDIS_EXPIRELOOKUPS_PER_CRON)
num = REDIS_EXPIRELOOKUPS_PER_CRON;
while (num--) {
dictEntry *de;
long long t;
dictGetRandomKey,从为非空的槽中,计算此槽的链表长度,然后随机挑选一个节点赋值de。获取过期时间t,如果now>t代表此key已经过期,那么对其进行回收。
if ((de = dictGetRandomKey(db->expires)) == NULL) break;
t = dictGetSignedIntegerVal(de);
if (now > t) {
sds key = dictGetKey(de);
robj *keyobj = createStringObject(key,sdslen(key));
//广播过期key到从节点以及AOF file中,进行数据同步
propagateExpire(db,keyobj);
//从过期key字典和主字典中删除此key
dbDelete(db,keyobj);
decrRefCount(keyobj);
expired++;
server.stat_expiredkeys++;
}
}
/* We can't block forever here even if there are many keys to
* expire. So after a given amount of milliseconds return to the
* caller waiting for the other active expire cycle. */
//判断是否达到停止条件
iteration++;
if ((iteration & 0xf) == 0 && /* check once every 16 iterations. */
(ustime()-start) > timelimit)
{
timelimit_exit = 1;
return;
}
} while (expired > REDIS_EXPIRELOOKUPS_PER_CRON/4);//限制次数
}
}
上面代码的逻辑就是 限制抽样key的个数,以及循环次数,是为了保证整个程序的执行效率。