redis源码阅读(5)-timeEvent事件之过期key回收

版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/hehe123456ZXC/article/details/53264359

redis的事件主要由两种fileEvent 也就是各种socked的异步操作,通过epoll网络模型监听。另外一种就是timeEvent事件,redis就有一个timeEvent事件,可以理解为定时器。

注册timeEvent事件时,事件函数是int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) ,此事件函数主要有以下操作:

在此函数中主要做一下工作:
(1)过期key的收集工作,收集的方式和rehash方式一样,分为active和lazy collection.
(2)软件狗–这地方还没明白意思,后面再看
(3) update 一些静态数据
(4)rehash 哈希表,也就是上面介绍的
(5)触发BGSAVE / AOF读写,以及处理中断的子进程,BGSAVE / AOF进程主要是数据持久化的操作,后面针对这两个再分别写一篇文章
(6) 处理不同类型的客户端超时操作
(7) Replication reconnection 应该是和集群操作有关,后面会专门看redis的集群操作

此事件函数中所有的操作一秒钟执行server.hz次,同时为了调节操作的执行频次,使用run_with_period(milliseconds)函数。

void databasesCron(void) {
    /* Expire keys by random sampling. Not required for slaves
     * as master will synthesize DELs for us. */
    if (server.active_expire_enabled && server.masterhost == NULL)
        activeExpireCycle();

在客户端进行写操作时,如果带有过期时间的参数,那么server端会在db->expires哈希表中,存储此key的过期时间,同时在主字典中保存value,但是key是公用一块内存。节省了内存消耗。

重要的操作在activeExpireCycle函数,此函数回收过时key,算法是动态的,如果有较多的过期key的话,会消耗cpu执行事件。
为了动态的执行此操作,定义了一下static变量,用来终止或者继续过期key的回收操作。如果有太多的过期key的话,一直回收也会带来客户端的阻塞,所以redis将其做了动态处理,比如说每次处理几个数据库,每次占用多少cpu时间。
见下面的代码:

void activeExpireCycle(void) {
    /* This function has some global state in order to continue the work
     * incrementally across calls. */
    static unsigned int current_db = 0; /* Last DB tested. */
    static int timelimit_exit = 0;      /* Time limit hit in previous call? */

    unsigned int j, iteration = 0;
    unsigned int dbs_per_call = REDIS_DBCRON_DBS_PER_CALL;
    long long start = ustime(), timelimit;

    /* We usually should test REDIS_DBCRON_DBS_PER_CALL per iteration, with
     * two exceptions:
     *
     * 1) Don't test more DBs than we have.
     * 2) If last time we hit the time limit, we want to scan all DBs
     * in this iteration, as there is work to do in some DB and we don't want
     * expired keys to use memory for too much time. */

    if (dbs_per_call > (unsigned)server.dbnum || timelimit_exit)
        dbs_per_call = (unsigned)server.dbnum;

下面的一小段代码,减小回收的频次,也就是一次回收只执行timelimit时间,如果不进行此操作,1ms内会执行server.hz次。同时限制了每次调用此函数回收的数据库的数量dbs_per_call。

    /* We can use at max REDIS_EXPIRELOOKUPS_TIME_PERC percentage of CPU time
     * per iteration. Since this function gets called with a frequency of
     * server.hz times per second, the following is the max amount of
     * microseconds we can spend in this function. */
    timelimit = 1000000*REDIS_EXPIRELOOKUPS_TIME_PERC/server.hz/100;
    timelimit_exit = 0;
    if (timelimit <= 0) timelimit = 1;
    for (j = 0; j < dbs_per_call; j++) {
        int expired;
        redisDb *db = server.db+(current_db % server.dbnum);

        /* Increment the DB now so we are sure if we run out of time
         * in the current DB we'll restart from the next. This allows to
         * distribute the time evenly across DBs. */
        //下次执行此函数时从此current_db 数据库开始执行
        current_db++;


        /* Continue to expire if at the end of the cycle more than 25%
         * of the keys were expired. */
        do {
            unsigned long num, slots;
            long long now;

如果此数据库中过期的hash表中size为0,那么回收下一个数据库

            /* If there is nothing to expire try next DB ASAP. */
            if ((num = (unsigned long)dictSize(db->expires)) == 0) break;
            slots = (unsigned long)dictSlots(db->expires);
            now = mstime();

如果hash表中元素/槽的比例小于0.1 ,如果还回收 从所有的key中随机选key会消耗代价较大,跳出此数据库,等待何时时机再回收。

            /* When there are less than 1% filled slots getting random
             * keys is expensive, so stop here waiting for better times...
             * The dictionary will be resized asap. */
            if (num && slots > DICT_HT_INITIAL_SIZE &&
                (num*100/slots < 1)) break;

回收的主循环处,从所有的过期key中随机抽样key,进行回收

            /* The main collection cycle. Sample random keys among keys
             * with an expire set, checking for expired ones. */
            expired = 0;
            if (num > REDIS_EXPIRELOOKUPS_PER_CRON)
                num = REDIS_EXPIRELOOKUPS_PER_CRON;
            while (num--) {
                dictEntry *de;
                long long t;

dictGetRandomKey,从为非空的槽中,计算此槽的链表长度,然后随机挑选一个节点赋值de。获取过期时间t,如果now>t代表此key已经过期,那么对其进行回收。

                if ((de = dictGetRandomKey(db->expires)) == NULL) break;
                t = dictGetSignedIntegerVal(de);
                if (now > t) {
                    sds key = dictGetKey(de);
                    robj *keyobj = createStringObject(key,sdslen(key));
                    //广播过期key到从节点以及AOF file中,进行数据同步
                    propagateExpire(db,keyobj);
                    //从过期key字典和主字典中删除此key
                    dbDelete(db,keyobj);
                    decrRefCount(keyobj);
                    expired++;
                    server.stat_expiredkeys++;
                }
            }
            /* We can't block forever here even if there are many keys to
             * expire. So after a given amount of milliseconds return to the
             * caller waiting for the other active expire cycle. */
            //判断是否达到停止条件
            iteration++;
            if ((iteration & 0xf) == 0 && /* check once every 16 iterations. */
                (ustime()-start) > timelimit)
            {
                timelimit_exit = 1;
                return;
            }
        } while (expired > REDIS_EXPIRELOOKUPS_PER_CRON/4);//限制次数
    }
}

上面代码的逻辑就是 限制抽样key的个数,以及循环次数,是为了保证整个程序的执行效率。

猜你喜欢

转载自blog.csdn.net/hehe123456ZXC/article/details/53264359
今日推荐