Redis from entry to abandonment series (7) Expiration, memory elimination strategy

Redis from entry to abandonment series (7) Expiration, memory elimination strategy

The example in this article is based on: 5.0.4

Expiration Policy

Redis has two expiration strategies for keys with expiration times set

• lazy delete
• Periodically delete

lazy delete

The timing of lazy deletion is to judge when you want to get the key. Here I use the String type as a demonstration drawing:

int expireIfNeeded(redisDb *db, robj *key) {
    if (!keyIsExpired(db,key)) return 0;

    /* If we are running in the context of a slave, instead of
     * evicting the expired key from the database, we return ASAP:
     * the slave key expiration is controlled by the master that will
     * send us synthesized DEL operations for expired keys.
     *
     * Still we try to return the right information to the caller,
     * that is, 0 if we think the key should be still valid, 1 if
     * we think the key is expired at this time. */
    if (server.masterhost != NULL) return 1;

    /* Delete the key */
    server.stat_expiredkeys++;
    propagateExpire(db,key,server.lazyfree_lazy_expire);
    notifyKeyspaceEvent(NOTIFY_EXPIRED,
        "expired",key,db->id);
    return server.lazyfree_lazy_expire ? dbAsyncDelete(db,key) :
                                         dbSyncDelete(db,key);
}

We found that when the key has an expiration time set, and the key has expired, then redis will determine whether it is enabled lazyfree_lazy_expire. If it is enabled, it will be deleted asynchronously. If not, it will be deleted synchronously. (The feature after 4.0, when the big key is deleted very useful time)

Periodically delete

When there is lazy deletion, part of the requirements are met, but in practical applications, there will be keys that have expired and have not been accessed, which will occupy memory flatly. So how does redis solve it?

Redis will have a scheduled task that runs 10 times per second.

• Randomly select 20 keys with expiration time set for expiration detection
• delete all expired keys
• If more than 25% of keys expire, start over from step 1.

Since redis is single-threaded, if the above strategy is randomly deleted at regular intervals, when a large number of keys expire, redis will be unable to process client requests? No, no, in fact, when redis is doing periodic random deletion, there is a limit, which is to set the upper limit of the scan time, the default is at most 25ms (timelimit = 1000000*ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC/server.hz/100;), so when the client request comes When the server is in the expired scan period, the client will wait up to 25ms before proceeding with its business processing.

Then if the timeout time set by redis is too short, there may be a large number of timeout connections. In order to avoid this problem, it is recommended to add a random range when setting the expiration time to avoid a large number of keys from expiring at the same time~

When there is a master-slave, the slave library will not start timing and random deletion. It depends on adding a del command to the aof file opened by the master, and then executing the statement from the library to delete the expired key.

Memory Retirement Policy

We know that redis is a pure memory database. If the use of redis exceeds the memory limit, the swap behavior will occur. Compared with memory, the performance of using disk is not a little bit degraded. At this time, we need to set the maximum memory used by redis

maxmemory <bytes>

By setting maxmemory, when the memory used exceeds maxmemory, redis provides several optional strategies to control the amount of memory used

• noeviction (when the memory is not enough to accommodate the newly written data, the new write operation will report an error)
• allkeys-lru (remove the least recently used key in the keyspace when there is not enough memory for newly written data)
• allkeys-random (when the memory is not enough to accommodate newly written data, in the key space, remove a key randomly)
• volatile-lru (when the memory is not enough to accommodate newly written data, in the key space with the expiration time set, remove the least recently used key)
• volatile-random (when the memory is not enough to accommodate newly written data, a key is randomly removed in the key space with the expiration time set)
• volatile-ttl (when the memory is not enough to accommodate the newly written data, in the key space with the expiration time set, the key with an earlier expiration time will be removed first)

write at the end

The three days of the Dragon Boat Festival have passed like this~ I hope you all have a good rest, and then continue to get up and move bricks~hhhhh

{{o.name}}

{{m.name}}