Foreword
Next, we studied together at Redis project architecture -related issues, the probability of this part of the emergence of a relatively large number, because not everyone will go to study the source code, if the interview blindly ask destined to be a source it may be embarrassed to talk.
Under the premise does not require candidates to interview Redis highly skilled engineering problem would be the best choice .
In this article you will learn the following:
memory recovery 1.Redis the Detailed
persistence mechanism of 2.Redis
Q1: understand the Redis memory recall it? Talk about your understanding
1.1 Why recovered memory?
Redis as a memory database, if not only into a simple to explode sooner or later, in fact, a lot of the guys Redis as the primary storage DB used sooner or later tasted the bitter fruit, unless of course your home yard really not bad money, the number of T-level memory are drizzle, data growth is no longer growing or after a certain degree of scene, is another matter.
For ordinary factory that we put cost savings as a KPI, or when the cache with the Redis more in line with economic conditions at home, so look at the problem so the interviewer is still relatively fit the actual, better than those Shredded RBTree, if your question is well within the knowledge firing range, give the interviewer a point of praise to say!
Redis service in order to allow safe and stable operation, so that the use of memory is maintained within a certain threshold is very necessary, so we need to remove the deleted, clean up the clean-up, the memory required to leave the key, imagine a several rivers need to set the warning level to ensure that no burst is not exhausted, Redis is the same, but only care about Redis can burst to a map:
The figure set the machine's memory to 128GB, 64GB occupy a relatively safe level, close to 80% if the memory is about 100GB, then think Redis current carrying capacity has been relatively large, the specific ratio can be based on company and personal business experience determine.
I just want to express consideration for security and stability, do not think it means 128GB of memory to store 128GB of data, it is to be discounted is.
1.2 Where recovered memory?
Redis amount of memory is divided into two parts: a key-value pair storage and consumption itself runs consumption. The latter obviously we can not be recycled, it can only key-value pairs from the start, can be divided into several key-value pairs: with expired without expired, hot data and cold data . For key with expired it is to be deleted, and then if you remove all of expired keys on the memory is still insufficient how to do? It would only be able to kick off a portion of the data.
Choose life everywhere , this allows the author to mind the "Titanic" hit an iceberg cruise instant influx of sea water, faced with insufficient number of lifeboats, people made a choice: let the women and children go first, gentlemen choose to stay under the sea to escape the scene shown:
1.3 How to implement key-value pairs delete outdated?
To delete the implementation of key-value pairs that we need to understand the following:
- Timeout expired key-value pair with stored where?
- How to determine the key with a timeout on whether can be deleted up?
- Delete mechanism What and how to choose ?
1.3.1 store key-value pairs
The old rules to look at github source, src / server.h in to the redisDb structure gives the answer:
typedef struct redisDb {
dict *dict; /* The keyspace for this DB */
dict *expires; /* Timeout of keys with a timeout set */
dict *blocking_keys; /* Keys with clients waiting for data (BLPOP)*/
dict *ready_keys; /* Blocked keys that received a PUSH */
dict *watched_keys; /* WATCHED keys for MULTI/EXEC CAS */
int id; /* Database ID */
long long avg_ttl; /* Average TTL, just for stats */
unsigned long expires_cursor; /* Cursor of the active expire cycle. */
list *defrag_later; /* List of key names to attempt to defrag one by one, gradually. */
} redisDb;On Redis is essentially a large key-value, key is the string, value There are several objects: strings, lists, ordered lists, sets, hash, etc. These key-value is stored in the dict redisDb and look at the painting Huang Jianhong a very praise chart:
See here, and clear mechanism for deleting a step, as long as we target key-value redisDb in dict is deleted on the line, but seemingly not so simple, Redis key-value pairs for expired must have their own organizational rules, let's move on study it!
redisDb的expires成员的类型也是dict，和键值对是一样的，本质上expires是dict的子集，expires保存的是所有带过期的键值对，称之为过期字典吧，它才是我们研究的重点。
对于键，我们可以设置绝对和相对过期时间、以及查看剩余时间：
- 使用EXPIRE和PEXPIRE来实现键值对的秒级和毫秒级生存时间设定，这是相对时长的过期设置
- 使用EXPIREAT和EXPIREAT来实现键值对在某个秒级和毫秒级时间戳时进行过期删除，属于绝对过期设置
- 通过TTL和PTTL来查看带有生存时间的键值对的剩余过期时间
上述三组命令在设计缓存时用处比较大，有心的读者可以留意。
过期字典expires和键值对空间dict存储的内容并不完全一样，过期字典expires的key是指向Redis对应对象的指针，其value是long long型的unix时间戳，前面的EXPIRE和PEXPIRE相对时长最终也会转换为时间戳，来看下过期字典expires的结构，笔者画了个图：
1.3.2 键值对的过期删除判断
判断键是否过期可删除，需要先查过期字典是否存在该值，如果存在则进一步判断过期时间戳和当前时间戳的相对大小，做出删除判断，简单的流程如图：
1.3.3 键值对的删除策略
经过前面的几个环节，我们知道了Redis的两种存储位置：键空间和过期字典，以及过期字典expires的结构、判断是否过期的方法，那么该如何实施删除呢？
先抛开Redis来想一下可能的几种删除策略：
- 定时删除：在设置键的过期时间的同时，创建定时器，让定时器在键过期时间到来时，即刻执行键值对的删除；
- 定期删除：每隔特定的时间对数据库进行一次扫描，检测并删除其中的过期键值对；
- 惰性删除：键值对过期暂时不进行删除，至于删除的时机与键值对的使用有关，当获取键时先查看其是否过期，过期就删除，否则就保留；
在上述的三种策略中定时删除和定期删除属于不同时间粒度的主动删除，惰性删除属于被动删除。
三种策略都有各自的优缺点：
定时删除对内存使用率有优势，但是对CPU不友好，惰性删除对内存不友好，如果某些键值对一直不被使用，那么会造成一定量的内存浪费，定期删除是定时删除和惰性删除的折中。
Reids采用的是惰性删除和定时删除的结合，一般来说可以借助最小堆来实现定时器，不过Redis的设计考虑到时间事件的有限种类和数量，使用了无序链表存储时间事件，这样如果在此基础上实现定时删除，就意味着O(N)遍历获取最近需要删除的数据。
但是我觉得antirez如果非要使用定时删除，那么他肯定不会使用原来的无序链表机制，所以个人认为已存在的无序链表不能作为Redis不使用定时删除的根本理由，冒昧猜测唯一可能的是antirez觉得没有必要使用定时删除。
1.3.4 定期删除的实现细节
定期删除听着很简单，但是如何控制执行的频率和时长呢？
试想一下如果执行频率太少就退化为惰性删除了，如果执行时间太长又和定时删除类似了，想想还确实是个难题！并且执行定期删除的时机也需要考虑，所以我们继续来看看Redis是如何实现定期删除的吧！笔者在src/expire.c文件中找到了activeExpireCycle函数，定期删除就是由此函数实现的，在代码中antirez做了比较详尽的注释，不过都是英文的，试着读了一下模模糊糊弄个大概，所以学习英文并阅读外文资料是很重要的学习途径。
由于笔者对Redis源码了解不多，只能做个模糊版本的解读，所以难免有问题，还是建议有条件的读者自行前往源码区阅读，抛砖引玉看下笔者的模糊版本：
- 该算法是个自适应的过程，当过期的key比较少时那么就花费很少的cpu时间来处理，如果过期的key很多就采用激进的方式来处理，避免大量的内存消耗，可以理解为判断过期键多就多跑几次，少则少跑几次；
- 由于Redis中有很多数据库db，该算法会逐个扫描，本次结束时继续向后面的db扫描，是个闭环的过程；
- 定期删除有快速循环和慢速循环两种模式，主要采用慢速循环模式，其循环频率主要取决于server.hz，通常设置为10，也就是每秒执行10次慢循环定期删除，执行过程中如果耗时超过25%的CPU时间就停止；
- 慢速循环的执行时间相对较长，会出现超时问题，快速循环模式的执行时间不超过1ms，也就是执行时间更短，但是执行的次数更多，在执行过程中发现某个db中抽样的key中过期key占比低于25%则跳过；
主体意思：定期删除是个自适应的闭环并且概率化的抽样扫描过程，过程中都有执行时间和cpu时间的限制，如果触发阈值就停止，可以说是尽量在不影响对客户端的响应下润物细无声地进行的。
1.3.5 DEL删除键值对
在Redis4.0之前执行del操作时如果key-value很大，那么可能导致阻塞，在新版本中引入了BIO线程以及一些新的命令，实现了del的延时懒删除，最后会有BIO线程来实现内存的清理回收。
1.4 内存淘汰机制
为了保证Redis的安全稳定运行，设置了一个max-memory的阈值，那么当内存用量到达阈值，新写入的键值对无法写入，此时就需要内存淘汰机制，在Redis的配置中有几种淘汰策略可以选择，详细如下：
- noeviction: 当内存不足以容纳新写入数据时，新写入操作会报错；
- allkeys-lru：当内存不足以容纳新写入数据时，在键空间中移除最近最少使用的 key；
- allkeys-random：当内存不足以容纳新写入数据时，在键空间中随机移除某个 key；
- volatile-lru：当内存不足以容纳新写入数据时，在设置了过期时间的键空间中，移除最近最少使用的 key；
- volatile-random：当内存不足以容纳新写入数据时，在设置了过期时间的键空间中，随机移除某个 key；
- volatile-ttl：当内存不足以容纳新写入数据时，在设置了过期时间的键空间中，有更早过期时间的 key 优先移除；
后三种策略都是针对过期字典的处理，但是在过期字典为空时会noeviction一样返回写入失败，毫无策略地随机删除也不太可取，所以一般选择第二种allkeys-lru基于LRU策略进行淘汰。
个人认为antirez一向都是工程化思维，善于使用概率化设计来做近似实现，LRU算法也不例外，Redis中实现了近似LRU算法，并且经过几个版本的迭代效果已经比较接近理论LRU算法的效果了，这个也是个不错的内容，由于篇幅限制，本文计划后续单独讲LRU算法时再进行详细讨论。
1.5 过期键删除和内存淘汰的关系
过期健删除策略强调的是对过期健的操作，如果有健过期而内存足够，Redis不会使用内存淘汰机制来腾退空间，这时会优先使用过期健删除策略删除过期健。
Memory elimination mechanism emphasized that out of the operation of the data memory, when memory is low, even if some do not reach health expiration date or no set expiration should be deleted according to a certain part of the strategy to ensure vacate the space to write new data .
Q2: talk about your understanding of Redis persistence mechanism.
Personally I think that the highlight in itself, but also the interview hot spots, mainly on the direction of both the persistence Redis database include: RDB mechanism principle, the principle AOF mechanisms, their advantages and disadvantages for RDB and AOF trade-offs, a new version of Redis on the project Mixed lasting strategy, etc., if they can grasp the main points of the persistent problem to cross the border.