RedisObject data structure, and Redis data type
RedisObject is the core of the Redis type system. Each key and value in the database, as well as the parameters processed by Redis itself, are represented as this data type.
The definition of redisObject is located in redis.h:
/*
* Redis 对象
*/
typedef struct redisObject {
// 类型
unsigned type:4;
// 对齐位
unsigned notused:2;
// 编码方式
unsigned encoding:4;
// LRU 时间（相对于 server.lruclock）
unsigned lru:22;
// 引用计数
int refcount;
// 指向对象的值
void *ptr;
} robj;
Type, encoding, and ptr are the three most important attributes.
type records the type of the value held by the object. Its value may be one of the following constants (definition is located in redis.h):
/*
* 对象类型
*/
#define REDIS_STRING 0 // 字符串
#define REDIS_LIST 1 // 列表
#define REDIS_SET 2 // 集合
#define REDIS_ZSET 3 // 有序集
#define REDIS_HASH 4 // 哈希表
encoding records the encoding of the value held by the object, its value may be one of the following constants (definition is located in redis.h):
/*
* 对象编码
*/
#define REDIS_ENCODING_RAW 0 // 编码为字符串
#define REDIS_ENCODING_INT 1 // 编码为整数
#define REDIS_ENCODING_HT 2 // 编码为哈希表
#define REDIS_ENCODING_ZIPMAP 3 // 编码为 zipmap
#define REDIS_ENCODING_LINKEDLIST 4 // 编码为双端链表
#define REDIS_ENCODING_ZIPLIST 5 // 编码为压缩列表
#define REDIS_ENCODING_INTSET 6 // 编码为整数集合
#define REDIS_ENCODING_SKIPLIST 7 // 编码为跳跃表
ptr is a pointer to the data structure that actually holds the value. This data structure is determined by the type attribute and the encoding attribute.
For example, if the type attribute of a redisObject is REDIS_LIST and the encoding attribute is REDIS_ENCODING_LINKEDLIST, then the object is a Redis list, its value is stored in a double-ended linked list, and the ptr pointer points to the double-ended linked list;
On the other hand, if the type attribute of a redisObject is REDIS_HASH and the encoding attribute is REDIS_ENCODING_ZIPMAP, then the object is a Redis hash table, its value is stored in a zipmap, and the ptr pointer points to the zipmap; and so on.
The following figure shows the relationship between redisObject, all Redis data types, and all Redis encoding methods (underlying implementation):
Command type checking and polymorphism
With the existence of the redisObject structure, it is much simpler to perform type checking and polymorphic operations on coding when executing commands that process data types.
When executing a command that deals with data types, Redis performs the following steps:
- According to the given key, find the corresponding redisObject in the database dictionary, if it is not found, it returns NULL.
- Check whether the type attribute of redisObject matches the type required to execute the command. If it does not match, a type error is returned.
- According to the encoding specified by the encoding attribute of redisObject, select the appropriate operation function to process the underlying data structure.
- The operation result of the returned data structure is used as the return value of the command.
As an example, the following shows the complete process of executing the LPOP command on the key:
Object sharing
Some objects are very common in Redis. For example, the return value of the command is OK, ERROR, WRONGTYPE and other characters. In addition, some small-range integers, such as single-digit, tens, and hundred-digit integers are very common.
In order to take advantage of this common situation, Redis uses a Flyweight mode internally: by pre-allocating some common value objects and sharing these objects among multiple data structures, the program avoids the trouble of repeated allocation and also saves some CPU time.
Redis pre-assigned value objects are as follows:
The return values of various commands, such as OK returned when the execution is successful, ERROR returned when the error is executed, WRONGTYPE returned when the type is wrong, QUEUED returned when the command is enqueued, and so on.
Including 0, all integers less than redis.h / REDIS_SHARED_INTEGERS (the default value of REDIS_SHARED_INTEGERS is 10000)
Because the reply value of the command is directly returned to the client, their values do not need to be shared; on the other hand, if a command The input value is an integer object less than REDIS_SHARED_INTEGERS, then when this object is to be saved into the database, Redis will release the original value and point the value pointer to the shared object.
As an example, the following figure shows three lists, all with pointers to a value object in the shared object array:
The values of the three lists are:
列表 A ： [20130101, 300, 10086] ，
列表 B ： [81, 12345678910, 999] ，
列表 C ： [100, 0, -25, 123] 。
Reference counting and object destruction
When redisObject is used as a database key or value, rather than used to store parameters, the lifetime of the object is very long, because the C language itself does not have a relevant mechanism to automatically release memory. If you only rely on the programmer's memory to object Tracking and destruction is basically impossible.
On the other hand, as mentioned earlier, a shared object may be referenced by multiple data structures. At this time, questions like "How many times is this object referenced?"
In order to solve the above two problems, Redis' object system uses reference counting technology to maintain and destroy objects. Its operating mechanism is as follows:
Each redisObject structure has a refcount attribute indicating how many times this object has been referenced.
When a new object is created, its refcount property is set to 1.
When sharing an object, Redis increments the object's refcount by one.
After using an object, or after dereferencing a shared object, the program decrements the object's refcount by one.
When the refcount of an object drops to 0, the redisObject structure and the memory of the data structure it references will be released.
summary
Redis uses its own implemented object mechanism to implement type determination, command polymorphism, and garbage collection based on reference counting.
A Redis-type key can have multiple underlying implementations.
Redis will pre-allocate some commonly used data objects and share these objects to reduce memory usage and avoid frequently allocating memory for small objects.
Redis design and implementation:
https://redisbook.readthedocs.io/en/latest/index.html
Annotated version of Redis 2.6 source code:
https://github.com/to-be-architect/annotated_redis_source
Kotlin developer community
The public account of the first Kotlin developer community in China, which mainly shares and exchanges related topics such as Kotlin programming language, Spring Boot, Android, React.js / Node.js, functional programming, and programming ideas.
The more noisy the world, the more peaceful thinking is needed.
