Redis data structure

Redis is a key-value storage system. Compared with Memcache, its value supports more data types.
Redis uses objects to identify keys and values in the database. Whenever a key-value pair is created in redis, two objects are created: key objects and value objects.
The object in Redis is represented by a redisObject structure. The three attributes related to saving data in this structure are: type attribute, encoding attribute, and ptr attribute. as follows:

typedef struct redisObject{
    
    
    //类型
    unsigned type:4;
    //编码
    unsigned encoding:4;
    指向底层实现数据结构的指针
    void *ptr;
}robj;

One, the data type of redis

The type attribute records the type of the object. The value of this attribute can be one of the following constants:

Type constant	The name of the object
REDIS_STRING	String object
REDIS_LIST	List object
REDIS_HASH	Hash object
REDIS_SET	Collection object
REDIS_ZSET	Ordered collection of objects

Among them, the Redis key is always a string object, and the value can be one of the above five types of objects.
Use TYPE keyto view a database key value of the object data types, such as:

redis> set name "zs"
OK
redis> TYPE name
"string"

Two, redis coding realization

The encoding attribute records the encoding used by the object, that is, what data structure the object uses as the underlying implementation of the object; the ptr pointer points to the underlying implementation data structure of the object.
Redis objects of each data type use different encodings. The following table lists the encodings that each type of object can use:

Types of	coding	Object
REDIS_STRING	REDIS_ENCODING_INT	String object implemented using integer values
REDIS_STRING	REDIS_ENCODING_EMBSTR	String object realized by using embstr coded simple dynamic string
REDIS_STRING	REDIS_ENCODING_RAW	String object implemented using simple dynamic string
REDIS_LIST	REDIS_ENCODING_ZIPLIST	List object implemented using compressed list
REDIS_LIST	REDIS_ENCODING_LINKEDLIST	List object implemented using double-ended linked list
REDIS_HASH	REDIS_ENCODING_ZIPLIST	Hash object implemented using compressed list
REDIS_HASH	REDIS_ENCODING_HT	Hash object implemented using dictionary
REDIS_SET	REDIS_ENCODING_INTSET	Collection objects implemented using integer collections
REDIS_SET	REDIS_ENCODING_HT	Collection object implemented using dictionary
REDIS_ZSET	REDIS_ENCODING_ZIPLIST	Ordered collection objects implemented using compressed lists
REDIS_ZSET	REDIS_ENCODING_SKIPLIST	Ordered collection of objects implemented using jump tables

Use OBJECT ENCODING keyto see the value of the objects of a database of key codes, such as:

redis> set name "zs"
OK
redis> OBJECT ENCODING name
"embstr"

Associating multiple codes for specific types of objects greatly improves the flexibility and efficiency of Redis. Redis can set different codes for an object according to different usage scenarios, so as to optimize the efficiency of the object in a certain scenario. Next, we will introduce how each type uses encoding in different scenarios.

Third, the encoding implementation and encoding conversion of different objects

1. String Object

The encoding of string objects can be int, raw, embstr.

int: The string object holds an integer value, and this integer value can be represented by the long type;
raw: The string object stores the string value, and the length of the string value is greater than 32 bytes;
embstr: The string object stores the string value, and the length of the string value is less than or equal to 32 bytes;

(1) int encoding

int encoded string object
Example:

redis> set number 10086
OK
redis> OBJECT ENCODING number
"int"

(2) Raw encoding

raw encoded string object
Example:

redis> set story "Long,long ago there lived a king..."
OK
redis> STRLEN story
(integer) 37
redis> OBJECT ENCODING story
"raw"

(3) Embtsr encoding

Embstr encoding is an optimized encoding method used to save short strings. This encoding, like raw encoding, uses redisObject structure and sdshdr structure to represent string objects, but raw encoding calls the memory allocation function twice to create them separately The redisObject structure and the sdshdr structure, while embstr encoding allocates a continuous space through a memory allocation function. The space contains the redisObject structure and the sdshdr structure in turn. Therefore, embstr encoding is more efficient than raw encoding in allocating and releasing memory, and it can make better use of the advantages of caching.
embstr encoded string object
Example:

redis> set msg "hello"
OK
redis> OBJECT ENCODING msg
"embstr"

(4) Code conversion

Int and embstr encoded objects will be converted into raw encoded string objects when the conditions are met.
(a) After some commands are executed for the string object of int encoding, the object is no longer an integer value, but a string value, and the encoding will change from int to raw. Such as:

redis> set number 10086
OK
redis> OBJECT ENCODING number
"int"
redis> APPEND number “ is a good number”
(integer) 23
redis> OBJECT ENCODING number
"raw"

(b) The embstr-encoded string object is read-only. When we execute any modification command on the embstr-encoded string object, the program will first convert the encoding of the object from embstr to raw, and then execute the modification command, so embstr The encoded string object will become raw encoding after modification. Such as:

redis> set msg "hello"
OK
redis> OBJECT ENCODING msg
"embstr"
redis> APPEND msg " world"
(integer) 11
redis> OBJECT ENCODING msg
"raw"

2. List objects

The encoding of the list object can be ziplist, linkedlist.

ziplist: At the same time: (1) The length of all string elements stored in the list object is less than 64 bytes; (2) The number of elements stored in the list object is less than 512;
linkedlist: When one of the above two conditions is not met, use the linkedlist encoding.

(1) ziplist encoding

ziplist encoded list object
Example:

redis> RPUSH numbers 1 "three" 5
(integer) 3
redis> OBJECT ENCODING numbers
"ziplist"

(2) Linkedlist encoding

List object encoded by linkedlist
The structure of stringObject

Example:

redis> RPUSH numbers 1 "three" 5 .......
(integer) 513
redis> OBJECT ENCODING numbers
"linkedlist"

(3) Code conversion

For a list object that uses ziplist encoding, when either of the two conditions for using ziplist encoding cannot be met, the encoding of the object is converted to linkedlist.

3. Hash objects

The encoding of the hash object can be ziplist or hashtable.

ziplist: At the same time: (1) The length of the key and value strings of all key-value pairs stored in the hash object is less than 64 bytes; (2) The number of key-value pairs stored in the hash object does not exceed 512;
Hashtable: When one of the above two conditions is not met, hashtable encoding is used.

(1) ziplist encoding

When a new key-value pair is to be added to the hash object, the compressed list node that stores the key is pushed to the end of the compressed list table, and then the compressed list node that stores the value is pushed to the end of the compressed list table.
ziplist encoded hash object
Low-level implementation of compressed list of hash objects

Example:

redis> HSET profile name "Tom"
(integer) 1
redis> HSET profile age 25
(integer) 1
redis> HSET profile career "Programmerr"
(integer) 1
redis> OBJECT ENCODING numbers
"ziplist"

(2) Hashtable encoding

The hash object encoded by hashtable uses a dictionary as the underlying implementation, and each key-value pair in the hash object is stored using a dictionary key-value pair.
hashtable encoded hash object

Example:

redis> HSET  book "long_long_long_long_long_long...."
(integer) 1
redis> OBJECT ENCODING numbers
"hashtable"

(3) Code conversion

For a hash object that uses ziplist encoding, when either of the two conditions of ziplist encoding cannot be met, the encoding of the object is converted to hashtable.

4. Collection objects

The encoding of the collection object can be intset, hashtable.

intset: At the same time: (1) All elements stored in the collection object are integer values; (2) The number of elements stored in the collection object does not exceed 512;
Hashtable: When one of the above two conditions is not met, hashtable encoding is used.

(1) intset encoding

Use the integer collection as the underlying implementation, and all the elements contained in the collection object are stored in the integer collection.
intset encoded collection object

Example:

redis> SADD numbers 1 3 5
(integer) 3
redis> OBJECT ENCODING numbers
"intset"

(2) Hashtable encoding

The hashtable-encoded collection object uses a dictionary as the underlying implementation. Each key of the dictionary is a string, and the values of the dictionary are all set to NULL.
hashtable encoded collection object

Example:

redis> SADD fruits "cherry" "apple" "banana"
(integer) 3
redis> OBJECT ENCODING fruits
"hashtable"

(3) Code conversion

For collection objects that use intset encoding, when either of the two conditions for intset encoding cannot be met, the encoding of the object will be converted to hashtable.

5. Ordered collection of objects

The coding of ordered set objects can be ziplist, skiplist.

ziplist: At the same time: (1) The number of elements stored in the ordered set is less than 128; (2) The length of all element members stored in the ordered set is less than 64 bytes;
skiplist: When one of the above two conditions is not met, skiplist encoding is used.

(1) ziplist encoding

The ziplist-encoded ordered collection object uses a compressed list as the bottom layer. Each collection element is saved using two compressed list nodes next to each other. The first node stores the members of the element, and the second element stores the element’s score. . The set elements in the compressed list are arranged in descending order of score. Elements with smaller scores are placed near the head of the table, and elements with larger scores are placed near the end of the table.
ziplist encoded ordered collection object
Ordered set elements are sorted from small to large in the compressed list

Example:

redis> ZADD price 8.5 apple 5.0 banana 6.0 cherry
(integer) 3
redis> OBJECT ENCODING price
"ziplist"

(2) Skiplist encoding

The ordered collection object encoded by skiplist uses zset as the underlying implementation. A zset structure contains both a dictionary (dict) and a skip list (zsl).
The zsl jump table saves all collection elements from small to large according to the score. Each jump table node saves a collection element: the object attribute of the jump table stores the members of the element, and the score attribute of the jump table stores the element's score. Through the jump list, you can perform range operations on the ordered set. For example, commands such as ZRANK and ZRANGE are implemented based on the jump list API.
The dict dictionary creates a mapping from members to scores for an ordered set. Each key-value pair of the dictionary stores a set element: the key of the dictionary stores the members of the element, and the value of the dictionary stores the score of the element. Through the dictionary, the program can look up the score of a given member with O(1) complexity, such as the ZSCORE command.
Ordered collection object encoded by skiplist
Ordered collection objects are stored in the dictionary and jump list at the same time

Example:

redis> ZCARD numbers
(integer) 128
redis> OBJECT ENCODING numbers
"ziplist"
redis> ZADD numbers 3.14 pi
(integer) 1
redis> ZCARD numbers
(integer) 129
redis> OBJECT ENCODING numbers
"skiplist"

(3) Code conversion

For an ordered set of objects encoded using ziplist, when either of the two conditions of encoding using ziplist cannot be met, the encoding of the object will be converted to skiplist.

Four, references

[1]: "Redis Design and Implementation" Part One Data Structure and Objects