The data structure of redis study notes-SDS (simple dynamic string)

table of Contents

SDS structure

Two forms of redis string storage (embstr and raw)

SDS and C string difference

Redis does not directly use the traditional string representation of the C language (character array ending with a null character), but instead builds an abstract type called simple dynamic string (SDS) by itself, and uses SDS as Redis's default string representation

• SDS structure

struct sdshdr{
    // 1bytes 数组容量
    int8 capacity;

    //1bytes 数组长度
    int8 len;

    //1bytes 特殊标志位
    int8 flags;
    
    //长度取决于实际情况 数组内容
    byte[] content;
};

In addition to saving the characters of the string, content also saves a null character'\0' at the end. The null character is not counted in the len attribute. The advantage of following the null character end is that a part of the C string function can be reused

• Two forms of redis string storage (embstr and raw)

Let's first look at the code running on the client:

127.0.0.1:6379> set test_str 11111111111111111111111111111111111111111111
OK
127.0.0.1:6379> debug object test_str
Value at:0x7f7fb8215100 refcount:1 encoding:embstr serializedlength:12 lru:11329907 lru_seconds_idle:16
127.0.0.1:6379> set test_str 111111111111111111111111111111111111111111111
OK
127.0.0.1:6379> debug object test_str
Value at:0x7f7fb826dc40 refcount:1 encoding:raw serializedlength:12 lru:11329942 lru_seconds_idle:4
127.0.0.1:6379> 

From the code, it can be found that when the string length is 44, it is stored in embstr form, and when it is greater than 44, it is stored in raw form, so why is 44 as the dividing line?

Let's take a brief look at the redis object header

struct redisObject {

    // 类型  4bits
    int4 type;

    // 编码  4bits
    int4 encoding;

    // lru 24bits
    int24 lru;
   
    // 引用计数 4bytes
    int32 refcount;

    // 8bttes (64位操作系统) 指向底层实现数据结构的指针  
    void *ptr;

} robj;

Embstr uses malloc ( dynamic memory allocation ) to allocate memory once, and the object header and SDS object are stored together consecutively, while raw needs two times, and the two object headers are not stored together consecutively. In the memory allocator (jemalloc), the unit of allocated memory size is 2/4/8/16/32/64 bytes, etc., so it can be seen that the longest is 64 bytes, 64 bytes minus the redis header There are 45 remaining parts (16) and SDS objects (3), and the end of the string is NULL, so the maximum length of the string is 44.

 embstr( embedded string ): The space of the string will be allocated together with the space of the redisObject object, and both are in the same memory block. Using this method can reduce the number of memory allocations, improve the efficiency of memory allocation, and reduce the rate of memory fragmentation.

raw: called twice each memory allocation function to create  redisObject the structure and  sdshdr configuration, while  embstr coding is assigned to a continuous space by calling a memory allocation function, which in turn contains a space  redisObject and  sdshdr two structures

Note: The value is an integer and the length meets the length of long, and the storage type is int. You can combine my above operations to try

SDS and C string difference

1. Time complexity of obtaining the length of the string: SDS: O(1), C:O(n)
2. Binary security : The characters in the C string must conform to a certain encoding (such as ASCII), and SDS will process content the data stored in the array in the SDS in a binary manner  (note that when writing the code, sometimes pay attention to the transcoding format. To be consistent with redis to prevent garbled codes)
3. Lazy space release: When the SDS API needs to shorten the string saved by SDS, the program does not immediately use memory reallocation to reclaim the extra bytes after shortening
4. Pre-allocation of space: when the string is less than 1MB, the expansion is doubled, and if it is greater than 1MB, 1MB will be allocated to him each time

 

This article comes from the Redis In-Depth Adventure, Redis Design and Implementation , and the official redis documents are waiting to be summarized. Please understand if there are any shortcomings. In the end, everyone has time to read Qian Wenpin's Redis In-Depth Adventures book, which is very powerful.