Interviewer: Do you understand the underlying objects to achieve it through Redis

The last chapter we talked about the underlying data structure Redis, who do not know may be in doubt: This and the five objects usually use so what does it matter? In this chapter we will mainly explain the link they create.

Before looking at this article, if ziplist, skiplist, intset and other data structures are not familiar with it, it is recommended to review the previous section: Interviewer: Have you seen Redis underlying data structures to achieve it?

0.5 What are the five objects

Five objects that we used string, list, set, zset, hash

1. Why should target

We usually operated mainly by redis api operation target, rather than through it to complete the call (appearance pattern) which the underlying data structures. But we also need to understand the underlying, the only way to write efficient code optimization.

1. Like with java objects allow developers to more simple and convenient, lower development threshold. Developers do not need to understand the complexities of the underlying API, directly call level interfaces to achieve development.

2. Redis commands according to the type of object to determine whether the law, if you set key value1 value2 on the error.

3. The object may contain a variety of data structures to store the data of more polymorphic. (Speaker below)

4. Reids do object-based garbage collection (reference counting).

5. Objects with richer property to help redis achieve more advanced functions. (Such as idle time objects).

2. Redis objects (RedisObject) source code analysis

typedef struct redisObject {

    // type 
    unsigned of the type: 4 ;

    // encoding 
    unsigned encoding:. 4 ;

    // points to a data structure of the underlying implementation pointer 
    void * PTR;

    // ... 

} Robj;

 

type field

The recorded object type.

We usually use command type <key>, in fact, return to the properties of this field.

 

127.0.0.1:6379> set hello world
OK
127.0.0.1:6379> type hello
string
127.0.0.1:6379> rpush list 1 2 3
(integer) 3
127.0.0.1:6379> type list
list
...

 

That type the number of the type it? Look at the following table:

encoding field

Recording encoding target used (data structures), Reids data structure called the encoding.

We can see that we redis object encoding:

127.0.0.1:6379> object encoding hello
"embstr"
127.0.0.1:6379> object encoding list
"quicklist"
...

 

Since it is indicated that redisObjectis what the data structures used, it must also have a corresponding table:

We can see, Redis underlying encoding of the object points is very small, String type there are three, four other objects are achieved, respectively, are two different underlying data structure. They have a law, is to use ziplist, intset, embstrto achieve a small amount of data, once the huge amount of data, will be upgraded to skiplist, raw, linkedlist, htto achieve, I will carefully explain later.

3. Analysis of each individual object base encoding implementation (Data Structure)

3.1 string (string)

There are three string code: int, raw, embstr.

3.1.1 int

When the value of all digital string object, int coding will be used.

127.0.0.1:6379> set number 123455
OK
127.0.0.1:6379> object encoding number
"int"

3.1.2 embstr

Length of string or floating-point less than or equal 39 bytes will be used to store encoding embstr, embstr storage memory generally small and redis-time allocation of contiguous memory (high efficiency).

127.0.0.1:6379> set shortStr "long long time"
OK
127.0.0.1:6379> object encoding shortStr
"embstr"

 

3.1.2 raw

When the length of a string or floating-point greater than 39 bytes, to save on the use of SDS, encoded as raw, due to the uncertain values ​​of the byte size, the keys and values ​​of the distribution of each of two memory so allocated (also recovered twice), the same way it must not contiguous memory.

127.0.0.1:6379> set longStr "hello everyone, we dont need to sleep around to go aheard! do you think?"
OK
127.0.0.1:6379> object encoding longStr
"raw"

 

3.1.3 transcoding

As mentioned above, Redis automatically converting encoded to adapt and optimize the storage of data.

int->raw

Conditions: The digital object append the letters, the conversion occurs.

127.0.0.1:6379> object encoding number
"int"
127.0.0.1:6379> append number " is a lucky number"
(integer) 24
127.0.0.1:6379> object encoding number
"raw"

 

embstr->raw

Conditions: The embstr modify, redis will first convert it into a raw, before modify it. So embstr effectively read-only nature.

127.0.0.1:6379> object encoding shortStr
"embstr"
127.0.0.1:6379> append shortStr "(hhh"
(integer) 18
127.0.0.1:6379> object encoding shortStr
"raw"

 

3.2 List (list)

List of objects can be coded: ziplist or linkedlist.

1. ziplistCompression list does not know if you remember, is the zlbytes zltail zllen entry1 entry2 ..endstructure, entry节点there are pre-length、encoding、contentattributes, you can forget to see lower returns.

2. linkedlistSimilar doubly linked list, is the last chapter of knowledge.

3.2.1 transcoding

ziplist->linkedlist

Conditions: length of all the elements of the list of objects of the string 64 bytes or larger than or equal to the number of elements & listing 512. In contrast, less than 64 and less than 512 will be used instead ziplist linkedlist.

This threshold can be modified, modify the options: list-max-ziplist-valueandlist-max-ziplist-entriess

3.3 hash (hash)

Hash coding objects are: ziplist and hashtable

3.3.1 transcoding

ziplist-> hashtable

Conditions: hash keys and values ​​of all objects string length 64 bytes or the number of key-value greater than or equal & 512

This threshold also can be modified, modify the options: hash-max-ziplist-valueandhash-max-ziplist-entriess

3.4. Collection (set)

Encoding a collection of objects are: intset and hashtable

3.4.1 intset

1. Collection object all elements are integers

2. Number of elements in a collection of objects is not more than 512

3.4.2 transcoding

intset->hashtable

Conditions: & element are integers not equal to the number of elements is greater than 512

3.5. Ordered set (zset)

Ordered set of encoding used: ziplist and skiplist

It may well Qie, ziplist of entry only attribute content can store data, but also a collection of key-valueforms, how to store it?

The first node holds key, save the second node value and so on ...

3.5.1 Why use these two codes

1. If only to achieve, can not do the sort element does not support range lookups, can be done quickly find elements. With ziplist

2. If only to achieve, can not be done quickly find, but can be done to sort the elements, the scope of the operation. With skiplist

3.5.2 transcoding

ziplist-> skiplist

Condition: the number of elements of an ordered set of> = 128 & element comprising a length of> = 64

This threshold also can be modified, modify the options: zset-max-ziplist-valueandzset-max-ziplist-entriess

4. Garbage Collection

Why say it recovered memory, because redisObject a field:

typedef struct redisObject {

    // ...

    // reference count 
    int the refcount;

    // ... 

} Robj;

 

redis use reference counting garbage collection method (and the same jvm), are based on the use of a variable behavior of an object is counted.

• Initialized to 1

• Object being referenced, + 1

• Object reference elimination, -1

• Counter == 0, recycled object

5. Shared Object

5.1 reflects the shared objects

1. redis, the value is an integer value and equal two objects, the objects will redis sharing, and reference count +

2. redis start automatically generates an integer value of 0-9999 into shared memory.

5.2 Why object sharing

Save memory

5.3 Why not share string

the cost is too high.

Verify integer equal only O (1) time complexity, the authentication string to O (n).

6. Long idle objects

Finally, redisObject there is a field that records the time the object was last accessed:

typedef struct redisObject {

    // ...

    unsigned lru:22;

    // ... 

} Robj;

 

Because of the time of this field records the object was last accessed, so it can be used to see how long the object is not in use, namely: with the current time -lru

127.0.0.1:6379> object idletime hello
(integer) 5110

 

It also relates to the hot data redis achieve, if we choose lr algorithm, when the memory is idle longer exceeds the threshold will be higher target release, reclaim memory.

references:

1. "Redis Design and Implementation" with Huang Jianhong

2. http://redisbook.com/index.html