Design and implementation of Redis
Data structures and internal encoding
What the type command actually returns is the data structure type of the current key, which are: string (string) hash (hash), list (list), set (set), zset (ordered set), but these are only Redis external data structure.
In fact, each data structure has its own underlying internal encoding implementation, and there are multiple implementations, so that Redis will choose the appropriate internal encoding in the appropriate scenario.
Each data structure has more than two internal encoding implementations. For example, the list data structure contains two internal encodings: linkedlist and ziplist. At the same time, some internal encodings, such as ziplist, can be used as internal implementations of various external data structures. The internal encoding can be queried through the object encoding command.
Redis has two advantages in designing this way:
First, the internal coding can be improved without affecting external data structures and commands. In this way, once better internal coding is developed, there is no need to change the external data structures and commands. For example, Redis3.2 provides quicklist, which combines ziplist and linkedlist. It provides a better internal coding implementation for the list type, which is basically invisible to external users.
Second, multiple internal encoding implementations can exert their own advantages in different scenarios. For example, ziplist saves memory, but when there are many list elements, the performance will decrease. At this time, Redis will change the list type according to the configuration options. The internal implementation converts to linkedlist.
redisobject object
All value objects stored by Redis are internally defined as redisobject structures, and the internal structure is as shown in the figure.
The data stored by Redis is encapsulated using redis0object, including all data types including string, hash, list, set, and zset. Understanding redis0object is very helpful for memory optimization. The following is a detailed description of each field:
type field
type field: indicates the data type used by the current object. Redis mainly supports 5 data types: string, hash, list, set, zset. You can use the type {key} command to check the type of the object. The type command returns the value object type, and the keys are all string types.
encoding field
Encoding field : Indicates the internal encoding type of Redis. Encoding is used internally in Redis and represents which data structure is used within the current object. Understanding the internal encoding method of Redis is very important for optimizing memory. There are obvious differences in the memory usage of the same object using different encodings.
lru field
lru field: records the time when the object was last accessed. When maxmemory and maxmemory-policy=volatile-lru or allkeys-lru are configured, it is used to assist the LRU algorithm in deleting key data. You can use the object idletime {key} command to view the idle time of the current key without updating the lru field.
You can use the scan + object idletime command to query in batches which keys have not been accessed for a long time, and find out the keys that have not been accessed for a long time and clean them up, which can reduce memory usage.
refcount field
Refcount field: records the number of times the current object has been referenced, and is used to reclaim memory based on the number of references. When refcount=0, the current object space can be safely reclaimed. Use object refcount(key} to get the current object reference. When the object is an integer and the range is [0-9999], Redis can use shared objects to save memory.
PS interview question, Redis's object garbage collection algorithm-----reference counting method.
*ptr field
*ptr field: related to the data content of the object. If it is an integer, the data is stored directly; otherwise, it represents a pointer to the data.
For the new version of Redis string data with length <= 44 bytes, the string sds and redisobject are allocated together, so only one memory operation is required.
PS : In high concurrent writing scenarios, if conditions permit, it is recommended that the string length be controlled within 44 bytes to reduce the number of memory allocations to create a redisobject, thereby improving performance.
