1，String： raw，int , embstr
2，hash： hashtable，ziplist
3：list：linkedlist，ziplist
4：set：hashtable，intset
5: zset: skiplist, ziplist
Design benefits:
1: coding can improve internal and external data structures and command has no effect
2: a variety of internal coding implementations can play to their strengths in different scenarios, such as ziplist more to save memory, but performance will drop list elements for a long time, then Redis will be implemented according to the conversion list of configuration options for the type of memory linkedlist
String
Redis using SDS ( "Simple Dynamic String") This structure stored string,
1: code defines five SDS structure:
1：struct __attribute__ ((__packed__)) sdshdr5 { unsigned char flags; char buf[]; };
2：sdshdr8
3: sdshdr16
4：sdshdr32
5：sdshdr64
2: The main function in the field:
len: length of the string (length actually used)
alloc: allocating memory size
flags: flag, represents the lower three types, the remaining five unused
buf: char array
3: Coding
int Code: Save the long-type 64-bit signed integer, the Redis key will be converted to long storage type, coding type corresponding to OBJ_ENCODING_INT
embstr encoding: length less than 44 bytes to save a character string (OBJ_ENCODING_EMBSTR)
raw Code: saving a string of length greater than 44 bytes (OBJ_ENCODING_RAW)
ziplist
1: ziplist structure
zlbytes: indicates the number of bytes occupied ziplist, resize operation performed at the time of use
zltail: represents the offset of the last node, but also to avoid traversing the whole list
zllen: ziplist represents the number of nodes (more than 65,535 nodes, zllen invalid field values, need to traverse to get the real number)
zlend: End identifier represents ziplist
2: ziplist node data structure (abstract)
Each compressed node list by previous_entry_length, encoding, content of three components (the field does not refer to the actual structure)
previous_entry_length: the length of the previous node, for traversing forward by the rear, according to the length of the previous node, or may require a five bytes.
encoding: node data records stored type and data length.
content: the content of the data stored in the node.
ziplist a period of continuous use of memory to store data, memory fragmentation is reduced compared hashtable, and a pointer memory footprint. And when the node is less, ziplist more likely to be loaded into the CPU cache.
3: ziplist advantages
Memory footprint less likely to be loaded into the CPU cache
Compact reduce memory fragmentation
4: ziplist shortcomings
Chain Update (head insertion or deletion operation on the head mainly when saving transmission type object list with ziplist, objects do not need to update the stored hash head.)
Each insert or modify operation initiated realloc have a greater probability of causing memory copies, thereby decreasing performance.
Once the cost of memory copy, memory copy corresponding increase occurs, because a larger data to be copied.
When ziplist too much data item on it to find specific performance data items will become very low because the lookup on ziplist need to be traversed.
From the query complexity becomes O (1) O (N) (when the object is stored hash),
5: ziplist Configuration
redis.conf configuration:
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
When the number of types of elements smaller than the hash hash-max-ziplist-entries configuration (default 512), while all values are hash-max-ziplist-value configuration (default 64 bytes) when, as a hash ziplist uses the Redis internal implementation, ziplist use of more compact storage to achieve a plurality of successive elements, so that more excellent in terms of saving memory than hashtable.
Redis of hash reason for this design is that when ziplist become great, it has the following disadvantages:
ship list
1: skipList jump table
skipList multilayer chain model, a node will assign a random number of layers (Level) pointing towards the record pointer after single or multiple nodes, only need to modify the pointer insert operation before and after the insertion node, without the need for many nodes Adjustment. Reducing the time complexity, an average of O (logN), worst case complexity of O (N), its performance generally comparable balanced tree.
2: Probability
The higher the number of layers of nodes to produce, the lower the probability. Quantitative analysis is as follows:
The number of layers of at least one node. Node 1 is greater than the number of layers satisfy a probability distribution.
Node layers is exactly equal probability 1-p.
The probability of the node number of layers greater than or equal to 2 p, the number of layers and node 2 is exactly equal to the probability p (1-p).
The number of layers is greater than the probability of the node 3 is equal to p2, the number of layers and the node 3 is exactly equal to the probability p2 (1-p).
Node layers 4 greater than or equal to the probability p3, and node probability of exactly the number of layers 4 of p3 (1-p).
3: Comparison of skiplist and balanced trees, hash tables
- and various skiplist balanced tree (e.g., AVL, red-black trees, etc.) elements are ordered, and the hash table is not ordered. Therefore, the hash table can only do a single key lookup, and unfit for range search. The so-called scope of the search, referring to those who find all the nodes in size between the two values specified.
- Find the time to do range, balanced tree operation is more complicated than skiplist. After balancing the trees, we find the small value of the specified range, but also the order in order to traverse the other nodes continue to seek no more than a large value. If you do not balance the tree a certain transformation, where the preorder is not easy to achieve. And be very simple to find the range on skiplist, just after finding a small value, the list of the first layer to traverse several steps can be achieved.
- Balanced tree insertion and deletion operations may lead to adjustment of the sub-tree logic complexity, and skiplist insertions and deletions only need to modify the pointer adjacent nodes, simple and fast.
- From the memory foot print, skiplist more flexible than some of balanced tree. Generally, each node comprises a balanced tree 2 pointer (pointing to the left and right subtree, respectively), while the number of pointers skiplist each node contains an average of 1 / (1-p), depending on the size of the parameter p. If, as Redis in the realization of the same, take p = 1/4, then each node contains an average of 1.33 pointer, balanced tree than an advantage.
- Finding a single key, skiplist and balanced trees are time complexity of O (log n), roughly; and hash table while maintaining a low probability value of hash conflicts, close to find time complexity O (1) performance higher. So various Map or dictionary structure we normally use, mostly hash table based implementation.
- Compare up from the algorithm implementation difficulty, skiplist much simpler than balanced tree.
