Overview
There are six data types at the bottom of Redis including: simple dynamic string, doubly linked list, compressed list, hash table, skip table and integer array . The relationship between the five data types of these six data structures is as follows:
- String: simple dynamic string
- List: doubly linked list, compressed list
- Hash: compressed list, hash table
- Sorted Set: compressed list, skip list
- Set: Hash table, integer array
Correspondence between data types and underlying data structures
The characteristics of each data structure are different, and the operation time is also different.
Time complexity of data structures
2. Data structure
As can be seen from the above figure, the underlying data structure of Redis consists of simple dynamic strings, doubly linked lists, compressed lists, hash tables, jump tables, and integer arrays. Basically, everyone is familiar with hash tables and integer arrays. The following focuses on the rest of the data structures.
1. Simple dynamic string (SDS)
Structure: alloc, len, buf
Simple dynamic string structure
buf: byte array, saving actual data. In order to indicate the end of the byte array, Redis will automatically add a "\0" at the end of the array, which will take up an additional 1 byte of overhead.
len: 4 bytes, indicating the used length of buf.
alloc: It also occupies 4 bytes, indicating the actual allocation length of buf, which is generally greater than len.
So what is the difference between SDS and C string? There are two main differences:
(1) The time complexity of getting the string length is O(1)
(2) When modifying the string, it will first check whether the length is long enough, and if it is not enough, it will be extended to avoid buffer overflow
2. Linked list
Redis uses a two-way acyclic linked list and has the following characteristics:
(1) Double-ended: The linked list has references to the front node and the back node, and the time complexity of obtaining these two nodes is O(1).
(2) Acyclic: the prev pointer of the head node and the next pointer of the tail node both point to NULL, and the access to the linked list ends with NULL.
(3) With a linked list length counter: the time complexity of obtaining the linked list length through the len attribute is O(1).
(4) Polymorphism: Linked list nodes use void* pointers to save node values, which can save various types of values.
3. Compressed list
The compressed list (ziplist) was developed by Redis to save memory. It is a sequential data structure composed of a series of specially encoded consecutive memory blocks. A compressed list can contain any number of nodes (entry), and each node can save an A byte array or an integer value. The compression list does not use a certain algorithm to compress the data, but encodes the data in a continuous memory area according to certain rules, with the purpose of saving memory.
A compressed list is actually similar to an array, and each element in the array corresponds to saving a piece of data. Different from the array, the compressed list has three fields at the header: zlbytes, zltail and zllen, which respectively represent the length of the list, the offset at the end of the list and the number of entries in the list; the compressed list also has a zlend at the end of the table, indicating end of list;
We want to find and locate the first element and the last element, which can be directly located by the length of the three header fields, and the complexity is O(1). When searching for other elements, it is not so efficient, and can only be searched one by one, and the complexity at this time is O(N).
Compressed table search process
4. Jump table
On the basis of the linked list, the skip list adds a multi-level index. Through several jumps in the index position, the data can be quickly located. The time complexity is O(logN). Compared with the linked list, the query efficiency of the skip list is greatly improved. O(logn).
Jump table search process
3. Basic data structure of Redis data type
1. String
1.1 Internal structure of String
Redis does not directly use the string representation in the C language, but builds a string itself called " simple dynamic string " (SDS). Among them, the string in the C language is only used as a string literal (usually used in places where the string value does not need to be modified).
The structural implementation of String is similar to ArrayList in Java (an initial array of size 10 is constructed by default). This is the idea of redundant allocation of memory, also called pre-allocation; this idea can reduce the performance consumption caused by expansion. .
Internal structure of String
1.2 Data encoding used by String
If the number is stored, the int type encoding is used; if it is not a number, the raw encoding is used;
1.3 Usage scenarios
(1) Simple character cache
(2) Distributed lock
(3) Counting function——》Counting service
2. List
2.1 Internal structure of List
The Redis list is equivalent to the LinkedList in the Java language. It is a doubly linked list data structure (but this structure is more cleverly designed, which will be introduced later) and supports sequential traversal. The insertion and deletion operations of the linked list structure are fast, the time complexity is O(1), the query is slow, and the time complexity is O(n).
Internal structure of List
2.2 Data encoding used by List
If the length of the string and the number of elements are less than a certain range, use ziplist encoding. If any condition is not met, it will be converted to linkedlist encoding.
2.3 Usage scenarios
(1) Use List to implement stack and queue
(2) redis as a message queue (redis is not recommended as a message queue)
(3) List cache
3. Hash (dictionary)
3.1 Internal structure of Hash
Redis's hash (dictionary) is equivalent to HashMap in Java language. It is an unordered dictionary distributed according to hash values. The internal elements are stored in key-value pairs.
The implementation of hash (dictionary) is also consistent with the structure of HashMap (JDK1.7) in Java. Its data structure is also a two-dimensional structure composed of array + linked list. The node elements are hashed on the array. If a hash collision occurs, it is used Linked lists are concatenated on array nodes.
The internal structure of Hash
3.2 Data encoding used by Hash
The length of the key and value strings in the key-value pair stored in the hash object is less than a certain value and the key-value pair.
3.3 Usage scenarios
(1) Storage object
4. Set
4.1 Internal structure of Set
The set (collection) of Redis is equivalent to the HashSet in the Java language, and its internal key-value pairs are unordered and unique. It internally implements a special dictionary whose values are all null.
After the last element in the collection is removed, the data structure is automatically deleted and the memory is reclaimed.
Internal structure of Set
4.2 Data encoding used by Set
Use intset encoding to store elements as integers and the number of elements is less than a certain range, and use hashtable encoding if any condition is not met.
4.3 Usage scenarios
(1) Tags, social networking, search for people with common interests, intelligent recommendation
(2) Lottery
(3) Likes in Moments
5. Zset (ordered set)
5.1 Internal structure of Zset
Zset (ordered set) is the most frequently asked data structure in Redis. It is similar to the combination of SortedSet and HashMap in the Java language. On the one hand, it uses set to ensure the uniqueness of the internal value, and on the other hand, it sorts by the score (weight) of the value. This sorting function is realized through Skip List (jump list). After the last element value of zset (ordered set) is removed, the data structure is automatically deleted and the memory is recycled.
Internal structure of Zset
5.2 Data encoding used by Zset
If the number of elements stored in the zset object is less than or the member length is less than a certain value, ziplist encoding is used. If any condition is not met, skiplist encoding is used.