1 List
data structure
Before version 3.2, Redis list list used two data structures as the underlying implementation:
- Ziplist
- Doubly linked list linkedlist
because doubly linked list occupies more memory than compressed list, so when creating a new list key, the list will give priority to using the compressed list, and only when necessary, convert from compressed list to doubly linked list. - Convert compressed list into doubly linked list condition
-
Redis uses redis_encoding_ziplist encoding by default when creating a new list. When any of the following conditions is met, the list will be converted to redis_encoding_linkedlist encoding:
Attempt to add a new string value to the list, and the length of this string exceeds server.list_max_ziplist_value (the default value is 64).
- The ziplist contains more nodes than server.list_max_ziplist_entries (default value is 512).
Note: These two conditions can be modified, in redis.conf:
list-max-ziplist-value 64
list-max-ziplist-entries 512
Command can be used to view help, help @list
1.1 Stack
You can use the list to achieve the function of the stack, named in the same direction, push on the left, and pop on the left;
Or push on the right and pop up on the right
1.2 Queue
You can use list to implement queues, reverse commands, push on the left and pop up on the right
Or push on the right and pop up on the left
1.3 Array
1.4 Blocking queue fifo
lpush + brpop
Rpush + blpop
2 hash
Hash is a mapping table between field and value of string type. Hash is especially suitable for storing objects.
2.1 Use simple commands, use view help, help @hash
Hset
Hget
Hmset hmget hgetall
Hkeys whale hlen
Hincrby hincrbyfloat logarithmic operations
3 Set
In Redis, the Set type is an unsorted collection of characters. Like the List type, operations such as adding, deleting, or determining whether an element exists can be performed on the data value of this type. It should be noted that the time complexity of these operations is O(1), that is, the operations are completed in constant time. The maximum number of elements that a set can contain is 4294967295. Unlike the List type, duplicate elements are not allowed in the Set collection. In other words, if you add the same element multiple times, only one copy of the element will be kept in the Set. Compared with the List type, the Set type still has a very important feature in function, that is, the aggregation calculation operation between multiple Sets is completed on the server side, such as unions, intersections, and differences. Since these operations are completed on the server side, the efficiency is extremely high, and it also saves a lot of network IO overhead.
3.1 Basic operation
S add adds an element to the set (the elements in the set are unordered and unique)
S card returns the number of collection elements
S members view all elements in the collection
SRandMember
If you don’t fill in the returned number, the default is 1. If the filled number is greater than the size of the collection, all elements of the collection are returned
If you fill in a negative number, if the absolute value is greater than the size of the collection, then there will be multiple occurrences of one element in the return value.
If the key does not exist, return nil
Other simple commands are not listed here.
3.2 Aggregate calculation
SUNION: union
SUNIONSTORE: Save the union to a new collection (if the result set is saved to an existing collection, it will overwrite the future data collection)
Intersection:
sinter/sinterstore
sdiff/sdiffstore difference set
sdiff [set1] [set2]
function: delete elements in set1 that also exist in set2, and return to set1 after the delete operation
sdiffstore [set1] [set2] [set3]
function: save the result of sdiff [set2] [set3] in set1
3.3 Random events, application scenarios
sRandMember ，sPop
The functions of these two commands are very similar, and both return an element value from the set. The difference is that sRandMember will not delete the returned elements from the collection, but sPop will delete them. These two commands can implement different lottery algorithms respectively.
For example, there are 100 elements in the set, with values ranging from number 1 to number 100. If we define the number 1 to be drawn, it means winning.
If you use sRandMember, no matter how many times you have drawn before, the probability of the next draw is 1%. With sPop, the probability of each draw is different. The probability of the first person to draw is 1%, when the first person does not draw, the probability of the second person to draw is 1/99, and so on.
There are two cases
1 The number of prizes exceeds the number of draws, such as 10 prizes, 3 people
Sadd k1 tom baby xx
2 The number of prizes is less than the number of draws, and the prize cannot be repeated. It is applicable to general companies after the year
Sadd k1 tom baby xx
Put everyone's names into the set, take out one at a time, and delete them to ensure fairness.
4 sorted set
Sorted Set is a bit like a combination of Set and Hash.
Like Set, the elements in it are unique, and the type is String, so it can be understood as a Set.
But the elements in the Set are unordered, and the elements in the Sorted Set have a floating-point value called a score, so this is a bit like Hash, because each element is mapped to a value.
Sorted Set is ordered, and the rules are as follows:
if A.score> B.score, then A> B.
If A.score == B.score, then the size of A and B is determined by comparing the strings, and the strings of A and B will not be equal, because the values in the Sorted Set are all unique.
4.1 Basic command operation
ZADD
ZADD can add elements to Sorted Set, just like the SADD command of Set
ZRANGE，ZREVRANGE
By default, ZRANGE displays the elements of the Sorted Set from low to high score
If you want to display the score from high to low, you need to use ZREVRANGE
You can also display the score together, using the parameter WITHSCORES
ZRANGEBYSCORE can display Sorted Set by range, the format is zrangebyscore key, lower limit of score, upper limit of score
ZRANK ， ZREVRANK
ZRANK command can get the ranking of the element, ZREVRANK vice versa
4.2 Set operations
I won’t list them one by one here.
4.3 Sorting realization principle
Skip List is a randomized data structure, based on parallel linked lists, simple to implement, and the complexity of insertion, deletion, and search is O(logN) (in most cases), because its performance is comparable to red-black trees and the implementation is relatively Simple, so in many famous projects, skip tables are used to replace red-black trees. For example, the underlying storage structure of LevelDB and Reddis uses SkipList.
There are currently three commonly used key-value data structures: Hash table, red-black tree, and SkipList, each of which has different advantages and disadvantages:
Hash table: Insertion and search are the fastest, O(1); if a linked list is used, lock-free can be achieved; data ordering requires explicit sorting operations.
Red-black tree: Insertion and search are O(logn), but the constant term is small; the complexity of lock-free implementation is very high, and generally requires locking; the data is naturally ordered.
SkipList: Insertion and search are O(logn), but the constant item is larger than the red-black tree; the underlying structure is a linked list, which can be implemented without locks; the data is naturally ordered.
A skip table should have the following characteristics:
1. A jump table should consist of several levels;
It is usually 10-20 layers, and the default is 12 layers in leveldb.
2. The 0th level of the jump table contains all the elements;
And the node values are ordered.
3. Each layer is an ordered linked list;
The higher the number of layers, the sparser it should be, so that many unqualified data can be'skip' in the higher layers.
4. If the element x appears in the i-th layer, all layers smaller than i include x;
5. Each node contains the key and its corresponding value and an array of pointers to the next node of the nth layer of the node
x->next[level] represents the next node of x at level level
skiplist query process
Query the previous value of the first node greater than vx to see if it is equal. If they are equal, they will exist, otherwise the next layer will be searched until the number of layers is 0.
Take the existing data 13, 22, 75, 80, 99 as examples
Start at the highest level (here 2)
1. If level2 finds the node Node75 is less than 80, and level2.Node75->next is greater than 80, then enter the level1 search (the node between 13~75 has been skipped here (22),
2、level1.Node75 < 80 < level1.Node75->next，进入level0
3. Level0.Node75->next is equal to 80, find the node
The insertion process of skiplist
Suppose you insert a new key value key, the value is 84, and the level is the current level
1. From the highest level, find the previous value of each node larger than 84 and store it in prev[level].
prev[2] = leve2.Node75
prev[1] = leve1.Node75
prev[0] = level0.Node80
2. Initialize a new node 84
3. Randomly choose a height h for x, choose 2 here
4、x->next[0..h-1] = prev[0..h-1]->next
5、prev[0..h-1]->next[0..h-1] = x
(Steps 4 and 5 are operations for inserting nodes in the linked list)
skiplist delete operation
The delete operation is similar to the insert operation, including the following 3 steps: 1. Find the node that needs to be deleted 2. Delete the node 3. Adjust the pointer