Power Node Redis7 Notes - Chapter 3 Redis Commands

3 Redis commands

Redis can be divided into three categories according to the objects operated by the commands: commands for basic operations on Redis, commands for Key operations, and commands for Value operations.

3.1 Redis basic commands

First enter the Redis command line client through the redis-cli command, and then run the following command.

3.1.1 Heartbeat command ping

Type the ping command, and you will see a PONG response, which means that the connection between the client and Redis is normal. This command is also known as the heartbeat command.

3.1.2 Read and write key-value commands

set key value will write the specified key-value to DB. get key will read the value of the specified key. For more set and get command formats, we will learn in detail later.

3.1.3 DB switching select

Redis has 16 databases by default. This can be seen visually in the Redis Desktop Manager (RDM) graphical client.

By default, DB No. 0 is used, and the DB can be switched by selecting the db index. For example, the following command will switch to DB3 and will write age-23 to DB3.

And this result can be seen intuitively in RDM.

3.1.4 Check the number of keys dbsize

The dbsize command can view the number of keys in the current database.

From the above inspection, there are 2 keys in DB0, no key in DB1, and 1 key in DB3.

3.1.5 Delete the data flushdb in the current database

The flushdb command only deletes the data in the current database and does not affect other databases.

3.1.6 Delete all data in the database command flushall

The flushall command can delete all data in all libraries. So use this command with caution.

3.1.7 Exit client command

Use the exit or quit command to exit the Redis command line client.

3.2 Key operation commands

The data stored in Redis is a Map as a whole, its key is String type, and the value can be String, Hash table, List, Set and other types.

3.2.1 keys

• Format: KEYS pattern
• Function: Find all keys matching the given pattern pattern, pattern is a regular expression.
• Note: KEYS is very fast, but using it in a large database may block the service of the current server. Therefore, this command is generally not used in the production environment, and the scan command is used instead.

3.2.2 exists

• Format: EXISTS key
• Function: Check if the given key exists.
• Description: If the key exists, return 1, otherwise return 0.

3.2.3 of

• Format: DEL key [key ...]
• Function: Delete one or more given keys. Keys that do not exist are ignored.
• Description: Return the number of deleted keys.

3.2.4 rename

• Format: RENAME key newkey
• Function: Rename key to newkey.
• Description: When the key is the same as newkey, or the key does not exist, an error is returned. When newkey already exists, the RENAME command will overwrite the old value. It prompts OK when the name is changed successfully, and returns an error when it fails.

3.2.5 move

• Format: MOVE key db
• Function: Move the key of the current database to the given database db.
• Explanation: If the current database (source database) and the given database (target database) have a given key with the same name, or the key does not exist in the current database, then MOVE has no effect. Returns 1 if the move was successful, and 0 if it failed.

3.2.6 type

• Format: TYPE key
• Function: Return the type of the value stored in the key.
• Description: The return value has the following six types
• none (key does not exist)
• string (string)
• list (list)
• set (collection)
• zset (ordered set)
• hash (hash table)

3.2.7 expire and pexpire

• Format: EXPIRE key seconds
• Function: Set the lifetime for a given key. When the key expires (time to live is 0), it is automatically deleted. The time unit of expire is seconds, and the time unit of pexpire is milliseconds. In Redis, keys with time-to-live are said to be "volatile".
• Description: Return 1 if the survival time is set successfully. Returns 0 if the key does not exist. The rename operation does not change the lifetime of the key.

3.2.8 ttl与pttl

• Format: TTL key
• Function: TTL, time to live, returns the remaining time to live for a given key.
• Note: There are three possible return values:
• Returns -2 when key does not exist.
• Returns -1 when the key exists but no remaining lifetime is set.
• Otherwise, returns the remaining lifetime of the key. The unit of time returned by the ttl command is seconds, and the unit of time returned by the pttl command is milliseconds.

3.2.9 persist

• Format: PERSIST key
• Function: Remove the lifetime of a given key, and convert this key from "volatile" to "durable".
• Note: When the lifetime is successfully removed, 1 is returned; if the key does not exist or the lifetime is not set for the key, it returns 0.

3.2.10 randomkey

• Format: RANDOMKEY
• Function: Randomly return (do not delete) a key from the current database.
• Description: When the database is not empty, return a key. Returns nil when the database is empty.

3.2.11 scan

• 格式：SCAN cursor [MATCH pattern] [COUNT count] [TYPE type]
• Function: used to iterate over the database keys in the database. The meaning of each option is:
• cursor: The cursor at the start of this iteration.
• pattern : The pattern of the key to be matched in this iteration.
• count : How many elements to return from the dataset in this iteration, the default value is 10.
• type: The type of value to be returned in this iteration, defaulting to all types.

The SCAN command is a cursor-based iterator: each time the SCAN command is called, it will return an array containing two elements to the user, the first element is a new cursor for the next iteration, and the second The elements are an array, which contains all the elements being iterated. The user needs to use this new cursor as the cursor parameter of the SCAN command in the next iteration, so as to continue the previous iteration process. When the cursor parameter of the SCAN command is set to 0, the server will start a new iteration. If the new cursor returns 0, the iteration is over.

• Description: Incremental iteration with intermittent, negative, out-of-range, or other abnormal cursors will not crash the server.

When the amount of data is large, the specification of count may not work, and Redis will automatically adjust the number of traversals each time. Since the scan command only returns a small number of elements each time it is executed, this command can be used in a production environment without the server blocking problem caused by the KEYS command.
The algorithm used by the incremental iteration command only guarantees that the iteration will stop when the size of the data set is bounded. In other words, if the size of the iterated data set continues to grow, the incremental iteration command may never stop. Unable to complete a full iteration. That is, when a data set is getting bigger and bigger, more and more work needs to be done to access all the elements in the data set. Whether an iteration can end depends on whether the speed at which the user executes the iteration is faster than the speed at which the data set grows. quick.

• Related commands: In addition, there are 3 scan commands for traversing the three types of values.
  - hscan: belongs to the set of Hash-type Value operation commands, used to traverse all field-value pairs of the specified Hash table in the current db.
  - sscan: belongs to the set of Set-type Value operation commands, used to traverse all elements of the specified set in the current db - zscan
  : belongs to the set of ZSet-type Value operation commands, used to traverse all elements of the specified ordered set in the current db (value and element value)

3.3 String Value operation command

The Value of Redis stored data can be a String type data. Value of String type is the most basic and common type in Redis. Any data can be stored in Value of String type, including numeric or even binary pictures, audio, video, serialized objects, etc. The maximum size of a String type Value is 512M.

3.3.1 set

• 格式：SET key value [EX seconds | PX milliseconds] [NX|XX]
• Function: In addition to directly setting the value of key as value, SET can also specify some parameters.
  • EX seconds: Set the expiration time for the current key, in seconds. Equivalent to the SETEX command.
  • PX milliseconds: Set the expiration time for the current key, in milliseconds. Equivalent to the PSETEX command.
  • NX: If the specified key does not exist, it will be set successfully, which is used to add the specified key. Equivalent to SETNX command.
  • XX: The specified key must exist to be set successfully, and is used to update the value of the specified key.
• Note: If the value string contains spaces, the string needs to be enclosed in double quotes or single quotes, otherwise it will be considered that the number of parameters of the set command is incorrect and an error will be reported.

3.3.2 setex and psetex

• Format: SETEX/PSETEX key seconds value
• Function: set expire, which not only specifies the value for the key, but also sets the survival time for it. The unit of setex is seconds, and the unit of psetex is milliseconds.
• Description: If the key already exists, the old value will be overwritten. This command is similar to the following two commands. The difference is that SETEX is an atomic operation, and the two actions of associating the value and setting the lifetime will be completed at the same time. This command is very practical when Redis is used as a cache.

SET key value
EXPIRE key seconds # Set the survival time

3.3.3 setnx

• Format: SETNX key value
• Function: SET if Not eXists, set the value of the key to value if and only if the key does not exist. If the given key already exists, SETNX does nothing. If successful, return 1, otherwise, return 0.
• Description: This command is equivalent to set key value nx

3.3.4 getset

• Format: GETSET key value
• Function: Set the value of the given key to value, and return the old value of the key.
• Description: When the key exists but is not a string type, return an error; when the key does not exist, return nil.

3.3.5 mset and msetnx

• Format: MSET/MSETNX key value [key value ...]
• Function: Set one or more key-value pairs at the same time.
• Note: If a given key already exists, MSET will overwrite the original old value with the new value. If this is not the effect you want, please consider using the MSETNX command: it will only be used when all given keys do not exist In this case, perform the setting operation. MSET/MSETNX is an atomic operation. All given keys will be set at the same time. It is impossible for some given keys to be updated while other given keys remain unchanged. The command never fails.

3.3.6 mget

• Format: MGET key [key ...]
• Function: Return all (one or more) values ​​of the given key.
• Explanation: If a certain key does not exist in the given key, then this key returns the special value nil. Therefore, the command never fails.

3.3.7 append

• Format: APPEND key value
• Function: If the key already exists and is a character string, the APPEND command will append the value to the end of the original value of the key. If the key does not exist, APPEND simply sets the given key to the value, just like executing SET key value.
• Description: After appending value, the length of the string in key.

3.3.8 increase and decrease

• Format: INCR key or DECR key

• Function: increment, auto-increment. Increments the numeric value stored in key by one.

  decrement，自动递减。将key中存储的数字值减一。
  

• Note: If the key does not exist, the value of the key will be initialized to 0 first, and then the increment/decrement operation will be performed. Returns an error if the value cannot be represented as a number. If executed correctly, returns the incremented/decremented value.

3.3.9 incrby and decrby

• Format: INCRBY key increment or DECRBY key decrement
• Function: Increase/decrease the specified value of the digital value stored in the key, this value can only be an integer, it can be a negative number, but it cannot be a decimal.
• Note: If the key does not exist, the value of the key will be initialized to 0 first, and then the increase/decrease operation will be performed. Returns an error if the value cannot be represented as a number. If done correctly, returns the incremented/decremented value.

3.3.10 incrbyfloat

• Format: INCRBYFLOAT key increment
• Function: add the floating-point number increment to the value stored in the key.
• Instructions: Same instructions as before. There is no decrbyfloat command, but the negative value of increment can achieve the effect of subtraction.

3.3.11 strl

• Format: STRLEN key
• Function: Return the length of the string value stored in key.
• Description: When the key does not store a string value, return an error; when the key does not exist, return 0.

3.3.12 cramped

• Format: GETRANGE key start end
• Function: Return the substring of the string value in the key, the interception range of the string is determined by the two offsets of start and end, including start and end.
• Note: end must be greater than start. Negative offsets are supported, which means counting from the end of the string, -1 means the last character, -2 means the penultimate character, and so on.

3.3.13 setrange

• Format: SETRANGE key offset value
• Function: Replace the string value str stored by the given key with the value parameter, starting from the offset offset.
• Note: When the offset value is greater than the length of str, the middle is filled with zero bytes \x00, that is, 0000 0000 bytes are filled; the non-existing key is treated as an empty string.

3.3.14 Bit manipulation commands

The commands containing BIT in the name are all commands for operating binary bits, such as setbit, getbit, bitcount, bittop, and bitfield, and these commands are not commonly used.

3.3.15 Typical Application Scenarios

There are many application scenarios where the value is of String type. Here are just examples of such typical application scenarios:

3.3.15.1 Data cache

Redis is used as the data cache layer, and MySQL is used as the data storage layer. The application server first obtains data from Redis. If there is no data in the cache layer, the data is obtained from MySQL, stored in the cache layer, and then returned to the application server.

3.3.15.2 Counters

Write a key whose value is a numeric value in Redis as a platform counter, video playback counter, etc. Every time a valid client visits, or every time a video is played, the counter in Redis is directly modified, and then persisted to other data sources in an asynchronous manner, such as MySQL.

3.3.15.3 Sharing Sessions

For a distributed application system, if the Session data such as user login information is saved in the server that provides login services, then if the user submits requests such as collection and payment again, problems may occur: when providing services such as collection and payment The server does not have the session data of the user, so the user needs to log in again. For users, this is unacceptable.
At this point, all the session data of all users in the system can be saved in Redis. After the user submits a new request, the system will first search for the corresponding session data in Redis. If it exists, then perform related operations, otherwise jump to log in page. This way it doesn't raise the "re-login" problem.

3.3.15.4 Speed ​​limiter

Nowadays, in order to prevent DoS (Denial of Service, denial of service) attacks, many platforms generally restrict an IP from being accessed more than n times within one second. And Redis can combine the key expiration time and the incr command to complete the speed limit function and act as a speed limiter.
Note that it cannot prevent DDoS (Distributed Denial of Service) attacks.

// 客户端每提交一次请求，都会执行下面的代码

// 等价于set 192.168.192.55 1 ex 60 nx
// 指定新ip作为key的缓存过期时间为60秒
Boolean isExists = redis.set(ip, 1, “EX 60”, “NX”);
if(isExists != null || redis.incr(ip) <= 5) {
    
    
    // 通过
} else {
    
    
    // 限流
}

3.4 Hash type Value operation command

The Value of Redis stored data can be a Hash type. The Hash type is also called Hash table, dictionary, etc.
The Hash table is a mapping table Map, which is also composed of key-value pairs. In order to distinguish it from the overall key, the key here is called field, and the value is called value. Note that the field-value pairs in the Hash table of Redis are all of String type.

3.4.1 hset

• Format: HSET key field value
• Function: Set the value of the field field in the hash table key to value.
• Description: If the key does not exist, a new hash table is created and HSET is performed . If the field field already exists in the hash table, the old value will be overwritten. Returns 1 if field is a new field in the hash table and the value is set successfully. Returns 0 if the field already exists in the hash table and the old value has been overwritten by the new value.

3.4.2 hget

• Format: HGET key field
• Function: Return the value of the given field field in the hash table key.
• Description: When the given domain does not exist or the given key does not exist, return nil.

3.4.3 hmset

• Format: HMSET key field value [field value ...]
• Function: Set multiple field-value (field-value) pairs to the hash table key at the same time.
• Description: This command overwrites existing fields in the hash table. If the key does not exist, an empty hash table is created and the HMSET operation is performed. If the command is executed successfully, return OK. Returns an error when key is not of hash type.

3.4.4 hmget

• Format: HMGET key field [field ...]
• Function: Return the values ​​of one or more fields in the hash table key in the given order.
• Description: Returns a nil value if the given field does not exist in the hash table. Since a non-existent key is treated as an empty hash table, an HMGET operation on a non-existent key will return a table with only nil values.

3.4.5 hgetall

• Format: HGETALL key
• Function: Return all fields and values ​​in the hash table key.
• Explanation: In the return value, immediately after each domain name (field name) is the value of the field (value), so the length of the return value is twice the size of the hash table. If key does not exist, returns an empty list. If the key contains a large number of elements, this command may block the Redis service. Therefore, this command is generally not used in the production environment, but the hscan command is used instead.

3.4.6 hsetnx

• Format: HSETNX key field value
• Function: Set the value of the field field in the hash table key to value, if and only if the field field does not exist.
• Note: If the field field already exists, this operation is invalid. If the key does not exist, a new hash table is created and the HSETNX command is executed.

3.4.7 hdel

• Format: HDEL key field [field ...]
• Function: Delete one or more specified domains in the hash table key, and the non-existing domains will be ignored.
• Description: Returns the number of successfully removed domains, excluding ignored domains.

3.4.8 hexits

• Format: HEXISTS key field
• Function: Check whether the given field field exists in the hash table key.
• Description: Returns 1 if the hash table contains the given field. Returns 0 if the given field does not exist, or if the key does not exist.

3.4.9 hincrby and hincrbyfloat

• Format: HINCRBY key field increment
• Function: add increment to the value of the field field in the hash table key. The hincrby command can only increment integer values, while hincrbyfloat can increment decimal values.
• Note: The increment can also be a negative number, which is equivalent to the subtraction operation on the given field. If the key does not exist, a new hash table is created and the HINCRBY command is executed. If the field field does not exist, then the value of the field is initialized to 0 before executing the command. Executing the HINCRBY command on a field that stores string values ​​will cause an error.

3.4.10 hkeys and whales

• Format: HKEYS key or HVALS key
• Function: Return all fields/values ​​in the hash table key.
• Description: When the key does not exist, return an empty table.

3.4.11 fulfilled

• Format: HLEN key
• Function: Return the number of domains in the hash table key.
• Description: When the key does not exist, return 0.

3.4.12 hstrlen

• Format: HSTRLEN key field
• Function: Return the string length (string length) of the value associated with the given domain field in the hash table key.
• Description: If the given key or field does not exist, the command returns 0.

3.4.13 Application scenarios

Hash type Value is very suitable for storing object data. The key is the name of the object, and the value is the Map describing the properties of the object. The modification of the properties of the object can be done directly in Redis. It is not like a String-type Value storage object, that object is serialized, such as serialized into a JSON string, the modification of the object attribute value needs to be deserialized into an object before modification, and then serialized into a JSON string after modification Write to Redis.

3.5 List type Value operation command

The Value of Redis stored data can be a String list type data. That is, each element in the list is String type data. The data in the list will be sorted in the order of insertion. However, the bottom layer of the list is actually a doubly linked list without a head node, so the operation performance on the head and tail of the list is high, but the performance on the insertion and deletion of intermediate elements is relatively poor.

3.5.1 lpush/rpush

• 格式：LPUSH key value [value …] 或 RPUSH key value [value …]
• Function: Insert one or more values ​​value into the header/footer of the list key (the header is on the left and the tail is on the right)
• Note: If there are multiple value values, for lpush, each value will be inserted into the header in order from left to right; for rpush, each value will be inserted into the end of the table in order from left to right . If the key does not exist, an empty list is created and the operation performed. Returns an error when key exists but is not a list type. Returns the length of the list on success.

3.5.2 curtain

• Format: LLEN key
• Function: Return the length of the list key.
• Description: If key does not exist, key is interpreted as an empty list and 0 is returned. Returns an error if key is not a list type.

3.5.3 line index

• Format: LINDEX key index
• Function: Return the element whose subscript is index in the list key. Lists are counted from 0.
• Explanation: If the value of the index parameter is not within the range of the list (out of range), return nil.

3.5.4 set

• Format: LSET key index value
• Function: Set the value of the element whose key subscript is index in the list to value.
• Description: An error is returned when the index parameter is out of range, or when LSET is performed on an empty list (key does not exist).

3.5.5 lrange

• Format: LRANGE key start stop
• Function: Return the elements in the specified interval [start, stop] in the list key, which includes two endpoints.
• Note: The subscript of List starts from 0, that is, 0 represents the first element of the list, 1 represents the second element of the list, and so on. Negative subscripts can also be used, with -1 representing the last element of the list, -2 representing the second-to-last element of the list, and so on. Out-of-range subscript values ​​do not cause an error. If the start index is greater than the list's largest index, then LRANGE returns an empty list. If the stop subscript is larger than the maximum subscript, Redis will set the value of stop to the maximum subscript.

3.5.6 lpushx and rpushx

• Format: LPUSHX key value or RPUSHX key value
• Function: Insert the value value into the header/footer of the list key, if and only if the key exists and is a list.
• Description: When the key does not exist, the command does nothing. If the execution is successful, the length of the table is output.

3.5.7 lensed

• Format: LINSERT key BEFORE|AFTER pivot value
• Function: Insert the value value into the list key, before or after the element pivot.
• Description: When the pivot element does not exist in the list, no operation is performed and -1 is returned; when the key does not exist, the key is regarded as an empty list, no operation is performed and 0 is returned; if the key is not a list type, one is returned Error; if the command succeeds, returns the length of the list after the insert operation is complete.

3.5.8 lpop / rpop

• Format: LPOP key [count] or RPOP key [count]
• Function: Remove count elements from the head/tail of the list key, and return the removed elements. count default value 1
• Description: When the key does not exist, return nil

3.5.9 blpop / brpop

• Format: BLPOP key [key …] timeout or BRPOP key [key …] timeout
• Function: BLPOP/BRPOP is a blocking popup command for the list. They are blocking versions of the LPOP/RPOP command. When there is no element in the given list to pop, the connection will be blocked by the BLPOP/BRPOP command until the waiting timeout expires or an element that can be popped is found. When multiple key parameters are given, each list is checked in turn according to the order of the parameter keys, and the head element of the first non-empty list is popped up. timeout is the blocking duration in seconds. If the value is 0, it means that as long as there is no pop-up element, it will always block.
• Explanation: If no element is popped up within the specified time, return a nil and the waiting time. Otherwise, return a list with two elements, the first element is the key to which the popped element belongs, and the second element is the value of the popped element.

3.5.10 rpoplpush

• Format: RPOPLPUSH source destination
• Function: Command RPOPLPUSH to perform the following two actions within an atomic time:
• Pop the last element (tail element) in the list source and return it to the client.
• Insert the popped-up element from source into the destination list as the head element of the destination list.

If source does not exist, the value nil is returned and no other action is performed. If source and destination are the same, the element at the end of the list is moved to the head, and that element is returned. This special case can be thought of as a rotation operation on the list.

3.5.11 brpoplpush

• Format: BRPOPLPUSH source destination timeout
• Function: BRPOPLPUSH is a blocking version of RPOPLPUSH. When the given list source is not empty, BRPOPLPUSH behaves the same as RPOPLPUSH. When the list source is empty, the BRPOPLPUSH command will block the connection until the wait times out, or another client executes the LPUSH or RPUSH command on the source. timeout is the blocking duration in seconds. If the value is 0, it means that as long as there is no pop-up element, it will always block.
• Explanation: If no element is popped up within the specified time, return a nil and the waiting time. Otherwise, return a list with two elements, the first element is the value of the popped element, and the second element is the waiting time.

3.5.12 lrem

• Format: LREM key count value
• Function: According to the value of the parameter count, remove the elements in the list that are equal to the parameter value. The value of count can be one of the following:
• count > 0 : Search from the head to the end of the table, remove elements equal to value, the number is count.
• count < 0 : Search from the end of the table to the head of the table, remove elements equal to value, the number is the absolute value of count.
• count = 0 : Remove all values ​​equal to value from the table.
• Description: Returns the number of elements removed. When the key does not exist, the LREM command returns 0, because the key that does not exist is regarded as an empty list (empty list).

3.5.13 ltrim

• Format: LTRIM key start stop
• Function: Trim a list, that is to say, let the list only keep the elements in the specified range, and the elements not in the specified range will be deleted.
• Explanation: The subscript (index) parameters start and stop are based on 0, that is to say, 0 represents the first element of the list, 1 represents the second element of the list, and so on. Negative subscripts can also be used, with -1 representing the last element of the list, -2 representing the second-to-last element of the list, and so on. Returns an error when key is not a list type. If the start subscript is greater than the list's largest subscript end (LLEN list minus 1), or if start > stop, LTRIM returns an empty list because LTRIM has already emptied the entire list. If the stop subscript is larger than the end subscript, Redis will set the value of stop to end.

3.5.14 Application scenarios

There are many application scenarios where the Value is a List type, and the corresponding business functions are mainly realized by constructing different data structures. Here we only summarize the implementation methods of these data structures, without giving specific examples.

3.5.14.1 Stack

The stack data structure effect can be realized through lpush + lpop: first in last out. Insert data from the left side of the list through lpush, and retrieve data from the left side of the list through lpop. Of course, the same effect can also be achieved through rpush + rpop, but the operation is on the right side of the list.

3.5.14.2 Queue

The queue data structure effect can be realized through lpush + rpop: first in first out. Insert data from the left side of the list through lpush, and retrieve data from the right side of the list through rpop. Of course, the same effect can also be achieved through rpush + lpop, but the direction of operation is just the opposite.

3.5.14.3 Blocking message queue

The blocking message queue effect can be achieved through lpush + brpop. A client as a message producer uses lpush to insert data from the left side of the list, and multiple clients as a message consumer use brpop blocking to "preempt" the data at the end of the list for consumption, ensuring load balancing and high availability of consumption. The timeout of brpop is set to 0, which means that as long as there is no data to pop, it will be blocked forever.

3.5.14.4 Dynamic Finite Sets

Finite collections can be achieved by lpush + ltrim. Add data to the list from the left of the list by lpush, and keep the dynamic finiteness of the collection by ltrim. Dynamic management such as the elimination of the bottom of the enterprise and the key classes of the school can be realized through this dynamic limited set. Of course, the same effect can also be achieved through rpush + ltrim, but the direction of operation is just the opposite.

3.6 Set type Value operation command

The Value of Redis data storage can be a Set collection, and each element in the collection is of String type. Set is very similar to List, but the difference is that elements in Set are disordered and non-repeatable, while List is orderly and repeatable.
The Set collection in Redis is similar to the Set collection in Java, and its bottom layer is a hash table whose value is null. It is precisely because of this that disorder and non-repeatability are caused.

3.6.1 sadd

• Format: SADD key member [member ...]
• Function: Add one or more member elements to the collection key, and the member elements that already exist in the collection will be ignored.
• Explanation: If the key does not exist, create a collection containing only member elements as members. Returns an error when key is not a collection type.

3.6.2 smembers

• Format: SMEMBERS key
• Function: Return all members in the set key.
• Note: A key that does not exist is considered an empty collection. If the key contains a large number of elements, this command may block the Redis service. Therefore, this command is generally not used in the production environment, and the sscan command is used instead.

3.6.3 scard

• Format: SCARD key
• Function: Return the length of the Set collection
• Description: When the key does not exist, return 0.

3.6.4 sismember

• Format: SISMEMBER key member
• Function: Determine whether the member element is a member of the set key.
• Description: Returns 1 if the member element is a member of the collection. Returns 0 if the member element is not a member of the set, or key does not exist.

3.6.5 moves

• Format: SMOVE source destination member
• Function: Move the member element from the source collection to the destination collection.
• Description: If the source collection does not exist or does not contain the specified member element, the SMOVE command does nothing and returns 0. Otherwise, the member element is removed from the source collection and added to the destination collection, and 1 is returned. When the destination collection already contains the member element, the SMOVE command simply deletes the member element in the source collection. Returns an error when source or destination is not a collection type.

3.6.6 average

• Format: SREM key member [member ...]
• Function: Remove one or more member elements in the set key, the non-existing member elements will be ignored, and return the number of successfully removed elements.
• Description: When the key is not a collection type, an error is returned.

3.6.7 srandmember

• Format: SRANDMEMBER key [count]
• Function: Return count random elements in the collection. The default value of count is 1.
• Explanation: If count is a positive number and less than the length of the collection, then return an array containing count elements, and the elements in the array are different. If count is greater than or equal to the collection length, then the entire collection is returned. If count is negative, returns an array containing the absolute value of count elements, but elements in the array may appear duplicated.

3.6.8 spops

• Format: SPOP key [count]
• Function: Remove and return count random elements in the collection. count must be a positive number, and the default value is 1.
• Description: If count is greater than or equal to the collection length, then remove and return the entire collection.

3.6.9 sdiff / sdiffstore

• Format: SDIFF key [key ...] or SDIFFSTORE destination key [key ...]
• Function: Return the difference between the first set and other sets. Difference set, difference.
• Explanation: The difference between these two commands is that sdiffstore can not only display the difference, but also store the difference in the specified set destination. If the destination collection already exists, it is overwritten. A key that does not exist is considered an empty set.

3.6.10 sinter / sinterstore

• Format: SINTER key [key ...] or SINTERSTORE destination key [key ...]
• Function: Return the intersection between multiple collections. Intersection, intersection.
• Explanation: The difference between these two commands is that sinterstore can not only display the intersection, but also store the intersection in the specified set destination. If the destination collection already exists, it is overwritten. A key that does not exist is considered an empty set.

3.6.11 sunion / sunionstore

• Format: SUNION key [key ...] or SUNIONSTORE destination key [key ...]
• Function: Return the union between multiple collections. Union.
• Explanation: The difference between these two commands is that sunionstore can not only display the union, but also store the union in the specified collection destination. If the destination collection already exists, it is overwritten. A key that does not exist is considered an empty set.

3.6.12 Application Scenarios

There are many application scenarios where the Value is a Set type, and these scenarios are only summarized here.

3.6.12.1 Dynamic black and white list

For example, a blacklist for access control needs to be set in a server. If the blacklist is directly written into the configuration file of the server, the problem is that the blacklist cannot be dynamically modified. At this time, the blacklist can be written directly to Redis. As long as there is a client to access the server, the server will first check whether the IP exists in the Redis blacklist after obtaining the client IP. If it exists, the access will be denied, otherwise the access will pass. .

3.6.12.2 Finite random numbers

The limited random number means that the returned random number is based on a random number within a certain set range, such as lottery and random selection. Through spop or srandmember, elements can be randomly selected from the specified collection.

3.6.12.3 User portrait

Various platforms such as social platforms and e-commerce platforms that require users to register and log in will make portraits for each user based on the information provided by the user and the user's usage habits, that is, define many tags for each user that can reflect the characteristics of the user. The tag can be added to the collection corresponding to the user using sadd. These tags are unordered and non-repetitive.
At the same time, the platform can also use sinter/sinterstore to recommend friends, products, and customers based on the intersection of user portraits.

3.7 Ordered Set type Value operation command

The Value of Redis stored data can be an ordered Set, and each element in this ordered Set is of String type. The difference between an ordered set and a set is that each element in an ordered set has a score, and Redis will sort the set from small to large according to the value of the score. It has the same requirements as the Set collection, the elements cannot be repeated, but the score of the elements can be repeated. Since all commands of this type begin with the letter z, this Set is also called ZSet.

3.7.1 zadd

• 格式：ZADD key score member [[score member] [score member] …]
• Function: Add one or more member elements and their score values ​​to the appropriate position in the ordered set key.
• Description: The score value can be an integer value or a double precision floating point number. If key does not exist, an empty sorted set is created and ZADD is performed. Returns an error when key exists but is not of sorted-set type. If the command succeeds, it returns the number of new members that were successfully added, excluding existing members that were updated. If the written member value already exists, but the score value is different, the new score value will overwrite the old score.

3.7.2 zrange与zrevrange

• Format: ZRANGE key start stop [WITHSCORES] or ZREVRANGE key start stop [WITHSCORES]
• Function: Return the members in the specified interval in the ordered set key. The zrange command will sort by increasing score value, and the zrevrange command will sort by score descending. Members with the same score value are sorted lexicographically/reverse-lexicographically. Members can be returned with their score values ​​by using the WITHSCORES option.
• Note: The subscript parameter starts from 0, that is, 0 represents the first member of the ordered set, 1 represents the second member of the ordered set, and so on. You can also use negative subscripts, -1 for the last member, -2 for the second-to-last member, and so on. Out-of-range subscripts do not cause an error. For example, when the value of start is greater than the largest subscript of the sorted set, or start > stop, the ZRANGE command simply returns an empty list. For another example, if the value of the stop parameter is greater than the maximum subscript of the ordered set, then Redis will treat stop as the maximum subscript.

If the specified range in the key contains a large number of elements, this command may block the Redis service. Therefore, if you want to query all the elements in the ordered set in the production environment, you generally do not use this command, but use the zscan command instead.

3.7.3 zrangebyscore与zrevrangebyscore

• Format: ZRANGEBYSCORE key min max [WITHSCORES] [LIMIT offset count]

ZREVRANGEBYSCORE key max min [WITHSCORES] [LIMIT offset count]

• Function: Return all members whose score value is between min and max (including equal to min or max) in the ordered set key. Ordered set members are arranged in increasing/decreasing order of score value. Members with the same score value are sorted lexicographically/reverse-lexicographically. The optional LIMIT parameter specifies the number and range of returned results (like SELECT LIMIT offset, count in SQL). Note that when the offset is large, the operation of locating the offset may need to traverse the entire ordered set, which may be less efficient Low. The optional WITHSCORES parameter determines whether the result set returns the ordered set members alone, or returns the ordered set members and their score values ​​together.
• Note: The values ​​of min and max are plus or minus infinity. By default, the value of the interval uses a closed interval (less than or equal to or greater than or equal to), and an optional open interval (less than or greater than) can also be used by adding a left bracket "(" before the parameter.

3.7.4 zcard

• Format: ZCARD key
• Function: Returns the length of the collection
• Description: When the key does not exist, return 0.

3.7.5 zcount

• Format: ZCOUNT key min max
• Function: Return the number of members whose score value is between min and max (including the score value equal to min or max by default) in the ordered set key.

3.7.6 zscore

• Format: ZSCORE key member
• Function: Return the score value of the member member in the ordered set key.
• Explanation: If the member element is not a member of the ordered set key, or the key does not exist, return nil.

3.7.7 zincrby

• Format: ZINCRBY key increment member
• Function: add increment to the score value of the member member of the ordered set key. The increment value can be an integer value or a double precision floating point number.
• Note: You can subtract the corresponding value from the score by passing a negative value increment. When the key does not exist, or the member is not a member of the key, ZINCRBY key increment member is equivalent to ZADD key increment member. Returns an error when key is not of sorted set type. If the command is executed successfully, the new score value of the member member will be returned.

3.7.8 zrank与zrevrank

• Format: ZRANK key member or ZREVRANK key member
• Function: Return the rank of the member member in the ordered set key. The zrank command will sort by increasing score value, and the zrevrank command will sort by score descending.
• Explanation: The rank of the member with the smallest score value is 0. Returns nil if member is not a member of the sorted set key.

3.7.9 age

• Format: ZREM key member [member ...]
• Function: Remove one or more members in the ordered set key, and the non-existing members will be ignored.
• Description: When the key exists but is not an ordered set type, an error is returned. On success, returns the number of successfully removed members, excluding ignored members.

3.7.10 zremrangebyrank

• Format: ZREMRANGEBYRANK key start stop
• Function: Remove all members in the specified rank interval in the ordered set key.
• Explanation: The ranking intervals are indicated by the subscript parameters start and stop, including start and stop. The rank interval parameter starts from 0, that is, 0 means the first-ranked member, 1 means the second-ranked member, and so on. Negative numbers can also be used, -1 for the last member, -2 for the second-to-last member, and so on. If the command is executed successfully, the number of removed members will be returned.

3.7.11 zremrangebyscore

• Format: ZREMRANGEBYSCORE key min max
• Function: Remove all members whose score value is between min and max (including equal to min or max) in the ordered set key.
• Note: If the command is executed successfully, the number of removed members will be returned.

3.7.12 zrangebylex

• Format: ZRANGEBYLEX key min max [LIMIT offset count]
• Function: This command is only applicable when all members in the set have the same score. When all members of a sorted set have the same score, the elements of the sorted set are sorted according to the lexicographical ordering of the members. That is, this command returns the members of the given set whose element values ​​are between min and max. If the members in the sorted set have different scores, the execution result of the command is the same as zrange key.
• Note: Legal min and max parameters must contain left parentheses "(" or left brackets "[", where left brackets "(" indicate open intervals, and left brackets "[" indicate closed intervals. min or max The special characters "+" and "-" can also be used to represent positive infinity and negative infinity respectively.

3.7.13 zlexcount

• Format: ZLEXCOUNT key min max
• Function: This command is only applicable when all members in the set have the same score. This command returns the number of elements in the set whose element value itself (not the score value) is within the range of min and max.

3.7.14 zremrangebylex

• Format: ZREMRANGEBYLEX key min max
• Function: This command is only applicable when all members in the set have the same score. This command removes all elements in the collection whose element values ​​themselves are within the range of min and max.

3.7.15 Application Scenarios

The most typical application scenario of an ordered set is the ranking list, such as the ranking list sorted by the number of playbacks on music and video platforms; the ranking list sorted by user evaluation or sales volume on the e-commerce platform, etc. Use the playback volume as the score, the work id as the member, the user evaluation points or sales volume as the score, and the merchant id as the member. Use zincrby to increase the sorting score, use zrevrange to get the Top few, use zrevrank to query the current ranking, use zscore to query the current sorting score, etc.

3.8 benchmark testing tools

3.8.1 Introduction

After Redis is installed, a redis-benchmark testing tool will be automatically installed, which is a stress testing tool for testing the performance of Redis.

You can view its usage through the redis-benchmark –help command: there

are many options. Let’s use examples to learn the usage of commonly used options.

3.8.2 Test 1

3.8.2.1 Command Analysis

The meaning of the options in the above command:

• -h: Specify the IP of Redis to be tested, if it is local, it can be omitted
• -p: Specify the port of Redis to be tested, if it is 6379, it can be omitted
• -c: Specifies the number of simulated clients, the default value is 50
• -n: Specifies the total number of requests made by these clients, the default value is 100000
• -d: Specifies the data length of the value operated when testing the get/set command, in bytes, and the default value is 3. This specification is not useful when testing other commands.

The meaning of the above command is to use 100 clients to connect to the Redis, and these clients will initiate a total of 100,000 requests, and the value of set/get is 8 bytes of data.

3.8.2.2 Test result analysis

This command will test all Redis commands one by one, each command will give a test report, and each test report consists of four parts:

3.8.2.2.1 Test environment report

The first is the test environment:

3.8.2.2.2 Latency Percentage Distribution

This is a statistical report by percentage: a statistic is given every time 50% of the remaining test volume is completed.

3.8.2.2.3 Cumulative Distribution of Latency

This is a report based on time interval statistics: basically every 0.1 milliseconds.

3.8.2.2.4 Overview report

This is an overview report.

3.8.3 Test 2

The meaning of the options in the above command:

• -t: Specify the command to test, multiple commands are separated by commas, no spaces
• -q: Specifies to give only a summary report

3.9 Simple Dynamic String SDS

3.9.1 Introduction to SDS

Whether it is Redis Key or Value, its basic data type is a string. For example, the field and value types of the Hash type Value, the element types of the List type, Set type, and ZSet type Value are all strings. Although Redis is developed using the standard C language, it does not directly use the traditional string representation in the C language, but customizes a string. The structure of this kind of string itself is relatively simple, but its function is very powerful. It is called a simple dynamic string, Simple Dynamic String, or SDS for short.
Note that not all strings in Redis are SDS, C strings also appear. C strings can only appear in string "literal constants", and that string cannot be changed.
redisLog(REDIS_WARNNING, “sdfsfsafsafds”);

3.9.2 SDS structure

SDS is different from C strings. A C string itself is a sequence of characters enclosed in double quotes and terminated by a null character '\0'. But SDS is a structure defined in src/sds.h in the Redis installation directory:

| struct sdshdr { // byte array, used to save the string char buf[]; // the number of used bytes in buf[], called the length of SDS int len; // unused words in buf[] number of sections int free;

}

例如执行SET country “China”命令时，键country与值”China”都是SDS类型的，只不过一个是SDS的变量，一个是SDS的字面常量。”China”在内存中的结构如下：

It can be seen from the above structure that the buf value of SDS is actually a C string, including the null character '\0', which occupies 6 bytes in total. But the len of SDS does not contain the null character '\0'.

The difference between this structure and the previous one is that there are 3 bytes of unused space here.

3.9.3 Advantages of SDS

The C string uses a character array with a length of Len+1 to represent a string whose actual length is Len. The character array ends with a null character '\0' at the end, indicating the end of the string. This structure is simple, but it cannot meet Redis's requirements for string functionality, security, and efficiency.

3.9.3.1 Prevent "string length acquisition" performance bottleneck

For a C string, if you want to get its length, you must traverse the entire string to get it. For the traversal of ultra-long strings, it will become the performance bottleneck of the system.
However, since the length data of the string is directly stored in the SDS structure, the system performance required to obtain the length of the string has nothing to do with the length of the string itself, and will not become a performance bottleneck of Redis.

3.9.3.2 Ensuring binary security

A C string can only contain characters that conform to a certain encoding format, such as ASCII, UTF-8, etc., and the null character '\0' cannot be contained in other positions except at the end of the string, otherwise the string will be deleted Program misinterpreted as ending prematurely. It is very common to use the null character '\0' as a delimiter in binary data such as pictures, audio, video, compressed files, and office files. Therefore, binary data such as pictures, audio, video, compressed files, and office files cannot be stored in C strings.
However, SDS does not use the null character '\0' as the end of the string, it uses the len attribute to determine whether the string ends. Therefore, for the program to process the string data in SDS, there is no need to make any restrictions, filters, or assumptions on the data, just read it. What is written to the data is what is read.

3.9.3.3 Reduce memory reallocation times

SDS adopts a space pre-allocation strategy and a lazy space release strategy to avoid memory reallocation problems.
The space pre-allocation strategy means that every time the SDS expands the space, the program not only allocates the required space, but also allocates additional unused space to reduce the number of memory reallocations. The additional allocated unused space depends on the value of the len attribute of the SDS after space expansion.

• If the value of the len attribute is less than 1M, the size of the allocated unused space free is the same as the value of the len attribute.
• If the value of the len attribute is greater than or equal to 1M, then the size of the allocated unused space free is fixed at 1M.

SDS adopts a lazy space release strategy for space release. This strategy means that if the length of the SDS string is shortened, the extra unused space will not be released temporarily, but will be added to free. In order to reduce the number of memory reallocations when expanding SDS later.
If you want to release the unused space of SDS, you can use the sdsRemoveFreeSpace() function to release it.

3.9.3.4 Compatible with C functions

Redis provides a lot of SDS APIs to facilitate users' secondary development of Redis. In order to be compatible with C functions, the strings in the underlying array buf[] of SDS still end with the null character '\0'.
The two sides to be compared now, one is SDS, and the other is C string, at this time, you can use the C language function
strcmp(sds_str->buf, c_str)

3.9.4 Commonly used SDS operation functions

The following table lists some commonly used SDS operation functions and their function descriptions.

function	Functional description
sdsnew()	Create an SDS using the specified C string
sdsempty()	Create an SDS that does not contain any string data
sdsdup()	Create a copy of the specified SDS
sdsfree()	Release the specified SDS
sdsclear()	Clear the string content of the specified SDS
sdslen()	Get the used space len value of the specified SDS
sdsavail()	Get the free value of the unused space of the specified SDS
sdsMakeRoomFor()	Increase the free space of the specified SDS by the specified size
sdsRemoveFreeSpace()	Release the free space of the specified SDS
sdscat()	Splice the specified C string to the end of the specified SDS string
sdscatsds()	Splice the specified SDS string to the end of another specified SDS string
sdscpy()	Copy the specified C string to the specified SDS, overwriting the original SDS string content
sdsgrouzero()	Extends the SDS string to the specified length. This expansion is padding with the null character '\0'
sdsrange()	Intercept the character string within the specified range in the specified SDS
sdstream()	Delete all occurrences of all characters in the specified C string in the specified SDS
sdsemp()	Compare whether two given SDS strings are the same
sdstolow()	Change all letters in the specified SDS string to lowercase
sdstoupper()	Make all letters in the specified SDS string uppercase

3.10 The underlying implementation principle of collections

The underlying implementation of the Set type in Redis directly uses hashTable. However, the underlying implementation of Hash, ZSet, and List collections has been specially designed to ensure the high performance of Redis.

3.10.1 Two implementation options

对于Hash与ZSet集合，其底层的实现实际有两种：压缩列表zipList，与跳跃列表skipList。这两种实现对于用户来说是透明的，但用户写入不同的数据，系统会自动使用不同的实现。只有同时满足以配置文件redis.conf中相关集合元素数量阈值与元素大小阈值两个条件，使用的就是压缩列表zipList，只要有一个条件不满足使用的就是跳跃列表skipList。例如，对于ZSet集合中这两个条件如下：

• The number of collection elements is less than the value of the zset-max-ziplist-entries attribute in redis.conf, and its default value is 128
• The size of each collection element is smaller than the value of the zset-max-ziplist-value attribute in redis.conf, and its default value is 64 bytes

3.10.2 zipList

3.10.2.1 What is zipList

A zipList, commonly called a compressed list, is a specially encoded doubly linked list for storing strings or integers. Its underlying data structure consists of three parts: head, entries and end. These three parts are stored consecutively in memory.

3.10.2.2 head

head又由三部分构成：

• zlbytes: 4 bytes, used to store the number of bytes occupied by the overall data structure of the zipList list, including the length of zlbytes itself.
• zltail: 4 bytes, used to store the offset (byte) of the last entry in the zipList in the entire data structure. The existence of this data can quickly locate the tail entry position of the list to facilitate operation.
• zllen: 2 bytes, used to store the number of entries contained in the list. Since it has only 16 bits, the maximum number of entries that zipList can contain is 216-1 = 65535.

3.10.2.3 entries

entries是真正的列表，由很多的列表元素entry构成。由于不同的元素类型、数值的不同，从而导致每个entry的长度不同。
每个entry由三部分构成：

• prevlength: This part is used to record the length of the previous entry to achieve reverse order traversal. The default length is 1 byte. As long as the length of the previous entry is less than 254 bytes, prevlength occupies 1 byte, otherwise it will be automatically extended to 3 bytes.
• encoding: This part is used for the specific type of data behind the flag. If data is an integer type, encoding has a fixed length of 1 byte. If data is a string type, the encoding length may be 1 byte, 2 bytes or 5 bytes. Different lengths of data strings correspond to different encoding lengths.
• data: the actual stored data. The data type can only be an integer type or a string type. Different data occupies different byte lengths.

3.10.2.4 end

end contains only part, called zlend. Occupying 1 byte, the value is fixed at 255, that is, the binary bits are all 1, indicating the end of a zipList list.

3.10.3 listPack

For ziplist, the implementation is complicated. In order to traverse in reverse order, each entry contains the length of the previous entry, which will cause cascading updates when modifying or inserting entries in the middle of ziplist. In the scenario of high concurrent write operation, the performance of Redis will be extremely reduced. In order to achieve more compact, faster parsing, and simpler implementation, the implementation of ziplist is rewritten and named listPack.
In Redis 7.0, all zipLists have been replaced by listPack, but for compatibility, the related attributes of zipList are also retained in the configuration.

3.10.3.1 What is listPack

listPack is also a specially encoded doubly linked list for storing strings or integers. Its underlying data structure also consists of three parts: head, entries and end, and these three parts are also stored continuously in memory.
The major difference between listPack and zipList lies in the structure of the head and each entry. The end indicating the end of the list is the same as the zlend of zipList, occupying one byte, and all 8 bits are 1.

3.10.3.2 head

head consists of two parts:

• totalBytes: 4 bytes, used to store the number of bytes occupied by the overall data structure of the listPack list, including the length of totalBytes itself.
• elemNum: 2 bytes, used to store the number of entries contained in the list. Its meaning is the same as that of zllen in zipList.

Compared with the head of zipList, there is no zltail that records the last entry offset.

3.10.3.3 entries

entries也是listPack中真正的列表，由很多的列表元素entry构成。由于不同的元素类型、数值的不同，从而导致每个entry的长度不同。但与zipList的entry结构相比，listPack的entry结构发生了较大变化。
其中最大的变化就是没有了记录前一个entry长度的prevlength，而增加了记录当前entry长度的element-total-len。而这个改变仍然可以实现逆序遍历，但却避免了由于在列表中间修改或插入entry时引发的级联更新。
每个entry仍由三部分构成：

• encoding: This part is used for the specific type of data behind the flag. If data is an integer type, the encoding length may be 1, 2, 3, 4, 5 or 9 bytes. Different byte lengths have different identification bits. If data is a string type, the encoding length may be 1, 2 or 5 bytes. Different lengths of data strings correspond to different encoding lengths.
• data: the actual stored data. The data type can only be an integer type or a string type. Different data occupies different byte lengths.
• element-total-len: This part is used to record the length of the current entry for reverse order traversal. Due to its special recording method, the byte data it occupies may be 1, 2, 3, 4 or 5 bytes.

3.10.4 shipList

3.10.4.1 What is skipList

skipList, skip list, referred to as skip list, is a randomized data structure based on a parallel linked list, which is simple to implement and has high search efficiency. Simply put, the jump list is also a kind of linked list, but it adds a jump function to the linked list. It is also this jump function that enables higher efficiency when searching for elements.

3.10.4.2 principle of skipList

Suppose there is an ordered linked list with head and tail nodes.

In this linked list, if you want to find a certain data, you need to compare one by one from the beginning until you find the node containing the data, or find the first node larger than the given data, or find the last end node, the latter two All are not found. Similarly, when we want to insert new data, we have to go through the same search process to determine the insertion position.
In order to improve the search efficiency, a pointer is added to the even node to point to the next even node.

In this way, all even-numbered nodes are connected into a new linked list (referred to as a high-level linked list). Of course, the number of nodes contained in the high-level linked list is only half of the original linked list. At this time, when you want to search for a certain data, first search along the high-level linked list. When encountering the first node that is larger than the data to be searched, immediately return to the original linked list from the previous node of the large node to search. For example, if you want to insert a data 20, first search in the linked list of (8, 19, 31, 42), find the first node 31 that is larger than 20, and then find the previous node of 31 in the high-level linked list 19, and then get the next node value 23 in the original linked list. is greater than 20, insert 20 between the 19 node and the 23 node. If the insertion is 25, which is larger than node 23, it will be inserted between node 23 and node 31.
This method can obviously reduce the number of comparisons and improve search efficiency. If there are many elements in the linked list, in order to further improve the search efficiency, the original linked list can be constructed as a three-layer linked list, or a higher-level linked list.

The higher the level, the higher the search efficiency.

3.10.4.3 Problems

This method of hierarchically linking the linked list does improve the search efficiency in principle, but there is a problem in actual operation: since the elements with fixed serial numbers have a fixed level, when the list element is added or deleted, it will This will lead to a large adjustment of the overall element level of the list, but this will inevitably greatly reduce system performance.
For example, for a linked list divided into two levels, it can be specified that the odd-numbered nodes are high-level linked lists, and the even-numbered nodes are low-level linked lists. For a linked list divided into three levels, it can be divided according to the node serial number and the modulo result of 3. However, if a new node is inserted, or some original nodes are deleted, then the hierarchy will be re-divided according to the original hierarchy division rules, and the system performance will be greatly reduced.

3.10.4.4 Algorithm optimization

In order to avoid the previous problems, skipList adopts the method of randomly assigning levels. That is, after the total level is determined, every time a new element is added, a level will be randomly assigned to it. This randomness solves the problem of the fixed relationship between the node serial number and the level.

The figure above demonstrates the process of randomly assigning levels to each element during the generation of the list. From the creation and insertion process of this skiplist, it can be seen that the number of levels of each node is randomly assigned, and a new insertion of a node will not affect the number of levels of other nodes. It is only necessary to modify the pointers before and after the inserted node, without adjusting many nodes. This reduces the complexity of the insert operation.
skipList refers to that in addition to the bottom 1 linked list, it will generate several layers of sparse linked lists. The pointers in these linked lists skip some nodes, and the higher the level of the linked list, the more nodes are skipped. When searching for data, first search in the high-level linked list, then lower it layer by layer, and finally may drop to the first-level linked list to accurately determine the data location. In this process, because some nodes are skipped, the search speed is accelerated.

3.10.5 quickList

3.10.5.1 What is quickList

quickList, quick list, quickList itself is a two-way non-circular linked list, and each node of it is a zipList. Starting from Redis3.2 version, for the underlying implementation of List, quickList is used instead of zipList and linkedList.
Both zipList and linkedList have obvious deficiencies, and quickList improves them: absorbing the advantages of zipList and linkedList, avoiding their deficiencies.
quickList is essentially a hybrid of zipList and linkedList. It divides the linkedList into segments, each segment uses zipList to compactly store several real data elements, and multiple zipLists are connected in series using bidirectional pointers. Of course, the maximum capacity of data elements that can be stored in each zipList can be specified in the configuration file through the list-max-ziplist-size attribute.

3.10.5.2 Retrieval operation

In order to have a deeper understanding of the working principle of quickList, deepen the understanding through the analysis of the implementation of retrieval, insertion, deletion and other operations.
The retrieval of List elements is based on its index index. quickList is composed of zipLists one by one, and the zllen of each zipList records the number of entries contained in the current zipList, that is, the number of real data elements contained. According to the index of the element to be retrieved, starting from the head node of the quickList, sum the zllen of the zipList one by one until the first zipList whose sum is greater than the index is found, then the element to be retrieved is in this zipList.

3.10.5.3 Insert operation

由于zipList是有大小限制的，所以在quickList中插入一个元素在逻辑上相对就比较复杂一些。假设要插入的元素的大小为insertBytes，而查找到的插入位置所在的zipList当前的大小为zlBytes，那么具体可分为下面几种情况：

• Case 1: When insertBytes + zlBytes <= list-max-ziplist-size, just insert directly into the corresponding position in zipList
• Case 2: When insertBytes + zlBytes > list-max-ziplist-size, and the insertion position is at the head of the zipList, you need to check the size prev_zlBytes of the previous zipList of the zipList.
• If insertBytes + prev_zlBytes<= list-max-ziplist-size, just insert the element directly at the end of the previous zipList
• If insertBytes + prev_zlBytes> list-max-ziplist-size, directly build the element itself into a new zipList and connect it to the quickList
• Case 3: When insertBytes + zlBytes > list-max-ziplist-size, and the insertion position is at the end of the zipList, you need to check the size next_zlBytes of the next zipList of the zipList.
• If insertBytes + next_zlBytes<= list-max-ziplist-size, just insert the element directly into the head position of the next zipList
• If insertBytes + next_zlBytes> list-max-ziplist-size, directly build the element itself into a new zipList and connect it to the quickList
• Situation 4: When insertBytes + zlBytes > list-max-ziplist-size, and the insertion position is in the middle of the zipList, divide the current zipList into two zipLists and connect them to the quickList, and then insert the element in front of the split The tail position of zipList

3.10.5.4 Delete operation

For the delete operation, you only need to pay attention to whether there are still elements in the zipList after the element is deleted in the corresponding zipList. If there are no other elements, delete the zipList and connect the two zipLists before and after it.

3.10.6 The number of elements in key and value

The design of the various special data structures of Redis described above not only greatly improves the performance of Redis, but also makes the number of keys that Redis can support and the number of elements that can be supported in the set value very large.

• ‎Redis can handle up to 232 keys (about 4.2 billion), and it has been tested in practice that each Redis instance can handle at least 250 million keys. ‎
• ‎Each Hash, List, Set, ZSet collection can contain 232 elements. ‎

3.11 BitMap operation commands

3.11.1 Introduction to BitMap

BitMap is a new data type introduced in Redis version 2.2.0. This data type is essentially a binary string containing only 0s and 1s. And all its related commands are operations on the binary bits of this string. There are three attributes used to describe the string: key, offset, bitValue.

• key: BitMap is a Value data type in the key-value of Redis, so the Value must have its corresponding key.
• offset: Each BitMap data is a string, and each character in the string has its corresponding index, which starts counting from 0. The index is called the offset of each character in the BitMap. The value range of this offset is [0, 232-1], that is, the maximum value of this offset is 4G-1, that is, 4294967295, more than 4.2 billion.
• bitValue: Each BitMap data is a binary string containing only 0 and 1, and the character on each offset bit is called the value bitValue of the bit. The value of bitValue is either 0 or 1.

3.11.2 setbit

• Format: SETBIT key offset value
• Function: Set the value for the offset position of the BitMap data of the given key. Its return value is the bitValue of the offset position before modification
• Note: When assigning a value to an offset that does not exist in the original BitMap string, the string will be automatically stretched to ensure that it can store the value at the specified offset. When string values ​​are stretched, empty positions are filled with 0's. Of course, the set value can only be 0 or 1. However, it should be noted that for SETBIT operations that use a large offset, the memory allocation process may cause the Redis server to be blocked.

3.11.3 getbit

Format: GETBIT key offset
Function: For the BitMap string value stored in the key, get the bit value bitValue at the specified offset offset.
Description: When the offset is greater than the length of the string value, or the key does not exist, return 0 .

3.11.4 bitcount

• Format: BITCOUNT key [start] [end]
• Function: Count the number of bits set to 1 in the given string. In general, the scope of statistics is the entire given BitMap string. But you can also specify additional start or end parameters to only count the strings in the specified byte range, including start and end. Note that the units of start and end here are bytes, not bits, and count from 0.
• Note: Both start and end parameters can use negative values: -1 means the last byte, -2 means the penultimate byte, and so on. In addition, the non-existent key is treated as an empty string, so the BITCOUNT operation is performed on a non-existent key, and the result is 0.

3.11.5 bits

• Format: BITPOS key bit [start] [end]
• Function: Return the position of the first binary bit whose value is the specified value bit (not 0 or 1) in the BitMap specified by key. pos, that is, position, position. By default, the command will detect the entire BitMap, but users can also specify the range to be detected through the optional start parameter and end parameter.
• Note: The meanings of start and end are the same as those in the bitcount command.

3.11.6 bitop

• Format：BITOP operation destkey key [key …]
• Function: Perform binary operation on one or more BitMap string keys, and save the result to destkey.
• operation can be any of the four operations AND, OR, NOT, XOR:
• BITOP AND destkey key [key …] : Performs a bitwise AND operation on one or more BitMaps and saves the result to destkey .
• BITOP OR destkey key [key …] : Performs a bitwise OR operation on one or more BitMaps and saves the result to destkey .
• BITOP XOR destkey key [key …] : Performs a bitwise XOR operation on one or more BitMaps and saves the result to destkey .
• BITOP NOT destkey key : Performs a bitwise NOT operation on the given BitMap and saves the result to destkey .
• illustrate:
• Except for the NOT operation, all operations can accept one or more BitMaps as input.
• Except for the NOT operation, other operations on a BitMap are actually a copy.
• If multiple BitMaps involved in the operation have different lengths, the shorter BitMap will use 0 as a supplementary bit to operate with the longer BitMap, and the length of the operation result is the same as that of the longer BitMap.

3.11.7 Application Scenarios

Due to the large value range of offset, it is generally applied to binary statistics of large data volumes. For example, platform active user statistics (binary: visit or not), support rate statistics (binary: support or not), employee attendance statistics (binary: on or off duty), image binarization (binary: black or white), etc.
However, for binary statistics with a small amount of data, it is not suitable for BitMap, and it may be more appropriate to use Set. Of course, the specific amount of data is suitable for using Set, and the excess amount of data is suitable for using BitMap, which requires specific analysis according to specific scenarios.
For example, a platform wants to count the number of daily active users.
If you use Set to count, you only need to go online and write its user ID into the Set collection. Finally, you only need to count the number of elements in the Set collection to complete the statistics. That is, the size of the memory occupied by the Set collection is proportional to the number of online users. Assuming that the user ID has m bits and the number of active users is n, the size of the Set should be at least m n bytes.
If you use BitMap for statistics, you need to define a BitMap first, and the bits it occupies are at least the number of registered users. Only one user needs to go online, and one of the bit positions is immediately set to 1, and finally the statistics can be completed by counting the number of 1s in the BitMap. That is, the size of memory occupied by BitMap is proportional to the number of registered users, and has nothing to do with the number of online users. Assuming that the number of registered users on the platform is N, the length of the BitMap is at least N bits, that is, N/8 bytes.
When is it more appropriate to use BitMap? Let m n bytes = N/8 bytes, that is, when n = N/8/m = N/(8 m), the memory size occupied by the Set collection is the same as that of the BitMap. Taking Taobao as an example, the length of its user ID is 11 digits (m), and the number of registered users is 800 million (N). When the number of active users is 800 million/(811) = 09 million = 9*106 = 9 million, the memory occupied by Set and BitMap is equal. However, Taobao's average daily active users are 80 million, so it is more appropriate for Taobao to use BitMap.

3.12 HyperLogLog operation command

3.12.1 Introduction to HyperLogLog

HyperLogLog is a new data type introduced in Redis version 2.8.9, which means hyperlog log, super log record. This data type can be simply understood as a set collection whose elements are strings. But in fact, HyperLogLog is a cardinality counting probability algorithm, through which the statistics of independent totals can be completed with a very small memory. All its related commands are operations on this "set collection".

3.12.2 pathd

• Format: PFADD key element [element ...]
• Function: Add any number of elements to the specified HyperLogLog collection. Returns 1 if the internal storage was modified, 0 otherwise.

3.12.3 pfcount

• Format: PFCOUNT key [key ...]
• Function: When this command acts on a single key, it returns the approximate cardinality of the HyperLogLog collection of the given key; when this command acts on multiple keys, it returns the approximate cardinality of the union of all the HyperLogLog collections of the given key; if the key does not exist, then return 0.

3.12.4 pfmerge

• Format：PFMERGE destkey sourcekey [sourcekey …]
• Function: Merge multiple HyperLogLog collections into one HyperLogLog collection and store it in destkey. The cardinality of the merged HyperLogLog is close to the union of all sourcekey HyperLogLog collections.

3.12.5 Application Scenarios

HyperLogLog can perform inaccurate de-duplication counting statistics on extremely large log data. Of course, the official error given by Redis for this inaccuracy is 0.81%. This error is allowed for most large data volume scenarios. For the daily UV data of each page on the platform, it is very suitable to use HyperLogLog for recording.

3.13 Geospatial operation commands

3.13.1 Introduction to Geospatial

Geospatial, geographic space.
Redis introduced Geospatial, a new data type in version 3.2. This type is still a collection in essence, but the collection element is special, and it is a data structure composed of three parts. This data structure is called a spatial element:

• Longitude: longitude. Valid longitudes are [-180, 180]. Positive means east longitude, negative means west longitude.
• Latitude: latitude. Valid latitudes are [-85.05112878, 85.05112878]. Positive indicates north latitude and negative indicates south latitude.
• Location name: the name given to the location marked by the latitude and longitude, also known as the spatial element name of the Geospatial collection.

Through this type, you can set and query the latitude and longitude of a geographic location, query spatial elements within a certain range, and calculate the distance between two spatial elements, etc.

3.13.2 geoadd

• 格式：GEOADD key longitude latitude member [longitude latitude member …]
• Function: Add one or more spatial elements to the specified spatial collection.
• Description: This command returns an error when the user tries to enter a longitude or latitude that is out of range.

3.13.3 geopos

• Format: GEOPOS key member [member ...]
• Function: Return the location of the specified element from the specified geographic space, that is, the latitude and longitude.
• Note: Because the command accepts a variable number of elements as input, the command returns an array even if the user gives only one element.

3.13.4 geodist

• Format: GEODIST key member1 member2 [unit]
• Function: Returns the distance between two given positions. Where unit must be one of the following units:
• m : meters, default
• km: kilometer
• mi: miles
• ft: feet
• Description: If one of the two locations does not exist, the command returns a null value. In addition, distance calculations assume that the Earth is perfectly spherical, and in the extreme case this assumption introduces a maximum error of 0.5%.

3.13.5 geohash

• Format: GEOHASH key member [member ...]
• Function: Returns the Geohash value of one or more location elements.
• Description: GeoHash is an address encoding method. He can encode two-dimensional spatial longitude and latitude data into a string. This value is mainly used for low-level applications or debugging, and has little effect in practice.

3.13.6 georadius

• 格式：GEORADIUS key longitude latitude radius m|km|ft|mi [WITHCOORD] [WITHDIST] [WITHHASH] [ASC|DESC] [COUNT count]
• Function: With the given latitude and longitude as the center, return the elements whose distance from the center does not exceed the given radius among all location elements contained in the specified geographic space. Additional information may also be carried on return:
• WITHDIST : While returning the position element, the distance between the position element and the center is also returned. The unit of the distance is consistent with the range unit given by the user.
• WITHCOORD : The longitude and dimension of the location element are also returned.
• WITHHASH: The Geohash of the position element is also returned, but this hash is expressed in integer form

command returns unsorted positional elements by default. Through the following two parameters, the user can specify the sorting method of the returned position elements:

• ASC : According to the position of the center, return the position element in the way from near to far.
• DESC : Returns positional elements from farthest to nearest, according to the position of the center.
• Description: By default, this command returns all matching positional elements. Although the user can use the COUNT option to obtain the first N matching elements, because the command may need to process all the matched elements internally, when searching a very large area, even if the COUNT option is used to obtain a small number element, the command may also execute very slowly.

3.13.7 georadiusbymember

• 格式：GEORADIUSBYMEMBER key member radius m|km|ft|mi [WITHCOORD] [WITHDIST] [WITHHASH] [ASC|DESC] [COUNT count]
• Function: This command is the same as the GEORADIUS command, which can find out the elements within the specified range, but the center point of this command is given by the position element form, instead of using the input longitude and latitude to specify the center point like GEORADIUS.
• Explanation: The returned result also contains the element of the center point position

3.13.8 Application Scenarios

The meaning of Geospatial is geographic location, so it mainly applies geographic location-related calculations. For example, the "nearby" function in WeChat discovery, the "radar plus friend" function in adding friends; the "nearby" function in QQ dynamics; the "check-in" function in DingTalk, etc.

3.14 Publish/Subscribe Commands

3.14.1 Message system

Publish/subscribe, or pub/sub, is a message communication mode: publishers, also known as message producers, produce and send messages to storage systems; subscribers, also known as message consumers, receive and consume messages from storage systems. This storage system can be file system FS, message middleware MQ, data management system DBMS, or Redis. The entire message publisher, subscriber and storage system is called a message system.
After a subscriber in the message system subscribes to a certain type of message, as long as the type of message exists in the storage system, it can continuously receive and consume these messages. When there is no such message in the storage system, the receiving and consumption of the subscriber are blocked. And when the publisher writes the message to the storage system, it will wake up the subscriber immediately. When the storage system is full, different publishers have different processing methods: some will block the publication of the publisher, waiting for the available storage space; some will lose the redundant messages.
Of course, different message system messages publish/subscribe in different ways. For example, in a messaging system composed of messaging middleware such as RocketMQ and Kafka, published/subscribed messages are classified by topic. In the message system composed of Redis, the publish/subscribe messages are classified by Channel.

3.14.2 subscribe

• Format: SUBSCRIBE channel [channel ...]
• Function: The Redis client can subscribe to any number of channels at the same time through a subscribe command. After outputting the subscribed topic, the command is blocked, waiting for messages from the relevant channel.

3.14.3 psubscribe

• Format: PSUBSCRIBE pattern [pattern ...]
• Function: Subscribe to one or more channels that match a given pattern.
• Note: The pattern here can only use wildcards  . For example, it  can match all channels starting with it, like it.news, it.blog, it.tweets, etc.; news.* can match all channels starting with news., like news.global.today, news.it, etc. .

3.14.4 publish

• Format: PUBLISH channel message
• Function: The Redis client can publish a channel message through a publish command. The return value is the number of subscribers who received the message.

3.14.5 unsubscribe

• Format: UNSUBSCRIBE [channel [channel ...]]
• Function: Redis client unsubscribes from the specified channel.
• Note: If no channel is specified, that is, an UNSUBSCRIBE command without parameters is executed, all channels subscribed by the client using the SUBSCRIBE command will be unsubscribed. In this case, the command returns a message telling the client of all unsubscribed channels.

3.14.6 punsubscribe

• Format: PUNSUBSCRIBE [pattern [pattern ...]]
• Function: Unsubscribe from one or more channels matching the given pattern.
• Note: The pattern here can only use the wildcard *. If no channel is specified, the effect is the same as the SUBSCRIBE command, and the client will unsubscribe from all subscribed channels.

3.14.7 pubsub

• Format: PUBSUB [argument [argument ...]]
• Function: PUBSUB is an introspection command set for viewing subscription and publishing system status. It consists of several subcommands in different formats. The usage of these subcommands is introduced below.

3.14.7.1 pubsub channels

• Format: PUBSUB CHANNELS [pattern]
• Function: List all currently active channels. Active channels are those that have at least one subscriber.
• Note: The pattern parameter is optional. If no pattern argument is given, all active channels in the subscribe/publish system will be listed. If the pattern argument is given, only those active channels matching the given pattern are listed. Only the wildcard * can be used in the pattern.

3.14.7.2 pubsub numsub

• Format: PUBSUB NUMSUB [channel-1 … channel-N]
• Function: Returns the number of subscribers for a given channel. Returns an empty list if no channels are given.

3.14.7.3 pubsub rents

• Format: PUBSUB NUMPAT
• Function: Query the sum of the number of all channel modes subscribed by all current Redis clients

3.15 Redis transactions

The essence of Redis transactions is the batch processing of a set of commands. This group of commands will be executed sequentially and all at once during execution. As long as there are no syntax errors, this group of commands will not be interrupted during execution.

3.15.1 Redis transaction features

Redis transactions only guarantee data consistency and do not have the same ACID characteristics as DBMS.

• The execution failure of some commands in this group of commands will not affect the execution of other commands and will not cause rollback. That is, it is not atomic.
• This set of commands implements simple isolation through optimistic locking. There are no complicated isolation levels.
• The execution results of this group of commands are written to the memory, and whether they are persistent depends on the persistence strategy of Redis and has nothing to do with transactions.

3.15.2 Redis transaction implementation

3.15.2.1 Three commands

Redis transactions are controlled through three commands.

• muti: open transaction
• exec: execute the transaction
• discard: cancel the transaction

3.15.2.2 Basic usage

The following is the usage of defining and executing a transaction:

After the transaction is executed, access the variable defined in the transaction, and its value is modified.

The following is an example of defining but canceling a transaction:

After the transaction is cancelled, the commands in the transaction are not executed.

3.15.3 Redis Transaction Exception Handling

3.15.3.1 Syntax errors

When a command in a transaction has a syntax error, the entire transaction will be canceled during exec execution.

The hint for exec is that exec was ignored and the transaction was canceled because of a previous error.
At this time, access the value of age and find that it is still 19, and has not changed to 20 set in the transaction.

3.15.3.2 Execution exception

If there is no syntax error in the command in the transaction, but an exception occurs during execution, the exception will not affect the execution of other commands.

An exception occurred during the execution of the second command in the above transaction. Because score is not an integer, it cannot be increased by 20. But this exception will not affect the correct execution of the commands before and after it. Check the values ​​of score and name, and find that it is the result of successful execution.

3.15.4 Redis transaction isolation mechanism

3.15.4.1 Why Isolation Mechanism is Needed

In concurrent scenarios, multiple clients may modify the same data.
For example: There are two clients C left and C right, C left needs to apply for 40 resources, C right needs to apply for 30 resources. They first looked at the number of resources currently owned, that is, the value of resources. All they saw were 50, and they all felt that the number of resources could meet their needs, so they modified the number of resources to occupy the resources. But the result is that the resource is "oversold".

In order to solve this situation, the Redis transaction implements execution isolation under multi-threading through the optimistic locking mechanism.

3.15.4.2 Implementation of isolation

Redis implements execution isolation under multi-threading through watch commands and transactions.

The time sequence of execution of the above two clients is:

time	C left	C right
T1	watch resources
T2	get resources	get resources
T3	multi
T4	decrby resources 40
T5		decrby resources 30
T6	exec

3.15.4.3 Implementation Principle

Its internal execution process is as follows:

1. When a client performs a watch on the key, the system will add a version optimistic lock to the key and initialize the version. For example, the initial value is 1.0.
2. Afterwards, client C wrote the modification statement of the key into the transaction command queue. Although it was not executed, it read the value and version of the key and saved it to the current client cache. At this time, the initial value of version 1.0 is read and saved.
3. Afterwards, the client C modifies the value of the key. This modification not only modifies the value of the key itself, but also increases the value of the version, for example, changing the version to 2.0, and records the version in the key information. .
4. Afterwards, client C executes exec and starts to execute the commands in the transaction. However, when it executes the command to modify the key, the command first compares the version value saved in the current client cache and the version value in the current key information. If the cache version is smaller than the key version, it means that the value of the key cached by the client is outdated, and the write operation may destroy data consistency if executed. So the write operation is not performed.