Redis is sorry that I am superficial (principle): data transfer protocol RESP

I. Introduction

 
  Before Redis version 4.0, Redis was a single-threaded program, mainly referring to the network I/O thread of Redis. Operations such as Redis persistence and cluster synchronization are performed by another thread. However, after Redis version 4.0, Redis added multi-threading support. At this time, multi-threading is mainly reflected in the asynchronous deletion function of big data. After the Redis 6.0 version, the read and write concurrency capability of multi-threaded I/O has been added.
 
  The operation of Redis is carried out in memory, and a single node runs the QPS of a CPU core (query rate per second, Queries-per-second, is a measure of how much traffic a specific query server handles within a specified time) Can reach 10w/s.
 
  Redis uses RESP协议data transmission, what is the RESP protocol? What is the specific manifestation?
 
  The installation and deployment blogs related to Redis are as follows:

Docker installs the latest Redis6 (redis-6.2.7) (refer to official documents)
 
Docker installs the latest stable version of Redis7 (redis-7.0.5) (refers to official documents)
 
Write the best latest Redis6 (redis-6.2.7) on the cloud server Centos7 installation and deployment tutorial (refer to official documents)
 
write the best latest stable version of Redis7 (redis-7.0.5) in the cloud server Centos7 installation and deployment tutorial (refer to official documents)

 
  Redis official release notes (with Google translation) are as follows:

Redis-7.0.x version official release note (with Google translation) [continuous update]
 
Redis-6.2.x version official release note (with Google translation) [continuous update]

 
  "Redis I'm Sorry I'm Superficial" series of articles:

1. Redis is sorry for my superficiality (principle): data transfer protocol RESP
 
2. Redis is sorry for my superficiality (basic and application): the magical use of bitmaps (Bitmaps) and an in-depth analysis of the usage of each command

  本文由 @大白有点菜 原创,请勿盗用,转载请说明出处!如果觉得文章还不错,请点点赞,加关注,谢谢!
 

2. Description of the RESP protocol

 
  [RESP protocol specification description - official document]:
  https://redis.io/docs/reference/protocol-spec/Redis
 
  official introduction RESP protocol specification (Google translation):

  Redis clients use a protocol called RESP (REdis Serialization Protocol) to communicate with the Redis server. While the protocol was designed specifically for Redis, it can be used for other client-server software projects.
  Redis 客户端使用称为 RESP(REdis 序列化协议)的协议与 Redis 服务器进行通信。虽然该协议是专门为 Redis 设计的,但它也可以用于其他客户端-服务器软件项目。
 
  RESP is a compromise between the following things:
  RESP 是以下各项之间的折衷:

  • Simple to implement.
  • 易于实施。
  • Fast to parse.
  • 快速解析。
  • Human readable.
  • 人员可读。
     

  RESP can serialize different data types like integers, strings, and arrays. There is also a specific type for errors. Requests are sent from the client to the Redis server as arrays of strings that represent the arguments of the command to execute. Redis replies with a command-specific data type.
  RESP 可以序列化不同的数据类型,如整数、字符串和数组。还有一种特定的错误类型。请求作为字符串数组从客户端发送到 Redis 服务器,这些字符串表示要执行的命令的参数。 Redis 使用特定于命令的数据类型进行回复。
 
  RESP is binary-safe and does not require processing of bulk data transferred from one process to another because it uses prefixed-length to transfer bulk data.
  RESP 是二进制安全的,不需要处理从一个进程传输到另一个进程的批量数据,因为它使用前缀长度来传输批量数据。
 
  Note: the protocol outlined here is only used for client-server communication. Redis Cluster uses a different binary protocol in order to exchange messages between nodes.
  注意:此处概述的协议仅用于客户端-服务器通信。 Redis 集群使用不同的二进制协议来在节点之间交换消息。
 
  The RESP protocol was introduced in Redis 1.2, but it became the standard way for talking with the Redis server in Redis 2.0.
  RESP 协议在 Redis 1.2 中引入,但在 Redis 2.0 中成为与 Redis 服务器通信的标准方式。

  In layman's terms, RESP 序列化协议is a text protocol that can be intuitively understood by programmers. It is simple to implement and has strong analytical performance.
 
  The RESP protocol supports the following data types: Simple Strings(简单字符串,即单行字符串), Errors(错误), Integers(整数)、Bulk Strings(批量字符串,即多行字符串)and Arrays(数组).
 
  The RESP protocol mainly divides the transmitted data into five smallest unit types, and " 回车换行符【\r\n】" is uniformly added after the end of each unit.

1. A simple character string (single-line character string, Simple Strings) +begins with a " " symbol.
 
2. Batch character strings (multi-line character strings, Bulk Strings) $begin with the " " symbol, followed by 字符串长度.
 
3. Integers ( Integers) :begin with the symbol " " and are followed by 整数的字符串形式.
 
4. An error ( Errors) -starts with a " " symbol.
 
5. The array ( Arrays) *begins with the symbol " ", followed by 数组的长度.

 

3. Application of RESP protocol

 
  【注意】: Add “ 回车换行符【\r\n】” after each unit.
 

1. Data transmission format

 

1.1 Single-line strings (simple strings, Simple Strings)

  Simple Strings are encoded as follows: a plus character, followed by a string that cannot contain a CR or LF character (no newlines are allowed), and terminated by CRLF (that is “\r\n”).
  简单字符串编码如下:一个加号,后跟一个不能包含 CR 或 LF 字符的字符串(不允许换行),并以 CRLF(即“\r\n”)结尾。
 
  Simple Strings are used to transmit non binary-safe strings with minimal overhead. For example, many Redis commands reply with just “OK” on success. The RESP Simple String is encoded with the following 5 bytes:
  简单字符串用于以最小开销传输非二进制安全字符串。例如,许多 Redis 命令在成功时只回复“OK”。 RESP 简单字符串使用以下 5 个字节进行编码:
 
  “+OK\r\n”
 
  In order to send binary-safe strings, use RESP Bulk Strings instead.
  为了发送二进制安全的字符串,请改用 RESP 批量字符串(Bulk Strings)。
 
  When Redis replies with a Simple String, a client library should respond with a string composed of the first character after the ‘+’ up to the end of the string, excluding the final CRLF bytes.
  当 Redis 回复一个简单字符串时,客户端库应该回复一个字符串,该字符串由“+”之后的第一个字符组成,一直到字符串末尾,不包括最后的 CRLF 字节。

  Simple strings (single-line strings, Simple Strings) +begin with a " " symbol. Example: hello dbydc

Transmission format:+hello dbydc\r\n

 

1.2 Multi-line strings (bulk strings, Bulk Strings)

  Bulk Strings are used in order to represent a single binary-safe string up to 512 MB in length.
  批量字符串用于表示长度最大为 512 MB 的单个二进制安全字符串。
 
  Bulk Strings are encoded in the following way:
  批量字符串按以下方式编码:
 

  • A “$” byte followed by the number of bytes composing the string (a prefixed length), terminated by CRLF.
  • “$”字节后跟组成字符串的字节数(前缀长度),以 CRLF 结尾。
  • The actual string data.
  • 实际的字符串数据。
  • A final CRLF.
  • 最终的 CRLF。

 
  So the string “hello” is encoded as follows:
  所以字符串“hello”编码如下:
 
  “$5\r\nhello\r\n”
 
  An empty string is encoded as:
  空字符串编码为:
 
  “$0\r\n\r\n”
 
  RESP Bulk Strings can also be used in order to signal non-existence of a value using a special format to represent a Null value. In this format, the length is -1, and there is no data. Null is represented as:
  RESP 批量字符串也可用于使用特殊格式表示值不存在来表示 Null 值。这种格式,长度为-1,没有数据。Null 表示为:
 
  “$-1\r\n”
 
  This is called a Null Bulk String.
  这称为Null批量字符串。
 
  The client library API should not return an empty string, but a nil object, when the server replies with a Null Bulk String. For example, a Ruby library should return ‘nil’ while a C library should return NULL (or set a special flag in the reply object).
  当服务器回复 Null Bulk String 时,客户端库 API 不应返回空字符串,而应返回 nil 对象。例如,Ruby 库应返回“nil”,而 C 库应返回 NULL(或在回复对象中设置特殊标志)。

  Bulk strings (multi-line strings, Bulk Strings) $start with the " " symbol and are followed by 字符串长度. Example: hello dbydc

Transmission format:$11\r\nhello dbydc\r\n

  Multi-line strings can also represent single-line strings. The multiline string "hello dbydc" has length 11 (hello takes 5, a space takes 1, and dbydc takes 5, so 5 + 1 + 5 = 11).
 

1.3 Integers

  This type is just a CRLF-terminated string that represents an integer, prefixed by a “:” byte. For example, “:0\r\n” and “:1000\r\n” are integer replies.
  这种类型只是一个以 CRLF 结尾的字符串,表示一个整数,前缀为“:”字节。例如,“:0\r\n”和“:1000\r\n”是整数回复。
 
  Many Redis commands return RESP Integers, like INCR, LLEN, and LASTSAVE.
  许多 Redis 命令返回 RESP 整数,例如 INCR、LLEN 和 LASTSAVE。
 
  There is no special meaning for the returned integer. It is just an incremental number for INCR, a UNIX time for LASTSAVE, and so forth. However, the returned integer is guaranteed to be in the range of a signed 64-bit integer.
  返回的整数没有特殊含义。它只是 INCR 的递增数字,LASTSAVE 的 UNIX 时间等等。但是,返回的整数保证在带符号的 64 位整数范围内。
 
  Integer replies are also used in order to return true or false. For instance, commands like EXISTS or SISMEMBER will return 1 for true and 0 for false.
  还使用整数回复来返回 true 或 false。例如,像 EXISTS 或 SISMEMBER 这样的命令将返回 1 表示真,0 表示假。
 
  Other commands like SADD, SREM, and SETNX will return 1 if the operation was actually performed and 0 otherwise.
  如果实际执行了操作,其他命令如 SADD、SREM 和 SETNX 将返回 1,否则返回 0。
 
  The following commands will reply with an integer: SETNX, DEL, EXISTS, INCR, INCRBY, DECR, DECRBY, DBSIZE, LASTSAVE, RENAMENX, MOVE, LLEN, SADD, SREM, SISMEMBER, SCARD.
  以下命令将以整数回复:SETNX、DEL、EXISTS、INCR、INCRBY、DECR、DECRBY、DBSIZE、LASTSAVE、RENAMENX、MOVE、LLEN、SADD、SREM、SISMEMBER、SCARD。

  Integers ( Integers) :begin with the " " symbol and are followed by 整数的字符串形式. Example: 666

Transmission format::666\r\n

 

1.4 Errors

  RESP has a specific data type for errors. They are similar to RESP Simple Strings, but the first character is a minus ‘-’ character instead of a plus. The real difference between Simple Strings and Errors in RESP is that clients treat errors as exceptions, and the string that composes the Error type is the error message itself.
  RESP 具有特定的错误数据类型。它们类似于 RESP 简单字符串,但第一个字符是减号“-”而不是加号。简单字符串和 RESP 中的错误之间的真正区别在于客户端将错误视为异常,而组成错误类型的字符串是错误消息本身。
 
  The basic format is:
  基本格式为:
 
  “-Error message\r\n”
 
  Error replies are only sent when something goes wrong, for instance if you try to perform an operation against the wrong data type, or if the command does not exist. The client should raise an exception when it receives an Error reply.
  错误回复仅在出现问题时发送,例如,如果您尝试对错误的数据类型执行操作,或者命令不存在。客户端在收到错误回复时应引发异常。
 
  The following are examples of error replies:
  以下是错误回复的示例:
 
  -ERR unknown command ‘helloworld’
  -WRONGTYPE Operation against a key holding the wrong kind of value
 
  The first word after the “-”, up to the first space or newline, represents the kind of error returned. This is just a convention used by Redis and is not part of the RESP Error format.
  “-”之后的第一个单词,直到第一个空格或换行符,表示返回的错误类型。这只是 Redis 使用的约定,不是 RESP 错误格式的一部分。
 
  For example, ERR is the generic error, while WRONGTYPE is a more specific error that implies that the client tried to perform an operation against the wrong data type. This is called an Error Prefix and is a way to allow the client to understand the kind of error returned by the server without checking the exact error message.
  例如,ERR 是一般错误,而 WRONGTYPE 是更具体的错误,表示客户端尝试对错误的数据类型执行操作。这称为错误前缀,是一种允许客户端了解服务器返回的错误类型而无需检查确切错误消息的方法。
 
  A client implementation may return different types of exceptions for different errors or provide a generic way to trap errors by directly providing the error name to the caller as a string.
  客户端实现可能会针对不同的错误返回不同类型的异常,或者通过直接将错误名称作为字符串提供给调用者来提供捕获错误的通用方法。
 
  However, such a feature should not be considered vital as it is rarely useful, and a limited client implementation may simply return a generic error condition, such as false.
  然而,这样的功能不应该被认为是重要的,因为它很少有用,而且有限的客户端实现可能只是返回一个一般的错误条件,比如 false。

  Errors ( Errors) -begin with a " " symbol. Example: (error) WRONGPASS invalid username-password pair or user is disabled.

Transmission format:-WRONGPASS invalid username-password pair or user is disabled.\r\n

 

1.5 Arrays

  Clients send commands to the Redis server using RESP Arrays. Similarly, certain Redis commands, that return collections of elements to the client, use RESP Arrays as their replies. An example is the LRANGE command that returns elements of a list.
  客户端使用 RESP 数组向 Redis 服务器发送命令。类似地,某些将元素集合返回给客户端的 Redis 命令使用 RESP 数组作为它们的回复。一个示例是返回列表元素的 LRANGE 命令。
 
  RESP Arrays are sent using the following format:
  RESP 数组使用以下格式发送:
 

  • A * character as the first byte, followed by the number of elements in the array as a decimal number, followed by CRLF.
  • * 字符作为第一个字节,后跟数组中的元素数(十进制数),然后是 CRLF。
  • An additional RESP type for every element of the Array.
  • Array 的每个元素的附加 RESP 类型。

 
  So an empty Array is just the following:
  所以一个空数组就是以下内容:
 
  “*0\r\n”
 
  While an array of two RESP Bulk Strings “hello” and “world” is encoded as:
  两个 RESP 批量字符串“hello”和“world”的数组被编码为:
 
  “*2\r\n$5\r\nhello\r\n$5\r\nworld\r\n”
 
  As you can see after the *<count>CRLF part prefixing the array, the other data types composing the array are just concatenated one after the other. For example, an Array of three integers is encoded as follows:
  如您所见,在数组前缀的 *<count>CRLF 部分之后,构成数组的其他数据类型只是一个接一个地连接在一起。例如,三个整数的数组编码如下:
 
  “*3\r\n:1\r\n:2\r\n:3\r\n”
 
  Arrays can contain mixed types, so it’s not necessary for the elements to be of the same type. For instance, a list of four integers and a bulk string can be encoded as follows:
  数组可以包含混合类型,因此元素不必是同一类型。例如,一个包含四个整数的列表和一个批量字符串可以编码如下:
 
  *5\r\n
  :1\r\n
  :2\r\n
  :3\r\n
  :4\r\n
  $5\r\n
  hello\r\n
 
  (The reply was split into multiple lines for clarity).
  (为清楚起见,答复分为多行)。
 
  The first line the server sent is *5\r\n in order to specify that five replies will follow. Then every reply constituting the items of the Multi Bulk reply are transmitted.
  服务器发送的第一行是 *5\r\n 以指定随后将有五个回复。然后发送构成多批量回复项目的每个回复。
 
  Null Arrays exist as well and are an alternative way to specify a Null value (usually the Null Bulk String is used, but for historical reasons we have two formats).
  Null 数组也存在,并且是指定 Null 值的另一种方法(通常使用 Null Bulk String,但由于历史原因,我们有两种格式)。
 
  For instance, when the BLPOP command times out, it returns a Null Array that has a count of -1 as in the following example:
  例如,当 BLPOP 命令超时时,它返回一个计数为 -1 的空数组,如下例所示:
 
  “*-1\r\n”
 
  A client library API should return a null object and not an empty Array when Redis replies with a Null Array. This is necessary to distinguish between an empty list and a different condition (for instance the timeout condition of the BLPOP command).
  当 Redis 使用空数组回复时,客户端库 API 应该返回空对象而不是空数组。这是区分空列表和不同条件(例如 BLPOP 命令的超时条件)所必需的。
 
  Nested arrays are possible in RESP. For example a nested array of two arrays is encoded as follows:
  嵌套数组在 RESP 中是可能的。例如,两个数组的嵌套数组编码如下:
 
  *2\r\n
  *3\r\n
  :1\r\n
  :2\r\n
  :3\r\n
  *2\r\n
 
  (The format was split into multiple lines to make it easier to read).
  (格式被分成多行以便于阅读)。
 
  The above RESP data type encodes a two-element Array consisting of an Array that contains three Integers (1, 2, 3) and an array of a Simple String and an Error.
  上面的 RESP 数据类型编码一个双元素数组,该数组由一个包含三个整数 (1, 2, 3) 的数组和一个简单字符串数组和一个错误组成。
 
  Single elements of an Array may be Null. This is used in Redis replies to signal that these elements are missing and not empty strings. This can happen with the SORT command when used with the GET pattern option if the specified key is missing. Example of an Array reply containing a Null element:
  数组的单个元素可能为 Null。这在 Redis 回复中用于表示缺少这些元素而不是空字符串。如果缺少指定的键,则在与 GET 模式选项一起使用时,SORT 命令可能会发生这种情况。包含 Null 元素的数组回复示例:
 
  *3\r\n
  $5\r\n
  hello\r\n
  $-1\r\n
  $5\r\n
  world\r\n
 
  The second element is a Null. The client library should return something like this:
  第二个元素是 Null。客户端库应该返回如下内容:
 
  [“hello”,nil,“world”]
 
  Note that this is not an exception to what was said in the previous sections, but an example to further specify the protocol.
  请注意,这不是前面部分所说的例外,而是进一步指定协议的示例。

  An array ( Arrays) *begins with a " " symbol and is followed by an array 数组的长度. Example: [2,4,8]

Transmission format:*3\r\n:2\r\n:4\r\n:8\r\n

 

1.6 NULL

  RESP Bulk Strings can also be used in order to signal non-existence of a value using a special format to represent a Null value. In this format, the length is -1, and there is no data. Null is represented as:
  RESP 批量字符串也可用于使用特殊格式表示值不存在来表示 Null 值。这种格式,长度为-1,没有数据。Null 表示为:
 
  “$-1\r\n”
 
  This is called a Null Bulk String.
  这称为Null批量字符串。
 
  The client library API should not return an empty string, but a nil object, when the server replies with a Null Bulk String. For example, a Ruby library should return ‘nil’ while a C library should return NULL (or set a special flag in the reply object).
  当服务器回复 Null Bulk String 时,客户端库 API 不应返回空字符串,而应返回 nil 对象。例如,Ruby 库应返回“nil”,而 C 库应返回 NULL(或在回复对象中设置特殊标志)。

  NULL values ​​are represented by batch strings (multi-line strings, Bulk Strings), and the length is " -1".

Transmission format:$-1\r\n

 

1.7 empty string

  Empty strings are represented by bulk strings (multi-line strings, Bulk Strings), with a length of " 0".

Transmission format:$0\r\n\r\n

 

2. The command format sent by the client (Client) to the server (Server)

 
  The client (Client) sends to the server (Server) 仅包含批量字符串(多行字符串,Bulk Strings)的 RESP 数组(Array). For example, if the set command sends "set author dbydc", it will be serialized into the following string.

*3\r\n$3\r\nset\r\n$6\r\nauthor\r\n$5\r\ndbydc\r\n
 
Another style display:
 
*3
$3
set
$6
author
$5
dbydc

 

3. The response format of the server (Server) to the client (Client)

 
  The response from the server (Server) to the client (Client) supports multiple data formats.
 

3.1 Single-line string (simple string, Simple Strings) response

127.0.0.1:6379> set author dbydc
OK

  The response of the reply is "OK", without using double quotes.

+OK

 

3.2 Multi-line string (bulk string, Bulk Strings) response

127.0.0.1:6379> get author
“dbydc”

  Strings for multiline string responses are enclosed in double quotes.

$5\r\ndbydc\r\n

 

3.3 Integers response

127.0.0.1:6379> incr age
(integer) 1

  An integer response returns 1.

:1

 

3.4 Errors Response

127.0.0.1:6379> incr author
(error) ERR value is not an integer or out of range

  If the string is incremented, the server throws a generic error.

-ERR value is not an integer or out of range

 

3.5 Arrays Response

127.0.0.1:6379> hset user name dbydc
(integer) 1
127.0.0.1:6379> hset user age 18
(integer) 1
127.0.0.1:6379> hset user sex male
(integer) 1
127.0.0.1:6379> hgetall user
1) “name”
2) “dbydc”
3) “age”
4) “18”
5) “sex”
6) “male”

  hgetall returns an array. The strings at the 0th, 2nd, and 4th positions are the keys of the hash table, namely "name", "age", and "sex". The strings at the 1st, 3rd, and 5th positions are the values ​​of the hash table, namely "dbydc", "18", and "male".

*6\r\n$4\r\nname\r\n$5\r\ndbydc\r\n$3\r\nage\r\n$2\r\n18\r\n$3\r\nsex\r\n$4\r\nmale\r\n
 
Another style display:
 
*6
$4
name
$5
dbydc
$3
age
$2
18
$3
sex
$4
male

 

3.6 Nested responses

127.0.0.1:6379> scan 0
1) “0”
2)  1) “author”
   2) “user”
   3) “age”

  The scan command uses a cursor to scan all the key lists contained in the server, and only obtains a part each time.
 
  The scan command returns a nested array, the first value represents the value of the cursor, if it is 0, it means that the traversal has been completed. If it is not 0, the scan command will use this value as a parameter for the next iteration. Array The second value is an array again, this array is a list of keys.

*2\r\n$1\r\n0\r\n*3\r\n$6\r\nauthor\r\n$4\r\nuser\r\n$3\r\nage\r\n
 
Another style display:
 
*2
$1
0
*3
$6
author
$4
user
$3
age

4. Data reference

  1. "Redis Deep Adventure Core Principles and Application Practice", author: Qian Wenpin

Guess you like

Origin blog.csdn.net/u014282578/article/details/128575040
Recommended