"Redis Design and Implementation" study notes - publish and subscribe, transactions, slow query logs

Publish and subscribe

Redis provides one-to-many or even many-to-many node message communication through publish and subscribe. Publish and subscribe are composed of commands such as PUBLISH, SUBSCRIBE, PSUBSCRIBE, and PUBSUB.

• SUBSCRIBE command: subscribe to a channel, save the subscription relationship of all channels of the current server through the pubsub_channels dictionary attribute in the redisServer structure, the dictionary key is the channel name, and the dictionary value is a linked list, which records all the clients subscribed to this channel.
• UNSUBSCRIBE command: Unsubscribe from a channel. After calling this command, the subscription relationship will be deleted from pubsub_channels. If the linked list corresponding to the key is empty, the key will be deleted from the dictionary.
• PSUBSCRIBE command: subscription mode, the server saves all mode subscription relationships in the pubsub_patterns attribute of the redisServer structure, which is a linked list, and each linked list node contains a pubsubPattern structure, which records the subscribed pattern and the subscribers subscribed to the pattern. client.
• PUNSUBSCRIBE command: Unsubscribe mode. After calling this command, the server will traverse the pubsub_patterns linked list and delete the node that matches both the client and the pattern.
• PUBLISH command: Send a subscription message. After the server receives the command, it first traverses pubsub_channels to find channel subscribers, sends the message to all channel subscribers, then traverses pubsub_patterns to find the pattern matching the channel, and sends the message to the subscribers clients of these modes.
• The PUBSUB command to view subscription information includes the following three subcommands:
• PUBSUB CHANNLES [pattern]: Returns the channel that the current server is subscribed to. If pattern is specified, it returns the channel matching the pattern
• PUBSUB NUMSUB [channel-1 ... channel-n]: Returns the number of subscribers for the specified channel
• PUBSUB NUMPAT: Returns the number of schemas the server is currently subscribed to

affairs

Redis implements transaction functions through commands such as MULTI, EXEC, WATCH, etc., to package multiple commands and execute them sequentially at one time. A transaction starts with a MULTI command and ends with an EXEC command. The command executed after executing the MULTI command is not executed immediately, but returns a QUEUED to the client, which will be executed after the EXEC command is executed.

the realization of the transaction

business begins

After executing the MULTI command, the server switches the client from the non-transactional state to the transactional state, and turns on the REDIS_MULTI flag by setting the flags flag of redisClient.

command enqueue

When the client is in a non-transactional state, the commands sent by the client are executed immediately. When in a transactional state:

If the command sent by the client is one of EXEC, DISCARD, WATCH, and MULTI, the server executes it immediately.
If the command sent by the client is another command, the server will not execute the command immediately, but put the command in the transaction queue, and then return QUEUED to the client.
There is a multiState structure attribute mstate in redisClient to record the transaction state of the client, and there is a commands list attribute in the multiState structure, which records the commands that the client queues after calling MULTI, and the count attribute records the number of queued commands. The multiCmd structure records the command implementation function pointer, the parameters of the command, and the number of parameters. The transaction queue saves the enqueued commands in a FIFO manner, and the first enqueued command is executed first.
Execute the transaction
Execute the EXEC command to submit the transaction, after the server receives the command:

1. Traverse client.mstate.commands, get the queued commands and execute them one by one, and add the return value of the command to the end of the reply queue.
2. Clears the client's transaction status, including clearing the enqueued command counter and releasing the transaction queue.
3. Return the execution result to the client.

WATCH command implementation

The WATCH command is an optimistic lock. It monitors the specified database keys before executing EXEC. When executing the EXEC command, it checks whether at least one of the monitored keys has been modified. If so, the server refuses to execute the command and sends a message to the client. Returns an empty reply representing transaction execution failure.
The redisDb structure of the Redis database stores a watched_keys dictionary. The key of the dictionary is a key monitored by the WATCH command, and the value is a linked list. The linked list records all the clients that monitor the corresponding database key.
When executing a command to modify the database, the multi.c/touchWatchKey function will be called after execution to check the watched_keys dictionary to see if any client is monitoring the database key that has just been modified by the command. If so, the touchWatchKey function will monitor The REDIS_DIRTY_CAS flag of the client whose key is modified is turned on, indicating that the client's transaction security has been violated. When the server receives an EXEC command from a client, the server decides whether to execute the transaction according to whether the client has turned on the REDIS_DIRTY_CAS flag.

ACID of the transaction

Atomicity: For a Redis transaction, multiple commands in the transaction are treated as a whole, and the commands in the transaction queue are either all executed or none of them are executed. Redis transactions are atomic, but the Redis transaction department supports rollback , even if an error occurs in the execution of a command in the queue, it will not affect the subsequent commands, and the entire transaction will continue to be executed.
Consistency: The consistency of the database will not be affected by command enqueue errors, execution errors, or server downtime. ps: When there is an error in the queue, the transaction will continue to be executed for versions before 2.6.5. After 2.6.5, the transaction will be refused to execute.
Isolation: Redis transaction will not be interrupted by other commands during execution, and it will run in serial mode. Transactions are
persistent with isolation:

• When the server is running in non-persistent in-memory mode, transactions are not durable: once the server goes down, all data is lost
• When the server runs in RDB persistence mode, data will be lost for a period of time when the server is down, and transactions are not durable
• When the server runs in AOF persistence mode and the appendsync option value is always, the data will be saved to the hard disk in time, and the transaction is persistent (provided that the no-appendfsync-on-rewrite option is not turned on, turn this option on After that, in order to reduce IO operations as much as possible, when the server executes BGSAVE or BGREWRITEAOF, it will stop the AOF file synchronization, so the transaction is not persistent at this time)
• When the server runs in AOF persistence mode and the appendsync option value is everysec, 1 second of data may be lost, and the transaction is not durable.
• When the server runs in AOF persistence mode and the appendsync option value is no, the data will be lost when the server stops, and the transaction is not durable.

slow query log

The slow query log function of Redis is used to record command requests whose execution time exceeds a given time. Users can monitor and optimize the query speed through the log generated by this function.

The server configuration has two options related to the slow query log:

• slowlog-log-slower-than The option specifies how many microseconds ( 1 seconds equals  1,000,000 microseconds) command requests that execute longer than will be logged.

  For example, if the value of this option is the value  100 , then the command execution time longer than  100 microseconds will be recorded in the slow query log; if the value of this option is  500 then  the 500 command execution time longer than microseconds will be recorded in the slow query log. log; and so on.

• slowlog-max-len The option specifies how many slow query logs the server can save at most.

  The server saves multiple slow query logs in a first-in, first-out manner: When the number of slow query logs stored by the server is equal to  slowlog-max-len the value of the option, the server will delete the oldest slow query log before adding a new slow query log. .

  For example, if  slowlog-max-len the value of the server is  100 , and assuming that the server has already stored a  100 slow query log, then if the server intends to add a new log, it must first delete the oldest currently saved log, and then add a new one. log.