Suppose there is such a business now. Some data obtained by users comes from third-party interface information. In order to avoid frequent requests for third-party interfaces, we often add a layer of cache. The cache must be time-sensitive. Assuming that the structure we want to store is hash (There is no atomic operation like'SET anotherkey "will expire in a minute" EX 60' of String), we have to put them in the cache in batches, and also ensure that each key is added with an expiration time (in case the key never expires ), this time transaction operation is a better choice.
In order to ensure the atomicity of multiple consecutive operations, our commonly used databases will all have transaction support, and Redis is no exception. But it is not the same as a relational database.
Each transaction operation has begin, commit and rollback, begin indicates the beginning of the transaction, commit indicates the commit of the transaction, and rollback indicates the rollback of the transaction. Its rough form is as follows
begin();try { command1(); command2(); .... commit();} catch(Exception e) { rollback();}
Redis looks similar in form, divided into three stages
- Open transaction (multi)
- Command enqueue (business operation)
- Execute transaction (exec) or cancel transaction (discard)
> multiOK> incr starQUEUED> incr starQUEUED> exec(integer) 1(integer) 2
The above instruction demonstrates a complete transaction process. All instructions are not executed before exec, but are cached in a transaction queue of the server. Once the server receives the exec instruction, the entire transaction queue will be disassembled and executed, and once executed Return the results of all instructions.
Redis transactions can execute multiple commands at once, and are essentially a collection of commands. All commands in a transaction will be serialized, serialized and executed in order without being inserted by other commands, and no jamming is allowed.
It can ensure that a series of commands are executed in a one-time, sequential, and exclusive manner in a queue (the main function of Redis transactions is actually to chain multiple commands to prevent other commands from jumping in the queue)
The official document says so
A transaction can execute multiple commands at once, with the following two important guarantees:
The transaction is a separate isolated operation: all commands in the transaction are serialized and executed sequentially. During the execution of the transaction, it will not be interrupted by command requests sent by other clients. A transaction is an atomic operation: all commands in the transaction are either executed or not executed
This atomic operation is not the same as the atomicity of relational DB. It cannot fully guarantee the atomicity, which will be introduced later.
Several commands for Redis transactions
Soul blow! Are Redis transactions that do not support atomicity also called transactions?
The MULTI command is used to start a transaction, and it always returns OK.
After MULTI is executed, the client can continue to send any number of commands to the server. These commands will not be executed immediately, but will be placed in a queue. When the EXEC command is called, all the commands in the queue will be executed.
On the other hand, by calling DISCARD, the client can clear the transaction queue and give up executing the transaction.
Don’t talk too much nonsense, it’s better to understand the results directly by operating it~
Smooth sailing
Normal execution (batch processing is possible, very cool, each operation will take what it needs if it succeeds, without affecting each other)
Soul blow! Are Redis transactions that do not support atomicity also called transactions?
Abandon the transaction (discard operation means abandon the transaction, the previous operations do not count)
Soul blow! Are Redis transactions that do not support atomicity also called transactions?
Consider a question: Suppose we have a key with an expiration time, and the key fails during transaction operations, will it succeed when executing exec?
Error in transaction
The above regular operations look pretty good, but transactions are proposed to solve data security operations . When we use Redis transactions, we may encounter the following two errors:
- Before the transaction executes EXEC, the enqueued command may be wrong. For example, the command may generate syntax errors (wrong number of parameters, wrong parameter names, etc.), or other more serious errors, such as insufficient memory (if the server uses maxmemory to set the maximum memory limit).
- The command may fail after EXEC is called. For example, the command in the transaction may handle the wrong type of key, such as using the list command on the string key, and so on.
Redis adopts different processing strategies for the above two errors. For errors that occur before the execution of EXEC, the server will record the failure of the command enqueue, and when the client calls the EXEC command, it will refuse to execute and automatically abandon the transaction. (The method before Redis 2.6.5 was to check the return value of the command enqueue: if the command returns QUEUED when enqueue, then enqueue is successful; otherwise, enqueue failed)
For those errors that are generated after the execution of the EXEC command, no special treatment is given to them: even if an error occurs during the execution of a certain/certain command in the transaction, other commands in the transaction will continue to be executed.
Sit all together (If an error is reported in an operation record, all operations after exec will not succeed)
Soul blow! Are Redis transactions that do not support atomicity also called transactions?
The accuser (in the example k1 is set to String type, decr k1 can be placed in the operation queue, because only when it is executed can the statement be judged wrong, and other correct ones will be executed normally)
Soul blow! Are Redis transactions that do not support atomicity also called transactions?
Why does Redis not support rollback
If you have experience with relational databases, the practice of "Redis does not roll back when the transaction fails, but continues to execute the remaining commands" may make you feel a little strange.
The following is the official boast :
Redis commands will only fail because of incorrect syntax (and these problems cannot be found when enqueuing), or the command is used on the wrong type of key: that is, from a practical point of view, the failed command Is caused by programming errors, and these errors should be found in the development process, and should not appear in the production environment. Because there is no need to support rollbacks, the internals of Redis can be kept simple and fast.
There is a view that Redis's transaction processing practices will produce bugs. However, it should be noted that under normal circumstances, rollbacks cannot solve the problems caused by programming errors. For example, if you originally wanted to add 1 to the value of the key through the INCR command, but accidentally added 2, or executed INCR for the wrong type of key, there is no way to handle these situations with rollback.
In view of the fact that there is no mechanism to avoid errors caused by programmers themselves, and such errors usually do not appear in the production environment, Redis chose a simpler and faster way to handle transactions without rollback.
Soul blow! Are Redis transactions that do not support atomicity also called transactions?
Transaction with Watch
The WATCH command is used to monitor any number of keys before the transaction starts: When the EXEC command is called to execute the transaction, if any monitored key has been modified by other clients, the entire transaction will be interrupted and no longer executed, directly Return failed.
The WATCH command can be called multiple times. The monitoring of the keys will take effect after WATCH is executed, until EXEC is called.
Users can also monitor any number of keys in a single WATCH command, like this:
redis> WATCH key1 key2 key3 OK
When EXEC is called, the monitoring of all keys will be cancelled regardless of whether the transaction is successfully executed . In addition, when the client disconnects, the client's monitoring of the keys will also be cancelled.
Let's look at a simple example, use watch to monitor my account balance (I have 100 pocket money a week) and spend normally
Soul blow! Are Redis transactions that do not support atomicity also called transactions?
But this card is also bound to my daughter-in-law’s Alipay. What if she also consumes while I consume?
Sleepy, I went to 711 downstairs to buy a pack of cigarettes and a bottle of water. At this time, my daughter-in-law brushed 100 at the supermarket. At this time, I was still picking gum when the balance was insufficient.
Soul blow! Are Redis transactions that do not support atomicity also called transactions?
At this time, I went to checkout and found that the credit card failed (transaction interrupted), an awkward batch
Soul blow! Are Redis transactions that do not support atomicity also called transactions?
You may not understand the use of watch. Let's look at it again. If it is still the same scenario, we don't have watch balance, the transaction will not fail, and the savings card becomes negative. Isn't it not suitable for business?
Soul blow! Are Redis transactions that do not support atomicity also called transactions?
Use the UNWATCH command without parameters to manually cancel the monitoring of all keys. For some transactions that need to change multiple keys, sometimes the program needs to lock multiple keys at the same time, and then check whether the current values of these keys meet the requirements of the program. When the value does not meet the requirements, you can use the UNWATCH command to cancel the current monitoring of the key, abandon the transaction halfway, and wait for the next attempt of the transaction.
The watch instruction is similar to optimistic locking . When the transaction is committed, if the value of the key has been changed by another client, for example, a list has been pushed/popped by another client, the entire transaction queue will not be executed. (Of course, Redis can also be used to implement distributed locks to ensure security, which is a pessimistic lock)
Multiple keys are monitored before the transaction is executed through the watch command. If any key value changes after the watch, the transaction executed by the exec command will be abandoned, and a Null response will be returned to notify the caller that the transaction failed.
Pessimistic lock
Pessimistic Lock (Pessimistic Lock), as the name suggests, is very pessimistic. Every time you get the data, you think that others will modify it, so every time you get the data, you will lock it, so that if someone wants to get the data, it will block until it gets it. To the lock. Many such locking mechanisms are used in traditional relational databases, such as row locks, table locks, etc., read locks, write locks, etc., all of which are locked before performing operations
Optimistic **** lock
Optimistic Lock (Optimistic Lock), as the name suggests, is very optimistic. Every time I get the data, I think that others will not modify it, so it will not be locked, but when updating, it will judge whether others have updated this during this period. For data, mechanisms such as version numbers can be used. Optimistic locking is suitable for multi-read application types, which can improve throughput. Optimistic locking strategy: the submitted version must be greater than the current version of the record to perform the update
The realization principle of WATCH command
In the server.h/redisDb structure type representing the database, a watched_keys dictionary is stored. The key of the dictionary is the monitored key of the database, and the value of the dictionary is a linked list. The linked list stores all the clients that monitor this key. ,As shown below.
Soul blow! Are Redis transactions that do not support atomicity also called transactions?
typedef struct redisDb { dict *dict; /* The keyspace for this DB */ dict *expires; /* Timeout of keys with a timeout set */ dict *blocking_keys; /* Keys with clients waiting for data (BLPOP)*/ dict *ready_keys; /* Blocked keys that received a PUSH */ dict *watched_keys; /* WATCHED keys for MULTI/EXEC CAS */ int id; /* Database ID */ long long avg_ttl; /* Average TTL, just for stats */ unsigned long expires_cursor; /* Cursor of the active expire cycle. */ list *defrag_later; /* List of key names to attempt to defrag one by one, gradually. */} redisDb;list *watched_keys; /* Keys WATCHED for MULTI/EXEC CAS */
The function of the WATCH command is to associate the current client with the key to be monitored in watched_keys.
For example, if the current client is client99, then when the client executes WATCH key2 key3, the watched_keys shown above will be modified to look like this:
Soul blow! Are Redis transactions that do not support atomicity also called transactions?
Through the watched_keys dictionary, if the program wants to check whether a key is monitored, it only needs to check whether the key exists in the dictionary; if the program wants to obtain all clients that monitor a key, then just take out the value of the key (a linked list). ), and then traverse the linked list.
After any command to modify the key space of the database is successfully executed (such as FLUSHDB, SET, DEL, LPUSH, SADD, etc.), the multi.c/touchWatchedKey function will be called-it will go to the watched_keys dictionary, see Is there a client monitoring the key that has been modified by the command? If so, the program turns on the REDIS_DIRTY_CAS option of all clients that monitor this/these modified keys:
Soul blow! Are Redis transactions that do not support atomicity also called transactions?
void multiCommand(client *c) { // 不能在事务中嵌套事务 if (c->flags & CLIENT_MULTI) { addReplyError(c,"MULTI calls can not be nested"); return; } // 打开事务 FLAG c->flags |= CLIENT_MULTI; addReply(c,shared.ok);}/* "Touch" a key, so that if this key is being WATCHed by some client the * next EXEC will fail. */void touchWatchedKey(redisDb *db, robj *key) { list *clients; listIter li; listNode *ln; // 字典为空，没有任何键被监视 if (dictSize(db->watched_keys) == 0) return; // 获取所有监视这个键的客户端 clients = dictFetchValue(db->watched_keys, key); if (!clients) return; // 遍历所有客户端，打开他们的 CLIENT_DIRTY_CAS 标识 listRewind(clients,&li); while((ln = listNext(&li))) { client *c = listNodeValue(ln); c->flags |= CLIENT_DIRTY_CAS; }}
When the client sends an EXEC command to trigger transaction execution, the server checks the status of the client:
- If the CLIENT_DIRTY_CAS option of the client has been turned on, it means that at least one key monitored by the client has been modified, and the security of the transaction has been destroyed. The server will abandon the execution of this transaction and directly return an empty reply to the client, indicating that the transaction has failed.
- If the CLIENT_DIRTY_CAS option is not turned on, then all monitoring keys are safe and the server officially executes the transaction.
Small summary:
3 stages
- Open: start a transaction with MULTI
- Enqueue: Enqueue multiple commands into the transaction. These commands will not be executed immediately after receiving them, but will be placed in the transaction queue waiting to be executed
- Execution: The transaction is triggered by the EXEC command
3 features
- Separate isolation operation: All commands in the transaction will be serialized and executed in order. During the execution of the transaction, it will not be interrupted by command requests sent by other clients.
- There is no concept of isolation level : the commands in the queue will not be actually executed before they are submitted, because any instructions will not be actually executed before the transaction is submitted, and there is no "query within the transaction to see the update in the transaction" , The query can’t be seen outside the transaction" this is an extremely headache problem
- Atomicity is not guaranteed: If a command fails to execute in the same transaction in Redis, the subsequent commands will still be executed without rollback
In traditional relational databases, ACID properties are often used to test the security of transaction functions. Redis transactions guarantee consistency (C) and isolation (I), but do not guarantee atomicity (A) and durability (D).
At last
Redis transactions must go through a network read and write when sending each instruction to the transaction cache queue. When there are many instructions in a transaction, the required network IO time will also increase linearly. Therefore, usually Redis clients will be used in conjunction with pipelines when executing transactions, so that multiple IO operations can be compressed into a single IO operation.