Ubiquitous locks skilled in Redis

In order to ensure the correctness of concurrent access, Redis provides two methods, namely locking and atomic operation

There are two problems with Redis locking: one is that if there are many locking operations, the concurrent access performance of the system will be reduced; the second is that when the Redis client wants to lock, it needs to use distributed locks, and the implementation of distributed locks is complicated , an additional storage system is required to provide lock and unlock operations

One: lock-free atomic operations

Atomic operations are another way to provide concurrency access control. Atomic operations refer to operations that maintain atomicity in the execution process, and no additional locks are required during execution of atomic operations, realizing lock-free operations. In this way, concurrency control can be guaranteed and the impact on system concurrency performance can be reduced.

1. What needs to be controlled during concurrent access?

1.1 What is concurrent access control?
It refers to controlling the process of multiple clients accessing and operating the same data, so as to ensure that the operations sent by any client are mutually exclusive when executed on the Redis instance. For example, when the access operation of the client A is being executed, the operation of the client B cannot be executed, and the operation of the client B cannot be executed until the operation of the client A is completed.
The operations corresponding to concurrent access control are mainly data modification operations. When the client needs to modify the data, the basic process is divided into two steps:
1. The client first reads the data locally and modifies it locally;
2. After the client modifies the data, it writes it back to Redis.
This process is called "read-modify-write back" operation (Read-Modify-Write, referred to as RMW operation). When multiple clients perform RMW operations on the same data, let the codes involved in RMW operations be executed atomically. The RMW operation code that accesses the same data is called the critical section code. When multiple clients execute critical section code concurrently, there are some potential problems

1.2 Example of the client updating the commodity inventory:
Suppose the client wants to perform the operation of deduction 1 on the commodity inventory, the pseudo code is as follows:

current = GET(id)
current--
SET( id,current)

It can be seen that the client will first read the current inventory value current (corresponding to Read) from Redis according to the commodity id, then, the client will subtract 1 from the inventory value (corresponding to Modify), and then write the inventory value back to Redis ( Corresponding to Write). When multiple clients execute this code, this is a critical section of code.

If there is no control mechanism for the execution of critical section code, data update errors will occur. In the previous example, assuming that there are two clients A and B executing the critical section code at the same time, an error will occur:

If processed according to the correct logic, clients A and B each deduct the inventory value once, and the inventory value should be 8. Therefore, the inventory value here is obviously updated wrongly.
The reason for this phenomenon is that the client in the critical section code involves three operations to read data, update data, and write back data, and these three operations are not mutually exclusive when executed, multiple clients Modify based on the same initial value, rather than based on the value modified by the previous client.

In order to ensure the correctness of concurrent modification of data, locks can be used to turn parallel operations into serial operations, and serial operations are mutually exclusive. After a client holds the lock, other clients can only wait until the lock is released before they can take the lock and modify it.
The following pseudocode shows the use of locks to control the execution of critical section code, you can look at it.

LOCK()
current = GET(id)
current--

Although locking ensures mutual exclusivity, locking will also reduce the concurrency performance of the system.
As shown in the figure below, when client A locks and executes operations, clients B and C need to wait. After A releases the lock, assuming that B gets the lock, then C still needs to continue to wait. Therefore, only A can access the shared data during the t1 period, and only B can access the shared data during the t2 period. Of course, the concurrency performance of the system will drop.

Similar to locking, atomic operations can also achieve concurrency control, but atomic operations have less impact on system concurrency performance.

2. Two atomic operation methods of Redis

In order to achieve the mutually exclusive execution of critical section codes required by concurrency control, Redis's atomic operations use two methods:

1. Implement multiple operations into one operation in Redis, that is, single command operation;
2. Write multiple operations into a Lua script and execute a single Lua script atomically.

2.1 Single-command operation of Redis itself.
Redis uses a single thread to serially process the client's request operation commands, so when Redis executes a certain command operation, other commands cannot be executed, which is equivalent to the fact that the command operations are mutually exclusive. Of course, Redis's snapshot generation and AOF rewriting operations can be executed using background threads or sub-processes, that is, they are executed in parallel with the operations of the main thread. However, these operations only read data, do not modify data, and do not require concurrency control for them.
Although a single command operation of Redis can be executed atomically, in practical applications, data modification may include multiple operations, including at least three operations of reading data, adding and subtracting data, and writing back data, which is obviously not a single command operation Now, what should I do then?
Value-added/decremented operations are performed, and they are a single command operation. When Redis executes them, it is mutually exclusive.
For example, in the example of inventory deduction just now, the client can use the following code to directly complete the operation of subtracting 1 from the inventory value of the product id. Even if multiple clients execute the following code, there is no need to worry about the problem of inventory value deduction errors.

decr id

If the RMW operation performed is to increase or decrease the value of the data, the atomic operations INCR and DECR provided by Redis can directly perform concurrency control.
The operation performed is not simply adding or subtracting data, so the single-command operation of Redis can no longer guarantee the mutually exclusive execution of multiple operations

2.2 Lua script:
Redis will execute the entire Lua script as a whole. It will not be interrupted by other commands during execution, thus ensuring the atomicity of operations in Lua scripts. There are multiple changes to be executed, which cannot be realized by command operations such as INCR/DECR. You can write the executed operations into a Lua script, and you can use the EVAL command of Redis to execute the script. In this way, these operations are mutually exclusive when performed.

2.3 Lua usage example:
When the number of access users of a business application increases, sometimes it is necessary to limit the number of visits of a certain client within a certain period of time, such as the purchase current limit of popular products, and the number of likes per minute in social networks restrictions etc.
So how to limit it?
You can use the client IP as the key and the number of client visits as the value, and save it in Redis. After each client visit, we use INCR to increase the number of visits.
In this scenario, the client-side traffic limit actually includes restrictions on the number of visits and the time range, for example, the number of visits per minute cannot exceed 20. You can set an expiration time for the corresponding key-value pair when the client accesses for the first time, for example, set it to expire after 60s.
Every time the client visits, read the current number of visits of the client. If the number exceeds the threshold, an error will be reported and the client will be restricted from visiting again.

//获敏ip对应的访同次数
current = GET(ip)
//如果超过访问次数超过20次,则报错
IF current != NULL  AND current >20 THEN
		ERROR "exceed 20 accesses persecond"
ELSE
    //如果访问次数不定20次,增加一次访问计数
    value = INCR(ip)
    //如果是第一次访间,将键值对的过期时间设置为60s后
		IF value == 1  THEN
      EXPiRE(ip,60)
	  END
    //执行其他噪作
		DO THINGS

In this example, INCR is used to increment the count atomically. However, the logic of client-side current limiting includes not only counting, but also access count judgment and expiration time setting. For these operations, it is also necessary to ensure their atomicity.

Otherwise, if the client uses multi-threaded access, the initial value of the number of visits is 0. After the first thread executes the INCR(ip) operation, the second thread also executes INCR(ip). At this time, the corresponding ip The number of visits has been increased to 2, and we can no longer set the expiration time for this ip. This will result in that after the number of client visits corresponding to this ip reaches 20 times, it will no longer be able to visit. Even after 60 seconds, you can no longer continue to visit, which obviously does not meet business requirements.

Therefore, the operation in this example cannot be implemented with a single Redis command. At this point, we can use Lua scripts to ensure concurrency control. We can write the three operations of adding 1 to the number of visits, judging whether the number of visits is 1, and setting the expiration time into a Lua script, as shown below:

local current

current = redis.call ( "incr", KEYS[1])
if tonumber (current) == 1 then
		redis.call( "expire" , KEYS[1],60)
end

Assuming that the name of the written script is lua.script, then you can use the Redis client with the eval option to execute the script. Parameters required by the script will be passed through keys and args in the following commands.
redis-cli --eval lua.script keys , args
In this way, the three operations of adding 1 to the number of visits, judging whether the number of visits is 1, and setting the expiration time can be performed atomically. Even if the client has multiple threads executing the script at the same time, Redis will execute the script code serially, avoiding data errors caused by concurrent operations.

3. Summary of lock-free atomic operations

During concurrent access, concurrent RMW (read-modify-write) operations will cause data errors, so concurrency control is required. The so-called concurrency control is to ensure the mutually exclusive execution of the critical section code.

Redis provides two methods of atomic operations to achieve concurrency control, namely single-command operations and Lua scripts. Because the atomic operation itself does not restrict access to too many resources, it can maintain high system concurrency performance.

However, the scope of application of single-command atomic operations is small, and not all RMW operations can be transformed into single-command atomic operations (for example, INCR/DECR commands can only perform atomic increase or decrease after reading data), and it is necessary to read When more judgments are made on the data, or when the modification of the data is not a simple increase or decrease, the single-command operation is not applicable.

Redis Lua scripts can contain multiple operations, which are executed atomically, bypassing the limitation of single-command operations. However, if many operations are executed atomically in Lua scripts, it will increase the time for Redis to execute scripts, and also reduce the concurrency performance of Redis.
Small suggestion: When writing Lua scripts, avoid writing operations that do not require concurrency control into the scripts.

When Redis executes Lua scripts, it can guarantee atomicity. Then, in the Lua script example (lua.script) I gave, do you think it is necessary to read the number of visits to the client ip, that is, GET(ip) , and the judgment logic for judging whether the number of visits exceeds 20, is it also added to the Lua script?

Answer: In this example, there are three operations to ensure atomicity, namely INCR, judging whether the number of visits is 1, and setting the expiration time. For the two operations of obtaining IP and judging whether the number of visits exceeds 20, they are only read operations. Even if the client has multiple threads executing these two operations concurrently, no value will be changed, so there is no need to guarantee atomicity. properties without putting them in Lua scripts.

Two: Distributed locks in Redis

Redis locking can deal with concurrency issues, to control the modification of shared data by concurrent write operations, so as to ensure the correctness of data. Note
:
Redis is a distributed system. When multiple clients need to compete for locks, they must be guaranteed. The lock cannot be a client-local lock. Otherwise, other clients will not be able to access the lock, and of course they will not be able to acquire the lock.
In a distributed system, when multiple clients need to acquire a lock, a distributed lock is required. At this point, the lock is stored in a shared storage system, which can be shared, accessed and acquired by multiple clients.
Redis itself can be shared and accessed by multiple clients, which happens to be a shared storage system that can be used to save distributed locks. Moreover, Redis has high read and write performance and can handle highly concurrent lock operation scenarios.

When writing programs every day, locks on a single machine are often used. Distributed locks are similar to locks on a single machine, but also
have some special requirements because distributed locks are used in distributed scenarios.

1. The connection and difference between locks on a single machine and distributed locks

1.1 Locks on a single machine
For a multi-threaded program running on a single machine, the lock itself can be represented by a variable.

• When the variable value is 0, it means that no thread acquires the lock;
• When the variable value is 1, it means that a thread has acquired the lock.

1.1.1 Instructions for threads to call lock and release locks
Locking
In fact, when a thread calls a lock operation, it actually checks whether the value of the lock variable is 0.
If it is 0, set the variable value of the lock to 1, indicating that the lock has been acquired.
If it is not 0, an error message is returned, indicating that the lock failed, and another thread has already acquired the lock.
Release lock
When a thread calls the release lock operation, it actually sets the value of the lock variable to 0 so that other threads can acquire the lock.
Operations of adding and releasing locks, where lock is a lock variable.

acquire__lock(){
    
    
  if lock ==0
     lock = 1
  	 return 1
	else
     return 0
}
release__lock(){
    
    
if lock = 0
  	return 1
}

1.1.2 Similarities between stand-alone locks and distributed locks:
Similar to locks on a stand-alone machine, distributed locks can also be implemented with a variable. The operation logic of locking and releasing the lock on the client side is also consistent with the operation logic of locking and releasing the lock on the single machine: when locking, it is necessary to judge the value of the lock variable, and judge whether the lock can be successfully locked according to the value of the lock variable; When locking, the value of the lock variable needs to be set to 0, indicating that the client no longer holds the lock.

1.1.3 Differences between stand-alone locks and distributed locks:
Unlike threads operating locks on a stand-alone machine, in a distributed scenario, lock variables need to be maintained by a shared storage system so that multiple clients can access shared storage system to access the lock variable. The operation of locking and releasing the lock becomes reading, judging and setting the value of the lock variable in the shared storage system.

1.2 Two requirements for implementing distributed locks.

1. Requirement 1: The process of locking and releasing a distributed lock involves multiple operations. Therefore, when implementing distributed locks, it is necessary to ensure the atomicity of these lock operations;
2. Requirement 2: The shared storage system saves the lock variable. If the shared storage system fails or goes down, the client cannot perform the lock operation. When implementing distributed locks, it is necessary to consider ensuring the reliability of the shared storage system, thereby ensuring the reliability of the lock.
   It can be implemented based on a single Redis node or using multiple Redis nodes. In these two cases, the reliability of the lock is not the same

2. Realize distributed lock based on a single Redis node

As a shared storage system in the process of implementing distributed locks, Redis can use key-value pairs to store lock variables, and then receive and process lock and release operation requests sent by different clients.
2.1 The key and value of the key-value pair are determined by
assigning a variable name to the lock variable as the key of the key-value pair—"the value of the lock variable is used as the value of the key-value pair—"Redis can save the lock variable, and the client can also The lock operation can be realized through the command operation of Redis.
It shows that Redis uses key-value pairs to save lock variables, and the operation process of two clients requesting locks at the same time.

It can be seen that Redis can use a key-value pair lock_key:0 to save the lock variable, where the key is lock_key, which is also the name of the lock variable, and the initial value of the lock variable is 0.

2.2 Locking operation:
In the figure, clients A and C request locking at the same time. Because Redis uses a single thread to process requests, even if clients A and C send lock requests to Redis at the same time, Redis will process their requests serially.

Assume that Redis processes the request of client A first, reads the value of lock_key, and finds that lock_key is 0, so Redis sets the value of lock_key to 1, indicating that it has been locked. Immediately afterwards, Redis processes the request of client C. At this time, Redis will find that the value of lock_key is already 1, so it returns the message that the lock failed.

2.3 Release the lock operation
To release the lock is to directly set the value of the lock variable to 0.
The picture below shows the process of client A requesting to release the lock. When client A holds the lock, the value of the lock variable lock_key is 1. After client A executes the lock release operation, Redis sets the value of lock_key to 0, indicating that no client holds the lock anymore.

Locking includes three operations (reading the lock variable, judging the value of the lock variable, and setting the value of the lock variable to 1), and these three operations need to ensure atomicity during execution.

2.4 How to ensure the atomicity of locked data:
To ensure the atomicity of operations, there are two general methods, namely, using Redis single-command operations and using Lua scripts.
Atomicity: Atomicity means that an operation is either executed successfully, or not executed at all, and there is no intermediate state. In other words, atomicity can ensure that a series of operations are an indivisible whole, either all of them succeed or all of them fail, thus ensuring the consistency and integrity of data.

In the distributed locking scenario, how should these two methods be applied?
2.4.1 What single-command operations can Redis use to implement locking operations: SETNX and del
The first is the SETNX command, which is used to set the value of the key-value pair. Specifically, this command will determine whether the key-value pair exists when it is executed. If it does not exist, set the value of the key-value pair. If it exists, it will not make any settings.

For example, if the key does not exist when the following command is executed, the key will be created and the value will be set to value; if the key already exists, SETNX will not perform any assignment.

SETNX key value

For the lock release operation, you can use the DEL command to delete the lock variable after executing the business logic. Don't worry that after the lock variable is deleted, other clients cannot request to lock it.
Because when the SETNX command is executed, if the key-value pair (that is, the lock variable) to be set does not exist, the SETNX command will first create the key-value pair and then set its value. Therefore, after the lock is released, when a client requests to lock again, the SETNX command will create a key-value pair to save the lock variable, and set the value of the lock variable to complete the lock.
To sum up, you can use the combination of SETNX and DEL commands to implement lock and release lock operations. The following pseudocode example shows the lock operation

/加锁
SETNX lock_key 1
/业务逻辑
Do THINGS
//释放锁
DEL lock_key

2.4.2 Using the combination of SETNX and DEL commands to implement distributed locks has two potential risks.
The first risk
If a client executes the SETNX command and locks, an exception occurs when operating the shared data, and as a result, the last DEL command has not been executed to release the lock. Therefore, the lock is always held by this client, and other clients cannot obtain the lock, access shared data and perform subsequent operations, which will affect business applications.
An effective solution is to set an expiration time for the lock variable. In this way, even if the client holding the lock has an exception and cannot actively release the lock, Redis will delete the lock variable after it expires according to the expiration time of the lock variable. After the lock variable expires, other clients can request to lock again, so there will be no problem of being unable to lock.

The second risk
If client A executes the SETNX command to lock, assuming that client B executes the DEL command to release the lock, at this time, the lock of client A is released by mistake. If client C happens to be applying for a lock, it can successfully obtain the lock, and then start to operate the shared data. Thus. Clients A and C are operating on the shared data at the same time, and the data will be modified incorrectly, which is also unacceptable by the business layer.
It is necessary to be able to distinguish lock operations from different clients.
It can be processed on the value of the lock variable:
in the method of locking using the SETNX command, by setting the value of the lock variable to 1 or 0, it indicates whether the lock is successful. 1 and 0 have only two states, which cannot indicate which client performs the lock operation. During the lock operation, each client can set a unique value for the lock variable, and the unique value here can be used to identify the current operating client. When releasing the lock, the client needs to judge whether the value of the current lock variable is equal to its own unique identifier. Only when they are equal can the lock be released, and the problem of releasing the lock by mistake will not occur.

In Redis, how is it implemented?

2.4.3 Single node guarantees atomic locking operation
SETNX command, for key-value pairs that do not exist, it will first create and then set the value (that is, "if it does not exist, set it"), in order to achieve the same effect as the SETNX command , Redis provides a similar option NX for the SET command, which is used to implement "set if it does not exist". If the NX option is used, the SET command will only be set when the key-value pair does not exist, otherwise no assignment will be performed. In addition, the SET command can also be executed with an EX or PX option to set the expiration time of the key-value pair.
Locking code implementation
For example, when the following command is executed, SET will create a key and assign a value to the key only if the key does not exist. In addition, the survival time of the key is determined by the seconds or milliseconds option value.

set key value  [EX seconds | PX milliseconds] [NX]

With the NX and EX/PX options of the SET command, we can use the following commands to implement the locking operation.

//加锁,unique_value作为客户端唯一性的标识
SET lock_key unique_value NX PX1000a

● unique_value is the unique identifier of the client, which can be represented by a randomly generated string.
● PX 10000 means that the lock_key will expire after 10s, so as to prevent the client from being abnormal and unable to release the lock during this period.
Lock release code implementation
Because each client uses a unique identifier in the lock operation, so when releasing the lock operation, it is necessary to judge whether the value of the lock variable is equal to the unique identifier of the client performing the lock release operation, as follows Show:

//释放锁比较unique_value是否相等
if redis.call("get",KEYS[1]==ARGV[1] then)
	return redis.call("del",keys[1])
else
	return 0
end

This is the pseudocode of the lock release operation implemented using Lua script (unlock.script), where KEYS[1] represents the lock_key, ARGV[1] is the unique identifier of the current client, both values ​​are when executing the Lua script passed in as a parameter.
Finally, execute the following command to complete the lock release operation.

redis -cil --eval unlock.script lock_key,unique_value

Use Lua script in the lock release operation —> this is because the logic of the lock release operation also includes multiple operations of reading the lock variable, judging the value, and deleting the lock variable, and when Redis executes the Lua script, it can atomically Execute in the same way, thus ensuring the atomicity of the lock release operation.

2.5 Disadvantages:
Use SET commands and Lua scripts to implement distributed locks on Redis single nodes. Use a Redis instance to save the lock variable. If the Redis instance fails and goes down, the lock variable will not have l. At this time, the client cannot perform lock operations, which will affect the normal execution of the business. When implementing distributed locks, it is also necessary to ensure the reliability of the locks

3. Realize highly reliable distributed lock based on multiple Redis nodes

When implementing a highly reliable distributed lock, you cannot rely solely on a single command operation, and you need to perform
adding and unlocking operations according to the algorithm of the distributed lock, otherwise, the lock may not work.

3.1 Distributed Lock Algorithm Redlock
Distributed Lock Algorithm Redlock Purpose:
In order to avoid the problem that the lock cannot work caused by the failure of the Redis instance, the
basic idea of ​​the Redlock algorithm is
to let the client and multiple independent Redis instances request locks in turn. If the client If the client can successfully complete the lock operation with more than half of the instances, the client successfully obtains the distributed lock, otherwise the lock fails. In this way, even if a single Redis instance fails, because the lock variable is also saved on other instances, the client can still perform lock operations normally, and the lock variable will not be lost.

Execution steps of the Redlock algorithm
The implementation of the Redlock algorithm requires N independent Redis instances. Next, it is divided into 3 steps to complete the locking operation.

1. The first step is that the client gets the current time.
2. The second step is that the client performs locking operations on N Redis instances in sequence.

The locking operation here is the same as the locking operation performed on a single instance, use the SET command, bring the NX, EX/PX options, and bring the unique identifier of the client. Of course, if a Redis instance fails, in order to ensure that the Redlock algorithm can continue to run in this case, it is necessary to set a timeout period for the locking operation.
If the client fails to request a lock with a Redis instance until the timeout expires, then the client will continue to request a lock with the next Redis instance. The timeout period of the lock operation needs to be much shorter than the effective time of the lock, which is generally set to tens of milliseconds.

The third step is that once the client completes the locking operation with all Redis instances, the client needs to calculate the total time spent on the entire locking process.
Only when the following two conditions are met, the client can consider the lock successful.
● Condition 1: The client has successfully acquired the lock from more than half (greater than or equal to N/2+1) of the Redis instances; ● Condition 2: The
total time spent by the client on acquiring the lock does not exceed the effective time of the lock.
After these two conditions are met, recalculate the effective time of the lock, and the calculated result: the initial effective time of the lock - the total time spent by the client to acquire the lock.
● If the effective time of the lock is too late to complete the operation of shared data –> release the lock to avoid the situation that the lock expires before the data operation is completed.
● Of course, if the client fails to meet these two conditions at the same time after performing lock operations with all instances, then the client initiates an operation to release locks to all Redis nodes.
In the Redlock algorithm, the operation of releasing the lock is the same as the operation of releasing the lock on a single instance, as long as the Lua script for releasing the lock is executed. In this way, as long as more than half of the N Redis instances can work normally, the normal operation of the distributed lock can be guaranteed.
Therefore, in actual business applications, the reliability of distributed locks is improved through the Redlock algorithm.

4. Summary:

A distributed lock is a variable maintained by a shared storage system. Multiple clients can send commands to the shared storage system to lock or release locks.
As a shared storage system, Redis can be used to implement distributed locks.
When implementing distributed locks based on a single Redis instance, three conditions need to be met for locking operations.

1. 1. Locking includes three operations of reading the lock variable, checking the value of the lock variable and setting the value of the lock variable, but it needs to be completed in an atomic operation. Use the SET command with the NX (set if it does not exist) option to achieve locking ;
2. 2. The lock variable needs to set an expiration time, so as to prevent the client from getting an exception after obtaining the lock, resulting in the lock being unable to be released, add the EX/PX (set expiration) option when the SET command is executed, and set its expiration time;
3. 3. The value of the lock variable needs to be able to distinguish locking operations from different clients, so as to avoid accidental release operations when releasing the lock. When using the SET command to set the value of the lock variable, the value set by each client is a unique value. Use Hand identifies the client unique_value.
   Similar to locking, releasing the lock also includes three operations: reading the lock variable value, judging the lock variable value, and deleting the lock variable, which cannot be implemented with a single command. Therefore, the Lua script is used to perform the lock release operation. The atomicity of the lock release operation is guaranteed by atomically executing the Lua script through Redis.
   However, when implementing distributed locks based on a single Redis instance, the instance will be abnormal or crashed, which will cause the instance to be unable to provide lock operations. Because of this, Redis also provides the Redlock algorithm to implement distribution based on multiple instances The lock variable is maintained by multiple instances. Even if an instance fails, the lock variable still exists, and the client can still complete the lock operation. The Redlock algorithm is an effective solution to realize highly reliable distributed locks.

Use the SET command with NX and EX/PX options to perform locking operations. Can the following methods be used to implement locking operations?

// 加锁
SETNX lock_key unique_value
EXPIRE lock_key 10S
// 业务逻辑
DO THINGS

Answer: If this method is used to implement locking, although the two commands SETNX and EXPIRE complete the atomic judgment and value setting of the lock variable and the operation of setting the expiration time of the lock variable respectively, when these two operations are executed together, Atomicity is not guaranteed. If the client fails after executing the SETNX command, but the lock variable has not been set with an expiration time, it cannot be released on the instance, which will cause other clients to be unable to perform the lock operation. Therefore, we cannot use this method for locking.