[Distributed] What are distributed lock implementations?

First, the business scene

Jvm, with a variable number of threads operating with a state guarantee can be locked by a thread-safe within the JVM.

If multiple JVM is operating with a state variable, how to ensure thread safe?

This time you need a distributed lock to play its role in the

Second, characteristics

Distributed systems are often traffic is relatively large, high concurrency, have high requirements for high availability and high-performance distributed lock. Generally distributed lock program needs to meet the following requirements:

• There are highly available acquiring the lock and release the lock function
• Get better performance lock and release the lock
• This lock If a reentrant lock (to avoid deadlock)
• This lock is a best blocking locks (whether or not this business needs to consider)
• It is best to lock a fair lock (whether or not this business needs to consider)

Third, based on a distributed database lock program

1, based on the primary key table only way to do distributed lock

Using the key unique characteristics of the main, if there are multiple requests simultaneously submitted to the database, the database will ensure that only one insertion operation can be successful, then we can assume that thread the success of the operation to obtain the lock of the method, when the method is performed after completion, wants to release the lock, then delete this database to record

1.1, shortcomings

• Database single point
• No lock timeout mechanism
• Non-reentrant
• Unfair lock
• Non-blocking lock

1.2, optimization points

• Master database from a backup, a single point of solving the problem. Because the master-slave synchronization with a delay, may lead to data inconsistency
• The lock timeout detection timing of the task, or by automatically released connection.commit()to release the lock operation
• Locking plus machine and thread information, first check before locking, support for reentrant
• Middle of the table, record locking thread machine failure, sorted by creation time
• Spin achieve the effect of blocking

1.3 Principle

General database use innodb storage engine, when inserting data will add row-level locking. So as to achieve the effect of concurrent requests are executed sequentially

2, to achieve optimistic locking through a database mvcc

Time to update the data to bring specify the version number, the update in advance if the other thread version number, the update fails

2.1, shortcomings

Intrusion of large database tables, each table need to increase the version field

The presence of many highly concurrent update failed

3, limiting the database

• Use exclusive locks to be distributed lock the lock, then an exclusive lock for a long time not to submit, will take up the database connection. Once the connection has become more similar, the database connection pool can be able to explode.
• Database is written to disk io, performance worse
• Database can support a maximum qps also limited, difficult to meet the needs of high concurrency

Fourth, based on the implementation of distributed lock redis

1, the principle

1.1, lock

Atomic Command: SET key value NX PX milliseconds

PX milliseconds expiration time, to prevent the locking thread to die can not be unlocked. Expiration time is too short, the thread may lock logic has not been executed properly, it is time to expiration

NX Without this key is set, there is a failure to return key

value random value (generally UUID), can only be unlocked by the lock used to implement the thread

1.2, Unlock

lua script to achieve get value, delete operations. value locked when the set is not going to repeat random value, unlock to unlock the same time must UUID

2, shortcomings

• Acquire a lock is non-blocking
• Unfair lock, does not support the scenario requires fair locks
• redis there is a delay from the master, the master is switched from the master down occurs, it may cause failure of the lock

Fifth, the implementation of distributed locking algorithm based Redlock. redisson Redlock algorithm for encapsulation

<dependency>
    <groupId>org.redisson</groupId>
    <artifactId>redisson</artifactId>
    <version>3.3.2</version>
</dependency>

1, the principle

In a distributed environment, Redis, we assume that there are N Redis master. These nodes are completely independent of each other, master-slave replication or other cluster coordination mechanisms do not exist . We will use the same method to ensure that acquiring and releasing locks at a single instance in Redis N instances. Now we assume that there are five Redis master node, and we need to run these Redis instances in the top five servers, thus ensuring that they would not shoot down all at the same time.

1.1, lock

Suppose cluster-1, cluster-2, cluster-3 in total three clusters cluster mode. If you want to get a distributed lock, you need to perform LUA script by EVAL command to the three cluster cluster, you need 3/2 + 1 = 2, that is, a successful response at least two cluster clusters. to set a unique value, redisson value by the UUID threadId ensure uniqueness value of +

1. Get the current time (in milliseconds).

Much smaller when compared with the same key 2. turn and requests a lock on the random value N nodes, in this step, the client requests a lock on each master, and have a total release of the lock timeout time. For example, if the automatic lock release time is 10 seconds, that each node lock request timeout may be in the range of 5-50 ms, this can prevent a client blocking too long on one master node shoot down, if a master node is not available, we should try next master node as soon as possible.

3. The client computing second step it takes to obtain a lock, only when the client on most master node successfully acquired lock (in this case three), and total time spent not more than lock release time, this that it is the lock acquisition a success.

4. If the lock acquisition is successful, it is now time to lock automatically releases the lock release is the first to obtain a lock before time minus consumed.

5. If the lock acquisition fails, either because of the successful acquisition of the lock is not more than half (N / 2 + 1) or more than the total elapsed time since the lock release time, the client will release the lock on each master node, even he believes that those who did not succeed lock.

1.2, release the lock

Require all nodes on the line in the lock is released, regardless of the success has not previously acquired the lock at the node.

If the client does not get a majority node to lock in, be sure to get a lock on the successful release of node lock as soon as possible, so no need to wait until after the key timeout to re-acquire the lock

2, safety demonstration

 Before we begin, let us assume that the client may have to get a lock on most of the nodes, the nodes will contain all of this with the same survival time of a key. However, note that this key is set up at different points in time, these key will time out at different times, but we assume the worst case, the first time a key is set at T1 (client connects to the first a time when the server), the last time a key is set at T2 (the client receives the server returns the result of the last time), starting from the time T2, the earliest time we can confirm that there will at least timeout key is MIN_VALIDITY = TTL- (T2-T1) -CLOCK_DRIFT, TTL is the lock timeout, (T2-T1) is the latest time-consuming acquired lock, CLOCK_DRIFT clock difference among processes, this is used to compensate for the earlier of ( T2-T1). Other key will take time out after this point in time, so we can identify these key at least until this point in time are simultaneously present.

If a client gets the most time-consuming node lock locks close to or above the maximum lifetime (TTL value is set for our SET operation), then the system will think that this lock is invalid and it will release the lock on these nodes , so we only need to consider the circumstances to obtain the majority node lock consuming less than the effective time. In this case, according to our previous proof, in MIN_VALIDITY time, the client can not reacquire lock success, so multiple clients can get results while successfully lock will only occur in the majority of nodes to obtain the lock time in the case are much higher than the TTL time, in fact, these locks will fail in this case

Sixth, based zookeeper achieve Distributed Lock

1, the basic exclusive lock (lock unfair)

1.1 Principle

Use of temporary nodes and watch mechanisms. Each lock takes a common node / lock, when you need to create a temporary node to acquire the lock in / lock directory, create successful it means to acquire the lock success, failure, then watch / lock node, and then there is a delete operation to fight the lock. When the benefit is temporary node process hang up automatically locked node lock is canceled automatically deleted.

1.2, shortcomings

All locks fail to take the process listens parent, it is prone to herd behavior, that is, when all wait for the process to release the lock together to create a large node, concurrency.

2, optimized exclusive lock (lock fair)

2.1 Principle

Instead, create a temporary lock orderly nodes, each node locked can create success, but different serial numbers. Only the minimum number can have a lock, if the node number is not a minimum number of the watch itself is smaller than the previous node (fair locks).

3, shared locks

3.1 Principle

Create a temporary order of nodes in the node lock. Read node number to R +, W + number of write node. After creating the node, get all the child nodes, child nodes of registration changes watcher node lock monitoring, determine the location of their number in all sub-node. For a read request, no less than their numbers written node, it means access to a shared lock, a read logic. For a write request, if he is not a minimum number of child nodes, you need to enter the wait. Watcher after receiving the notification, acquires the lock is repeated.

3.2, shortcomings

Shared lock herding. A large number of watcher notification and obtain a list of child nodes, the two operations run repeatedly. Under cluster size is relatively large, it will cause great zookeeper server and network performance impact shocks

3.3 Optimization

Read request, the listener is smaller than write their own node. Write requests, monitor smaller than its last node.

4, zookeeper limitations

• Performance caching service may not be so high, because every time during the creation and release locks in, to be dynamically created and destroyed temporary node to achieve lock.
• ZK create and delete nodes can only be performed by Leader server, then the data is synchronized to all Follower machines.
• Concurrency support is better redis