In distributed services, if each service node needs to compete for resources, thread locks cannot be used like single-machine multi-threaded applications, and a distributed lock mechanism is required to ensure node access to resources. Usually distributed locks are implemented as separate services. There are three commonly used distributed lock implementations: zookeeper implementation, redis implementation and memcache implementation. The latter two are essentially the same.
A typical scenario that requires distributed locks is that each node of a distributed service is registered with the server for service discovery, and the registered nodes need to be in order. At this time, a lock is required to ensure that each node is registered in sequence.
The principles of the three implementations are described below.
1. Zookeeper
1. Implementation principle Distributed locks are implemented
based on zookeeper temporary sequential nodes. The general idea is:
(a) When each client locks a function, the directory of the designated node corresponding to the function on zookeeper Next, generate a unique temporary sequence node;
(b) The smallest sequence number among all temporary sequence nodes is the node that currently holds the lock;
(c) When releasing the lock, delete the temporary sequence node registered by itself.
The above implementation method can monitor the changes of the registered node directory on the client, and notify each client to check whether it holds the lock when the node is deleted.
If the scale of the cluster is large, this may cause a herd effect. At this time, it can be optimized for the client to monitor the changes of nodes smaller than its own node in the directory of registered nodes. When all nodes smaller than yourself are deleted, then you hold the lock yourself.
A typical scenario that requires distributed locks is that each node of a distributed service is registered with the server for service discovery, and the registered nodes need to be in order. At this time, a lock is required to ensure that each node is registered in sequence.
The principles of the three implementations are described below.
1. Zookeeper
1. Implementation principle Distributed locks are implemented
based on zookeeper temporary sequential nodes. The general idea is:
(a) When each client locks a function, the directory of the designated node corresponding to the function on zookeeper Next, generate a unique temporary sequence node;
(b) The smallest sequence number among all temporary sequence nodes is the node that currently holds the lock;
(c) When releasing the lock, delete the temporary sequence node registered by itself.
The above implementation method can monitor the changes of the registered node directory on the client, and notify each client to check whether it holds the lock when the node is deleted.
If the scale of the cluster is large, this may cause a herd effect. At this time, it can be optimized for the client to monitor the changes of nodes smaller than its own node in the directory of registered nodes. When all nodes smaller than yourself are deleted, then you hold the lock yourself.
--See https://blog.csdn.net/xiaoliuliu2050/article/details/51226237
2. Advantages High
lock security, zookeeper data is not easy to lose
3. Disadvantages High
performance overhead. Because it needs to dynamically generate and delete temporary sequence nodes, it also needs to monitor node changes to realize the lock function.
4. Open source implementation
The curator open source library provides the implementation of distributed locks, and the python library Kazoo can also directly call the acquire method through lock.Lock in kazoo.recipe.
Menagerie implements a distributed version of the java.util.concurrent package based on Zookeeper.
2. Redis distributed lock
1. Implementation principle
Use the set command in redis to realize distributed lock.
Since Redis 2.6.12, set can take the following parameters:
EX second : Set the expiration time of the key to second seconds. SET key value EX second has the same effect as SETEX key second value .
PX millisecond : Set the key's expiration time to millisecond milliseconds. SET key value PX millisecond has the same effect as PSETEX key millisecond value .
NX : Set the key only when the key does not exist. SET key value NX has the same effect as SETNX key value .
XX : Set the key only when the key already exists.
Return value:
SET returns OK only when the set operation is successfully completed.
If NX or XX is set, but the set operation is not performed because the condition is not met, the command returns a NULL Bulk Reply.
Command:
> SET key value EX ttl NX
| SET KEY VALUE [EX seconds] [PX milliseconds] [NX|XX] |
PX millisecond : Set the key's expiration time to millisecond milliseconds. SET key value PX millisecond has the same effect as PSETEX key millisecond value .
NX : Set the key only when the key does not exist. SET key value NX has the same effect as SETNX key value .
XX : Set the key only when the key already exists.
Return value:
SET returns OK only when the set operation is successfully completed.
If NX or XX is set, but the set operation is not performed because the condition is not met, the command returns a NULL Bulk Reply.
Command:
> SET key value EX ttl NX
The general idea is:
(a) SET lock currentTime+expireTime EX 600 NX, use set to set the lock value, and set the expiration time to 600 seconds, if successful, acquire the lock;
(b) After the lock is acquired, if the node goes offline, the ock value will automatically become invalid at the expiration time;
(c) When the lock is released, use del to delete the lock key value;
if the redis single machine is used for distributed lock service, there may be a single Some problems lead to poor service availability. Therefore, in the case of high service stability requirements, the official recommendation is to use redis clusters (for example, 5 sets, if the lock is successfully requested for more than 3 sets, it is considered to acquire the lock) to realize the redis distributed lock. See RedLock for details.
2. Advantages High
performance , redis can be persistent, and can also ensure that data is not easily lost;
redis clustering improves stability.
3. Disadvantages Some data may be lost when
using redis master-slave switch.
4. Open source implementation
An open source implementation of the python version: python-redis-lock.
3. Memcached distributed lock
1. Implementation principle
Use the add function of memcached to realize distributed lock.
The difference between add and set is that if multiple threads set concurrently, each set will succeed, but the last stored value is subject to the thread of the last set. The opposite is true for add. Add will add the first value that arrives and return true, and subsequent additions will return false. Distributed locks can be easily implemented using this point.
2. Advantages
Because it is full-memory storage, concurrency is efficient.
3. Disadvantages
(1) memcached adopts the LRU replacement strategy, so if the memory is not enough, the lock information in the cache may be lost.
(2) Memcached cannot be persistent, and once restarted, information will be lost.
4. Open source implementation
pending.
(c) When the lock is released, use del to delete the lock key value;
if the redis single machine is used for distributed lock service, there may be a single Some problems lead to poor service availability. Therefore, in the case of high service stability requirements, the official recommendation is to use redis clusters (for example, 5 sets, if the lock is successfully requested for more than 3 sets, it is considered to acquire the lock) to realize the redis distributed lock. See RedLock for details.
2. Advantages High
performance , redis can be persistent, and can also ensure that data is not easily lost;
redis clustering improves stability.
3. Disadvantages Some data may be lost when
using redis master-slave switch.
4. Open source implementation
An open source implementation of the python version: python-redis-lock.
3. Memcached distributed lock
1. Implementation principle
Use the add function of memcached to realize distributed lock.
The difference between add and set is that if multiple threads set concurrently, each set will succeed, but the last stored value is subject to the thread of the last set. The opposite is true for add. Add will add the first value that arrives and return true, and subsequent additions will return false. Distributed locks can be easily implemented using this point.
2. Advantages
Because it is full-memory storage, concurrency is efficient.
3. Disadvantages
(1) memcached adopts the LRU replacement strategy, so if the memory is not enough, the lock information in the cache may be lost.
(2) Memcached cannot be persistent, and once restarted, information will be lost.
4. Open source implementation
pending.
Refer to
http://surlymo.iteye.com/blog/2082684
http://www.cnblogs.com/moonandstar08/p/5705619.html
zookeeper implementation
http://graduter.iteye.com/blog/2024190
http://surlymo.iteye.com/blog/2082684
http://www.cnblogs.com/moonandstar08/p/5705619.html
zookeeper implementation
http://graduter.iteye.com/blog/2024190
http://surlymo.iteye.com/blog/2082684
http://blog.csdn.net/xiaoliuliu2050/article/details/51226237
Redis implementation
https://redis.io/topics/distlock
http://blog. csdn.net/daiyudong2020/article/details/51760648
http://www.weizijun.cn/2016/03/17/Talk about the design of distributed locks/
http://blog.csdn.net/xiaoliuliu2050/article/details/51226237
Redis implementation
https://redis.io/topics/distlock
http://blog. csdn.net/daiyudong2020/article/details/51760648
http://www.weizijun.cn/2016/03/17/Talk about the design of distributed locks/
http://blog.csdn.net/u013970991/article/details/52722680