Implementing distributed locks based on Zookeeper

Preface

In distributed systems, ensuring data consistency and avoiding conflicts is a core issue, which is usually solved through distributed locks. Distributed locks are essentially a synchronization mechanism used to control access to shared resources or critical sections.

As a distributed coordination service, Zookeeper provides an effective platform for the implementation of distributed locks. This article will use a simple example to introduce how to implement distributed locks based on the interfaces and mechanisms provided by Zookeeper.

statement

The code provided in the article is for reference only. It aims to provide developers with a practical distributed lock implementation method and help readers understand how to use Zookeeper's features and mechanisms to manage locks in distributed systems. Please note that these codes are not intended for use in real applications .

prerequisite knowledge

Distributed lock design principles

Implementing a distributed lock must meet the following basic requirements:

1. Mutual exclusivity/exclusivity : Only one client is allowed to hold the lock at the same time.
2. Availability : When an exception occurs on the client, the lock can be released normally to avoid deadlock.
3. Homology : The lock cannot be released by other threads, otherwise mutual exclusion/exclusiveness will be destroyed.
4. Reentrancy : The same client can call the lock repeatedly and recursively without deadlock.

In addition, it is also necessary to consider whether the client blocks and waits before acquiring the lock, or whether it is regarded as an acquisition failure, which depends on the business scenario.

Zookeeper

Zookeeper is a traditional distributed coordination service. It is used more as a coordinator, such as coordinating and managing Hadoop clusters, coordinating Kafka leader election, etc.

Which features and mechanisms of Zookeeper can efficiently implement distributed lock requirements?

1. Temporary node : The life cycle of a temporary node depends on the session that created it. When the session ends, the temporary node will be deleted. This feature can meet the availability of distributed locks.
2. Sequential node : When creating a sequential node, Zookeeper will allocate an incrementing counter, and the top node can acquire the lock. This feature enables fair locking. (No basic developer can understand creating a node this way: creating a node to the same directory is acquiring a lock.)
3. Watcher mechanism : Through the Watcher mechanism, the current node can monitor the changes of the previous node. When the previous node is deleted, the current node can know that the lock is released, thereby acquiring the lock.
4. Node data : Set the client session unique identifier as a value when creating a node to achieve reentrancy.

Mutual exclusivity/exclusivity and homology need to be controlled by the client, which will be explained in the code example.

Distributed lock implementation

Create a Maven project, import the zkclient dependency and start coding

<dependency>
    <groupId>com.101tec</groupId>
    <artifactId>zkclient</artifactId>
    <version>0.10</version>
</dependency>

The following example code meets the basic requirements of distributed locks and is a blocking distributed lock, that is, the client blocks and waits until the lock is obtained.

public class DistributedLock {
    
    

    private ZooKeeper client;

    // 连接信息
    private String connectString = "127.0.0.1:2181";

    // 超时时间
    private int sessionTimeOut = 30000;

    // 等待zk连接成功
    private CountDownLatch countDownLatch = new CountDownLatch(1);

    // 等待节点变化
    private CountDownLatch waitLatch = new CountDownLatch(1);

    //当前节点
    private String currentNode;

    //前一个节点路径
    private String waitPath;

    private final String ROOT_PATH = "/locks";

    //1. 在构造方法中获取连接
    public DistributedLock() throws Exception {
    
    
        client = new ZooKeeper(connectString, sessionTimeOut, watchedEvent -> {
    
    
            //  连上ZK，可以释放
            if (watchedEvent.getState() == Watcher.Event.KeeperState.SyncConnected) {
    
    
                countDownLatch.countDown();
            }

            //waitLatch 需要释放 (节点被删除并且删除的是前一个节点)
            if (watchedEvent.getType() == Watcher.Event.EventType.NodeDeleted && watchedEvent.getPath().equals(waitPath)) {
    
    
                waitLatch.countDown();
            }
        });

        //等待Zookeeper连接成功，连接完成继续往下走
        countDownLatch.await();
        //2. 判断节点是否存在
        Stat stat = client.exists(ROOT_PATH, false);
        if (stat == null) {
    
    
            //创建一下根节点
            client.create(ROOT_PATH, ROOT_PATH.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);


        }

    }

    //3.对ZK加锁
    public boolean zkLock() {
    
    

        try {
    
    
            String sessionId = String.valueOf(client.getSessionId());
            List<String> children = client.getChildren(ROOT_PATH, false);
            if (!children.isEmpty()) {
    
    
                Collections.sort(children);
                String path = children.get(0);
                byte[] data = client.getData(ROOT_PATH + "/" + path, false, null);
                //最小序号节点是当前客户端创建的不用再次获取
                if (sessionId.equals(new String(data))) {
    
    
                    System.out.println("重入锁");
                    return true;
                }
            }

            //创建 临时带序号节点，将当前客户端id作为值，实现可重入
            currentNode = client.create(ROOT_PATH + "/seq-", sessionId.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL);
            children = client.getChildren(ROOT_PATH, false);
            //如果创建的节点只有一个值，就直接获取到锁，如果不是，监听它前一个节点
            if (children.size() == 1) {
    
    
                return false;
            } else {
    
    
                //先排序
                Collections.sort(children);

                //获取节点名称
                String nodeName = currentNode.substring((ROOT_PATH + "/").length());

                //通过名称获取该节点在集合的位置
                int index = children.indexOf(nodeName);

                if (index == -1) {
    
    
                    System.out.println("数据异常,nodeName:" + nodeName);
                    return false;
                } else if (index == 0) {
    
    
                    //创建的节点是否是最小序号节点，如果是 就获取到锁；如果不是就监听前一个节点
                    return true;
                } else {
    
    
                    //需要监听前一个节点变化
                    waitPath = ROOT_PATH + "/" + children.get(index - 1);
                    client.getData(waitPath, true, null);

                    //等待监听执行
                    waitLatch.await();
                    return true;
                }
            }

        } catch (KeeperException e) {
    
    
            e.printStackTrace();
        } catch (InterruptedException e) {
    
    
            e.printStackTrace();
        }
        return false;
    }

    public void unZkLock() throws KeeperException, InterruptedException {
    
    
        //删除节点
        client.delete(currentNode, -1);
    }


}

In the example, a root node named is created /locksas the identifier of the lock. It is called when the client needs to acquire the lock zkLock(). This method will first determine whether the current client already holds the lock. If it does, no node will be created (here is the implementation that can Reentrancy), otherwise /locksa temporary sequence node will be created at the root node. This is the case when multiple clients acquire the lock node directory at the same time.

├── locks
│   └── seq-0000000006
│   └── seq-0000000005
│   └── seq-0000000004
│   └── seq-0000000003
│   └── seq-0000000002
│   └── seq-0000000001

If the node created by the client is the smallest node, the lock processing service is successfully obtained. Otherwise, the previous node is monitored and blocked and waited. When the previous node is deleted, the client is notified to acquire the lock.

When the client calls, unZkLock()delete the node it created to release the lock. Because the node it created is deleted, homology is naturally satisfied.

The entire process and interaction is as shown below

What needs to be noted here is that when a node changes, Zookeeper will notify the client one by one in the order of the nodes. Therefore, if the graph is seq-0000000002deleted first due to a fault, /seq-0000000003it will also need to wait for /seq-0000000001the deletion before receiving seq-0000000002the deletion notification, so only Just listen to the previous node being deleted.

After the above code is written, you can call it directly wherever you need to use distributed locks. The code is as follows:

public static void main(String[] args) {
    
    
    try {
    
    
        DistributedLock lock = new DistributedLock();
        if (lock.zkLock()) {
    
    
            System.out.println(Thread.currentThread() + "获取到锁");
            Thread.sleep(20 * 1000);
            lock.unZkLock();
            System.out.println(Thread.currentThread() + "释放锁");
        }

    } catch (InterruptedException | KeeperException e) {
    
    
        e.printStackTrace();
    } catch (Exception e) {
    
    
        e.printStackTrace();
    }
}

Curator framework implements distributed locks

Curator is a client framework encapsulated based on Zookeeper's native API interface. It solves the development problems of the underlying details and provides a set of high-level APIs to implement functions such as distributed lock services, cluster leader election, shared counters, caching mechanisms, distributed queues, etc. Various application scenarios.

Using Curator to implement distributed locks can greatly simplify code writing. You only need to introduce relevant dependencies and directly call the encapsulated interface. The principle is similar to the distributed lock implementation described above. code show as below:

Curator related dependencies

<dependency>
    <groupId>org.apache.curator</groupId>
    <artifactId>curator-framework</artifactId>
    <version>4.0.0</version>
</dependency>
<dependency>
    <groupId>org.apache.curator</groupId>
    <artifactId>curator-recipes</artifactId>
    <version>4.0.0</version>
</dependency>

public static void main(String[] args) {
    
    
        CuratorFramework client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", (i, l, retrySleeper) -> false);
        client.start();
        InterProcessMutex lock = new InterProcessMutex(client, "/locks");
        try {
    
    
            // 获取互斥锁
            lock.acquire();

            // 执行需要互斥访问的代码
            // 释放互斥锁
            lock.release();
        } catch (Exception e) {
    
    
            e.printStackTrace();
        } finally {
    
    
            // 关闭Curator Framework客户端
            client.close();
        }
}

InterProcessMutexIt is an implementation of distributed lock provided by Curator. It InterProcessMutexcan ensure mutually exclusive access to shared resources between multiple processes, thereby avoiding data conflicts and concurrency problems.

Summarize

The features of Zookeeper provide great help in implementing distributed locks, and its high availability and strong consistency make distributed locks more reliable and efficient. This article provides two distributed solutions based on Zookeeper. Whether using Zookeeper API or Curator API, the principles are the same. Therefore, we can directly use these mature frameworks based on understanding their principles.