Dubbo + Zookeeper (a) the Zookeeper acquaintance

Front took a while to learn the knowledge SpringCloud, mainly to understand the concept of micro-services and using a series of micro components SpringCloud implementation service landing. Learning these components themselves are simple, follow the basic operation will be over, and it also benefited from Springboot brought us a lot of convenience. Practical applications might also run into some pit, but as long as we grasp the basic principles can be resolved.

Also spoke in front of the micro-service solutions include two, one is SpringCloud, the other one is Dubbo + Zookeeper, let's learn Dubbo + Zookeeper implement micro-services.

In the first study micro service, understand the concept of micro-services, compared the advantages and disadvantages of prior service and micro monomer applications, micro spoke to achieve major technical service point, two of the most important services that the communication between the management and service.

Dubbo + Zookeeper to implement micro-services, we must solve these two technical points, Dubbo is an RPC communication framework that communication between the service can be achieved. ZooKeeper is a distributed coordination service for managing mainframe. In a distributed environment, coordination and management services is a complex process.

First, the distributed coordination technology

It says Zookeeper is a distributed coordination technology, we have to first learn what is distributed coordination technology. Distributed coordination technology is mainly used to solve the synchronization control between the distributed environment among multiple processes, allowing them to access some kind of orderly critical resources, prevent the consequences of "dirty data".

First, we must understand why we need a distributed lock, a simple example, there are some regular tasks on the general system, such as doing some clearing data, if we deploy multiple servers, each server that will be executed at this time this regular tasks, if not a lock mechanism, in accordance with that principle, the timing of this task will be executed multiple times, so it is very likely dirty data.

This refers to a timed task, since it is timed task, then it is bound to enter the same time, this does not require high concurrency, we also need to consider. Another is high concurrency, in a distributed environment, a great probability there will be a process on multiple nodes simultaneously access a particular method, but many times, we need a method to ensure that at the same time can only be the same a thread of execution , in a standalone environment, Java in fact, provides a lot of concurrent processing related API, but the API in a distributed scenario can not do anything. That simply does not provide the ability to Java Api distributed lock. So for the implementation of distributed lock There are a variety of programs.

We need to be distributed lock is kind of how? (This method to lock, for example, resource lock empathy)

Application Clusters can be guaranteed in distributed deployment, the same method can only be executed on a thread of a machine at the same time.
This lock If a reentrant lock (to avoid deadlock)
This lock is a best blocking locks (whether or not this business needs to consider)
There are highly available acquiring the lock and release the lock function
Get better performance lock and release the lock

Implementation of Distributed Lock There are several options:

Memcached: Memcached use the addcommand. This command is an atomic operation, and only in keythe absence of in order to addsucceed, it means that the thread has been locked.
Redis: Memcached and a way similar to the use Redis setnxcommands. This command also is an atomic operation, and only in keythe absence of in order to setsucceed.
Zookeeper : Zookeeper node using the temporary order to achieve lock and distributed queue. Zookeeper originally designed, is to achieve a distributed lock service.
Chubby: Google the company to achieve coarse-grained distributed lock service, the underlying use of the Paxos consensus algorithm.

Second, the realization of the basic principles of distributed lock

To achieve lock, that there are three basic functions: lock, unlock, lock timeout , we realize redis way to simply introduce the basic principles of distributed lock, the following codes are pseudo-code.

2.1 Locking

redis achieve locking operation is the easiest is to use the setnxcommand, where the key can be named according to the business name, the basic model is the namespace + corresponding parameters, such as we want to lock a certain commodity inventory, we can use the inventory id It is locked as an argument, in fact, where we can add the same lock in different ways, for example, there are two methods require processing of inventory, although they are not a method, but we use the same namespace and parameters It can also be locked.

setnx (lock_sale_ commodity ID, 1)

When we conduct set method, if the key already exists, indicating that other threads have been locked, grab the lock fails, you need to wait, if the key does not exist, it means to get a lock, the method will work correctly.

2.2 Unlock

Since there are locking procedure, it will have to unlock the process, the thread executing the task, we need to release the lock, so that other threads can enter. The easiest way is to perform the release of lock delinstruction. Once the delete key, then other threads can normally get locked.

del (lock_sale_ Goods ID)

2.3 lock out

Lock timeout What does it mean? If a thread get locked during the execution of the task hang up, too late to explicitly release the lock, this resource will always be locked ( deadlock ), other threads can no longer even think about coming. Therefore, setnxthe keyneed to set a time-out to ensure that even if not explicitly released, the lock should be released automatically after a certain time. setnxIt does not support timeout parameter, so the need for additional instruction.

expire (lock_sale_ commodity ID, 30)

That is the whole pseudo-code:

if（setnx（lock_sale_商品ID，1） == 1）{
    expire（lock_sale_商品ID，30）
    try {
        do something ......
    } finally {
        del（lock_sale_商品ID）
    }
}

2.4 Problems

redis implement basic distributed lock is in accordance with this idea, but there are some problems here.

setnxAnd expirenon-atomic

If we use the java command to control the timeout, set the timeout lock and not an atomic operation, and that if the process of locking hung up, this time it will not execute the code sets the timeout, then this lock will never be solution, this is clearly not acceptable.

Solution: Use the set command, directly into the overtime period , then this problem is to redis overtime to resolve itself, so even if there is a problem java process, redis itself can be unlocked.

set (lock_sale_ commodity ID, 1,30, NX)

Mistakenly deleted

A successful one thread has been locked, and a timeout period of 30 seconds, but due to the implementation of this method itself is relatively slow, not more than 30 seconds to complete execution, automatic lock release expired, then thread B to give the lock, then thread A execution finished, it performs the task to unlock, delete the lock, but this time we know the solution is not lock their own, but the lock thread B, which is mistakenly deleted the thread B lock.

Solution: You can put the current thread ID as when locked value, and verify before deleting keythe corresponding valueis not their own thread ID.

That lock code:

String threadId = Thread.currentThread().getId()
set（key，threadId ，30，NX）

Unlock code:

if（threadId .equals(redisClient.get(key))）{
    del(key)
}

Possibility of concurrent

In the above description, if a method execution time is too long, complicated by the possibility that it is possible there will be, but if the expiration time is set too long there will wait for some time he acquires the lock thread may be for no reason.

Understand finished redis achieve distributed lock, we have an emotional understanding of distributed lock now to learn Zookeeper

Three, Zookeeper acquaintance

Apache ZooKeeper is a service used by the cluster (group of nodes) for coordination between themselves and to maintain share data through a robust synchronization technology. ZooKeeper itself is a distributed application that provides services to write distributed applications.

ZooKeeper common services provided are as follows:

Naming Service - by name identifies the nodes in the cluster. It is similar to DNS, but only for the node.
Configuration management - adding nodes recent and latest system configuration information.
Cluster Management - in real-time joining / leaving nodes in the cluster and node status.
Election algorithm - a node election as leader coordination purposes.
Locking and synchronization service - lock data while modifying data. This mechanism can help you automatically failover when connected to other distributed applications (such as Apache HBase).
Registry highly reliable data - data can be obtained even when turned off in one or several nodes.

Distributed applications provide many benefits, but they also throw some complex and difficult to solve challenges. ZooKeeper framework provides a complete mechanism to overcome all challenges. Race conditions and deadlocks using fail-safe synchronization method for processing. Another major disadvantage is data inconsistency, ZooKeeper using atomic resolution.

The following are the benefits of using ZooKeeper:

Simple distributed coordination process
Synchronization - mutually exclusive and collaboration between the server process. This process helps Apache HBase configuration management.
Ordered news
Serialization - encoding the data according to specific rules. Ensure consistent application runs. This method can be used to coordinate the execution queue in a thread of the MapReduce.
reliability
Atomic - Data transfer complete success or a complete failure, but no transaction is part of.

3.1 Zookeeper overall architecture

Let's understand the overall architecture of Zookeeper, look at the following chart:

Client (client): The client, a node in the cluster our distributed applications, and access information from the server. For a particular time interval, each client sends a message to the server so that the server knows the client is active. Similarly, when the client is connected, the server sends confirmation code. If the connected server does not respond, clients will automatically redirect the message to another server.
Server (server): Server, a node ZooKeeper our population, and provides the client with all the services. Sends a confirmation code to tell the server to the client is active.
Ensemble: ZooKeeper server group. Minimum number of nodes is 3 to form the desired ensemble.
Leader: server nodes, if any connected node fails, the automatic recovery. Leader is elected when the service starts.
Follower: Follow the leader instruction server node.

3.2 Zookeeper data model

Zookeeper data model is like? It is much like the data among the trees, but also very much like a file system directory.

ZooKeeper node called znode . Each znode identified by a name, and (/) sequences separated by a path, this hierarchical structure, so that each node has a unique path Znode, just make the same namespace clear separation of different information.

Znode that there are elements which contain it?

data: data storage Znode.
ACL: access Znode record, that is who or what can access this IP node.
stat: Znode containing various metadata, such as transaction ID, version number, timestamp, size, etc. The total amount of data stored in the data length is znode. You can store up to 1MB of data.
child: a reference to the current node's children

Znode type

Znode is divided lasting (persistent) node, the sequence (Sequential) node and the temporary (ephemeral) node.

Persistent node - even after the creation of this particular znode client disconnects, lasting node still exists. By default, unless otherwise indicated, all znode are persistent.
Temporary node - client active, temporarily node is valid. When the client is disconnected with ZooKeeper collection, temporary node will be automatically deleted. Therefore, only temporary node does not allow children. If the temporary node is deleted, the next appropriate node to its filling position. Interim leader node plays an important role in the election.
Order node - the node order can be permanent or temporary. Znode When a new node is created as a sequence, ZooKeeper by 10-bit serial number appended to the original name set znode path. For example, if a path having / myapp created znode a sequential node, then the path will change ZooKeeper / myapp0000000001 , and the next sequence number to 0,000,000,002. If two sequential nodes are created at the same time, it will not be the same for each ZooKeeper znode using digital. In order node locking and synchronization play an important role.

Znode support operations and exposed API:

create / path data: create a / path of znode named data is data.
delete / path: delete the name / path of znode.
exists / path: Check for znode the name / path of
setData / path data: Set the name / path of the data for the data znode
getData / path: Returns the name / path of the data znode
getChildren / path: Returns all / path nodes node list of all children

3.3 Watches

Monitoring is a simple mechanism that allows the client to receive notification of ZooKeeper set to change. The client may be provided at the time of reading a particular Watches znode. Watches will send any znode (client registry) to registered clients notified of the change.

Znode change is a child or znode modify data associated with znode the changes. Only trigger once watches. If the client wants to notice Again, it must be done by another read operation. When the connection session expires, the client will be disconnected from the server, related watches will also be deleted.

Specific interactions are as follows:

The client calls the getDatamethod watchparameters true. The server receives the request and returns the data node, and inserted into the Watch Znode path, and Watcher list corresponding to the hash table.

When the Watch of Znode has been removed, the server looks up the hash table to find all Watcher, asynchronous notification of the Znode corresponding client, and delete the hash table corresponding Key-Value.

Here is somewhat similar to the observer pattern, think about where we can take advantage of the implementation of the registration and discovery services, later we will be in-depth study.

The basic concept is that these Zookeeper, next to its basic principle further, and learn how to achieve consistency Zookeeper Zookeeper distributed lock is achieved.