Preface
In fact, regardless of whether redis or Mysql is a data storage medium, there is a common problem in distributed scenarios: service routing in cluster scenarios . For example: in the redis cluster scenario, we originally deployed 3 masters and 3 slaves. But in the event that there is a surge in visits or one of the machines is down one day, then the service routing (usually using HASH modulo positioning) will face the data not being found in the new node after recalculation, so it will be recalculated. Will use the DB to query data into the cache, if it is a scene with a lot of traffic, it will cause a lot of pressure on the database. If there is an algorithm, regardless of the expansion or shrinkage problems, the final impact is small enough, that is, only part of the data may need to be reset to the DB, and the other can correctly find the corresponding cache machine node, then this is the best. The following article is excerpted from an original article by Zhihushang Ali engineers. It is still a good idea for the author to use graphic ideas, relatively easy-to-understand text, and clarify the consistent HASH algorithm; I recommend you to read...
Let's analyze step by step from a historical perspective, and explore what exactly is a Hash consensus algorithm!
One, the use of Redis cluster
When we use Redis, in order to ensure the high availability of Redis and improve the read and write performance of Redis, the easiest way is to do master-slave replication to form a Master-Master or Master-Slave, or to build a Redis cluster for data processing Read-write separation is similar to database master-slave replication and read-write separation. As follows:
It is also similar to a database. When the single table data is larger than 500W, it needs to be divided into database and table. When the amount of data is large (the standard may be different, depending on the Redis server capacity), we can also perform similar operations on Redis. , Is the sub-database and sub-table.
Suppose, we have a social networking site that needs to use Redis to store image resources. The storage format is a key-value pair, the key value is the image name, and the value is the path of the file server where the image is located. We need to find the file server where the file is located based on the file name. The above path, the data volume is about 2000W, according to the rules we agreed to divide the database, the rule is random allocation, we can deploy 8 cache servers, each server contains about 500W pieces of data, and master-slave replication, the schematic diagram is as follows :
Since the rules are random, all of our data may be stored in any group of Redis. For example, in the above picture, our user looks for a picture named "a.png". Since the rules are random, we are not sure about the specifics. It is on which Redis server, so we need to perform 1, 2, 3, 4, 4 queries to be able to query (that is, traverse all Redis servers), which is obviously not the result we want, understand Friends who have passed may think that random rules are not good. You can use the rules of sub-database and table in the database: according to the Hash value, taking the modulus, according to the category, according to the value of a certain field, etc. common rules can come out. ! Well, according to our theme, we will use the Hash method.
Two, use Hash for Redis cluster
It is conceivable that if we use the Hash method, each picture can be located to a specific server when it is partitioned. The schematic diagram is as follows:
In the above figure, suppose we are looking for "a.png". Since there are 4 servers (excluding from the library), the formula is hash(a.png) % 4 = 2 , we can see that the second server is located, so that it will not traverse all the servers, which greatly improves Performance!
Third, the problem of using Hash
Although the above method improves performance, we no longer need to traverse the entire Redis server! However, when the above Hash algorithm is used for caching, there will be some defects, mainly reflected in the changes in the number of servers, all cache locations must change!
Imagine if the 4 cache servers can no longer meet our caching needs, then what should we do? It's very simple, just add a few more cache servers!
Assumption: We add a cache server, then the number of cache servers has changed from 4 to 5. Then the original hash(a.png) % 4 = 2 formula becomes hash(a.png) % 5 = ？ ,
It is conceivable that this result is definitely not 2. The result of this situation is that when the number of servers changes, all cache locations must change! In other words, when the number of servers changes, all caches are invalid for a certain period of time. When the application cannot get data from the cache, it will request data from the backend database (remember the " Cache Avalanche " in the previous article. ?)!
Similarly, suppose one of the 4 caches suddenly fails and cannot be cached, then we need to remove the faulty machine, but if one cache server is removed, the number of cache servers will change from 4 to 4 With 3 units, the above-mentioned problems will also occur!
Therefore, we should find a way to prevent this from happening, but due to the above Hash algorithm itself, this situation is unavoidable when the modulus method is used for caching. In order to solve these problems, the Hash consensus algorithm (consistency) Hash algorithm) is born!
Fourth, the mystery of the consistent Hash algorithm
The consistent Hash algorithm also uses a modulo method, but the modulo method just described is to modulate the number of servers, while the consistent Hash algorithm modulates 2^32. What does it mean?
In simple terms, the consistent Hash algorithm organizes the entire hash value space into a virtual circle. For example, suppose that the value space of a hash function H is 0-2^32-1 (that is, the hash value is a 32-bit Symbol shaping), the entire hash ring is as follows:
The entire space is organized in a clockwise direction, the point directly above the circle represents 0, the first point to the right of the 0 point represents 1, and so on, 2, 3, 4, 5, 6... until 2^32- 1, that is to say, the first point to the left of the 0 point represents 2^32-1, and 0 and 2^32-1 overlap in the zero point. We call this ring composed of 2^32 points Hash ring.
In the next step, each server uses Hash to perform a hash. Specifically, you can select the server's IP or host name as the key to hash, so that each machine can determine its position on the hash ring. The location of each server in the ring space after using the IP address hash is as follows:
Next, use the following algorithm to locate the data to access the corresponding server: use the same function Hash to calculate the hash value of the data key, and determine the position of the data on the ring, from this position to "walk" clockwise along the ring, the first encounter The server you arrive is the server it should locate!
For example, we have four data objects: Object A, Object B, Object C, and Object D. After hash calculation, their positions in the ring space are as follows:
According to the consistent Hash algorithm, data A will be assigned to Node A, B will be assigned to Node B, C will be assigned to Node C, and D will be assigned to Node D.
Five, the fault tolerance and scalability of the consistent Hash algorithm
Now suppose that Node C is down unfortunately. You can see that objects A, B, and D will not be affected at this time, only the C object is relocated to Node D. Generally, in the consistent Hash algorithm, if a server is unavailable, the affected data is only the server to the previous server in its ring space (that is, the first server encountered when walking in a counterclockwise direction) ), the other will not be affected, as shown below:
Consider another situation below. If you add a server Node X to the system, as shown in the following figure:
At this time, Object A, B, and D are not affected, only object C needs to be relocated to the new Node X! Generally, in the consistent Hash algorithm, if one server is added, the affected data is only the new server to the previous server in its ring space (that is, the first server encountered when walking in the counterclockwise direction) Data in between, other data will not be affected.
In summary, the consistent Hash algorithm only needs to relocate a small part of the data in the ring space for the increase or decrease of nodes, which has good fault tolerance and scalability.
Sixth, the data skew problem of the Hash ring
When the consistent Hash algorithm has too few service nodes, it is easy to cause data skew due to uneven node distribution (most of the cached objects are cached on a certain server). For example, there are only two servers in the system, and the ring is distributed. as follows:
At this time, a large amount of data will inevitably be concentrated on Node A, and only a very small amount will be located on Node B. In order to solve this data skew problem, the consistent Hash algorithm introduces a virtual node mechanism, that is, multiple hashes are calculated for each service node, and a service node is placed at each calculation result location, which is called a virtual node. The specific method can be realized by adding a number after the server IP or host name.
For example, in the above case, three virtual nodes can be calculated for each server, so "Node A#1", "Node A#2", "Node A#3", "Node B#1", and "Node A#1" can be calculated separately. The hash values of B#2" and "Node B#3", thus forming six virtual nodes
At the same time, the data positioning algorithm remains unchanged, but there is an extra step of mapping the virtual node to the actual node. For example, the data located to the three virtual nodes of "Node A#1", "Node A#2" and "Node A#3" are all located Go to Node A. This solves the problem of data skew when there are few service nodes. In practical applications, the number of virtual nodes is usually set to 32 or even greater, so even a few service nodes can achieve relatively uniform data distribution.
Seven, summary
In the above, we have analyzed step by step what is a consistent Hash algorithm, mainly considering that each node of the distributed system may fail, and new nodes are likely to be dynamically added. How to ensure the number of nodes in the system When changes occur, our system can still provide good services to the outside world, which is worth considering!