At present, distributed kv caches such as Memcached and Redis are very common. Starting from this article, this series will analyze the principles, usage strategies and best practices related to distributed caching.
We know that the distribution of Memcached is actually a "pseudo-distribution", that is, its server nodes are actually unrelated to each other, and there is no network topology relationship between them. The client decides where a key is stored. machine.
Specifically, suppose I have multiple memcached servers numbered m0,m1,m2,…. For a key, it is up to the client to decide which machine to store. The simplest hash formula is key % N, where N is the total number of machines.
But there is a problem. Once the number of machines is reduced, or the number of machines is increased, and N changes, all the previously stored data will be invalid. Because the machine number you calculated based on the new N value modulo is definitely different from the machine number calculated based on the old N value modulo at that time, which means that most of the caches will be invalid.
The solution to this problem is to use a special Hash function to minimize the number of cache invalidations when adding machines/reducing machines as much as possible. This is the Hash ring, or Consistent Hash.
Interested friends can follow the public account "The Way and Technique of Architecture" to get the latest articles.
Or scan the QR code below:
Hash ring
The Hash function mentioned above has only undergone 1 hash, that is, the key is hashed to the corresponding machine number.
The Hash ring has 2 hashes:
(1) Hash all machine numbers to this ring
(2) Hash keys to this ring. Then match on this ring to see which machine this key matches.
Specifically, as follows:
Suppose there is such a Hash function whose value space is (0 to 2 to the 32nd power -1), that is, its hash value is a 32-bit integer-free number, and these numbers form a ring.
Then, hash the machine (for example, according to the ip of the machine), and calculate the position of each machine on the ring; then hash the key to calculate the position of the key on the ring, and then move forward from this position. The first machine that arrives is the machine corresponding to the key, and the (key, value) is stored on the machine.
As shown below:
First calculate the position of each Cache server on the ring (the big circle in the figure); then each time a (key, value) comes, calculate the position on the ring (the small circle in the figure), and then go clockwise, The first machine it encounters is the one it wants to store.
The key point here is that when you add/remove machines, the position of the other machines on the ring doesn't change. In this way, only the data near the increased machine or the decreased machine will be invalid, and the data on other machines will still be valid.
data skew problem
When you have few machines, it is very likely that several machines are attached very close to the ring, not evenly distributed on the ring. This will cause most of the data to be concentrated on one machine.
In order to solve this problem, the concept of "virtual machine" can be introduced, that is to say: 1 machine, I calculate multiple positions on the ring. How to do it? Assuming that the ip of the machine is used to hash, I can add several numbers after the ip, ip_1, ip_2, ip_3, .. to generate a number of multiple virtual machines for one physical machine.
Data is first mapped to the "virtual machine" and then from the "virtual machine" to the physical machine. Because there can be many virtual machines, they are evenly distributed on the ring, so as to ensure that the data is evenly distributed to the physical machines.
Introduction of ZK
We mentioned the increase and decrease of the server's machines above. The question is how does the client know?
A stupid way is to do it manually, when the server machine increases or decreases, reconfigure the client and restart the client.
The other is to introduce ZK, the server's node list is registered on ZK, and the client listens to ZK. If the number of nodes is found to have changed, it will automatically update its own configuration.
Of course, instead of ZK, it is also possible to use another central node, as long as the notification of such changes can be realized.