1 Consistency of the Consistent Hash Algorithm
Consistency here means that the algorithm can maintain the consistency of the data in memcached and the database.
2 What is a consistent hash algorithm
2.1 Why do we need a consistent hash algorithm?
Now there is a large amount of key value data that needs to be scattered and stored in memcached on different machines, how to assign machines according to the key.
The simple method is to directly modulate the number of machines. For example, if there are 3 machines, first hash the key. If the hashes are 1, 2, 3, 4, 5, 6, 7, and 8, then
3, 6 are dispatched to the first machine
1, 4, 7 are dispatched to the second machine
2, 5, 8 are dispatched to the third machine
However, if one machine and 4 machines are added, then it is necessary to take the modulo of 4,
4, 8 are dispatched to the first machine
1, 5 are dispatched to the second machine
2, 6 are dispatched to the third machine
3, 7 are dispatched to the fourth machine
In this way, 3 was originally the third machine, but now it is assigned to the fourth machine, then the original data is invalid, and the data of the fourth machine needs to be re-fetched from the database. 4, 5, 6, 7, 8 are the same.
The problem caused by this is that a large amount of data cached by the original memcached is invalid, and it needs to be re-fetched, which is very expensive.
2.2 How to do the consistent hash algorithm
The hash value is also calculated for the machine according to the ip. For example, machine 1 is 3, machine 2 is 5, and machine 3 is 9.
Now the assignment strategy becomes, if the hash value of the key is greater than 9, it is assigned to machine 1, if it is greater than 3, it is assigned to machine 2, and if it is greater than 5, it is assigned to machine 3.
If a new machine 4 is added, the hash value is 8, then if the key is greater than 5, it is assigned to machine 4, and if it is greater than 8, it is assigned to machine 3, and the rest remain unchanged.
In this way, only the memcached on machine 3 needs to update some data, and the memcached on other original machines is consistent with the database.
2 memcached
It is a distributed cache, which implements a distributed cache through the client, and caches data to different machines according to the key.
Clients can use xmemcached, spymemcached, etc.
boolean set(final String key, final int exp, final Object value);
It can be seen that any object can be stored in memcached.