Table of contents
When the pressure on the database increases, the response to the service accessing the database becomes slower, causing the pressure on the service to increase, which may eventually lead to service downtime.
1. Cache penetration
1.1 Problem description
The data corresponding to the key does not exist in the database. Each request for the key cannot be obtained from the cache, so the database will be requested. When there are many requests for this key, the pressure on the database will increase, causing database downtime.
At this point the cache is still running smoothly, but it has no effect.
For example: if you use a non-existent user ID to obtain user information, the data does not exist in either the cache or the database. If a hacker exploits this vulnerability, the database may be overwhelmed.
1.2 Solution
A cache that must not exist and data that cannot be queried. If the data cannot be found in the database and is not written to the cache, then the data cannot be found in the cache and each request needs to be queried in the database. This will cause the cache to become meaningless.
- Cache null values : If no data is queried from the database, the return value is null. Regardless of whether it exists or not, we still cache the null value (null), but the expiration time of the null value is very short, no more than 5 minutes. . It can only be used as a temporary solution.
- Set the accessible list (whitelist) : Use the bitmaps type to define an accessible list. The list id is used as the offset of the bitmaps. Each access is compared with the id in the bitmaps. If the id is not in bitmaps, it will be intercepted and access will not be allowed.
- Use Bloom filter : The bottom layer of Bloom filter also uses hash. The advantage is high space efficiency and query time efficiency. The disadvantage is that there is a certain misrecognition rate and deletion difficulty.
- Conduct real-time monitoring : When it is found that the hit rate in the redis cache drops sharply, check the access objects and access data, and cooperate with the operation and maintenance personnel to set up a blacklist to restrict access.
2. Cache breakdown
2.1 Problem description
The data corresponding to the key exists, but expires in redis. If there are a large number of concurrent requests at this time, these requests find that the cache has expired, and the request will be sent to the database. After the data is found, the data will be cached in redis. At this time, a large number of concurrent requests may instantly overwhelm the database.
There are a large number of requests at the same time, and the requests are for expired data in redis, and the requests are forwarded to the database. The large number of requests causes excessive pressure on the database, causing database downtime.
Phenomenon: 1. The pressure on the database increases instantaneously. 2. There are not a large number of key expirations in redis. 3. redis is running normally and the pressure has not increased.
Reason: 1. A certain key in redis has expired, and a large number of accesses use this key.
2.2 Solution
The key may be accessed very concurrently at a certain point in time and is a very hot data. If the key in the cache expires, it will increase the pressure on the database, leading to downtime and cache breakdown.
- Set popular data in advance : Before redis peak access, store popular data into redis in advance, and increase the expiration time of these popular data keys.
- Real-time adjustment : On-site monitoring of which popular data is monitored, which data is frequently accessed, and the key expiration length adjusted in real time.
- Use lock :
- When the cache expires (the value taken out is judged to be empty), the database is not accessed immediately.
- First use some operations of the cache tool with a successful operation return value (such as redis's setnx) to set a mutex key.
- When the operation returns successfully, the operation of requesting the database is performed, the cache is set back, and the mutex key is finally deleted.
- When the operation fails to return, it proves that there is a thread accessing the database, and the current thread waits for a period of time before retrying the entire get cache method.
- The disadvantages of locking will lead to lower efficiency. Only one thread can access the database each time.
3. Cache avalanche
3.1 Problem description
The data corresponding to the key exists, but expires in redis. If there are a large number of concurrent requests at this time, these requests will generally send the request to the database and set the data back to the cache if they find that the cache has expired. At this time, a large number of concurrent requests will be instantaneously Overwhelm the database, causing database downtime.
The difference between cache avalanche and cache breakdown is that cache avalanche targets the expiration of many keys, while cache breakdown only targets the expiration of one key.
Phenomenon: 1. In a very short period of time, a large number of keys in redis expire.
3.2 Solution
The avalanche problem of cache invalidation has a terrible impact on the underlying system.
- Build a multi-level cache architecture : nginx cache + redis cache + other caches (echache, etc.). If redis cannot handle it, search in other caches.
- Use locks or queues : Use locks or queues to ensure that a large number of threads do not access the database in large quantities at one time. This prevents a large number of concurrent requests from falling on the underlying storage system during failure, which is not suitable for high concurrency situations .
- Set the expiration flag to update the cache : record whether the cached data has expired (set the advance amount). If it expires, it will trigger a notification to another thread to update the actual key cache in the background.
- Spread the cache expiration time : For example, we can add a random value to the original expiration time, such as random branch 1-5, so that the repetition rate of each cache's expiration time will be reduced, and it will be difficult to cause a collective failure time. .