redis cache avalanche
If we hang the cache, which means that our requests were all ran the database.
Data is not loaded into the cache, a cache miss or a large area at the same time, causing all requests to check the database, the database resulting in high CPU and memory load, and even downtime.
We all know that Redis can not put all the data is cached (expensive and limited memory), so the need for data redis set the expiration time, and the use of inert + Delete to delete the two strategies to expire periodically delete key.
If the expiry time is set the same cached data, and data of this part Redis just delete all the light, by this time will result in, at the same time these caches fails, all the requests to the database.
This is the cache Avalanche: Redis hung up, go all database requests.
If the cache avalanche occurs, it is likely to bring down put our database, resulting in paralysis of the entire service.
Such as a simple process of avalanche:
1, large fault redis cluster;
2, a cache miss, but still a large number of requests diverted to MySql database;
. 3, redis large inactivation, a large number of requests diverted to mysql database;
4, calls the amount of mysql explosion, soon could not carry, or even directly down;
5, due to the large number of applications depend on mysql server and redis service, this time quickly turn into an avalanche cluster servers, and finally the total collapse of the website.
How to prevent caching avalanche
to the expiration time plus a random value when the cache, which would greatly reduce the cache expire at the same time.
Of the "Redis hung up, go all database requests" This situation, we can have the following ideas:
Prior to the incident, to achieve high availability of Redis (master-slave architecture + sentinel or Redis cluster), try to avoid hang Redis this happens.
The incident, the case redis really hang up, we can set up a local cache + current limiting (hystrix), try to avoid our database was to kill (or at least ensure that our services work properly).
After the incident, redis persistent, restart automatically load the data from the disk, cache data fast recovery.
1, high availability cache
buffer layer is designed to be highly available, fault prevent caching large area, even if the individual nodes, individual machines, even shoot down the engine room,
2, cache downgrade
may be utilized ehcache like local cache (temporary support), but mainly to limit the current source service access, resource isolation (fuse), demotion, etc.
When the traffic surge, the service problems still need to ensure the availability of services, the system can automatically downgrade, according to some critical data, you can configure the switch to achieve artificial downgrade, there will be designed to coordinate operation and maintenance.
The aim is to ensure that the core downgrade service is available, even for lossy.
For example, recommendation services, many of which are individual needs, while adding individual needs can not provide the service, you can downgrade to add hot data, and will not cause a big front page blank.
To sort out the system prior to downgrading, for example: those services is the core business (must ensure) that those services can not be allowed to temporarily provide services (the use of static pages substitution, etc.) as well as with the core index server, set up after the overall plan to, for example, : (1)
general: some services such as occasional jitter or the network service is timed out on the line, to automatically downgrade;
(2), warning: some service success rate fluctuations over time (e.g., between 95-100%) . It can automatically or manually downgrade downgrade. And send alerts.
(3), the error: such as less than 90% can be used. Or database connection pool was beaten storm. Or views suddenly jumped to the system can withstand the maximum threshold, this time may be automatically or manually downgrade degraded under the circumstances.
(4), a critical error: for example, for special reasons data error, this time in need of emergency manual downgrade.
3, redis backup and rapid warm-up
(1) redis data backup and recovery
(2) preheating fast cache
4, ahead of the exercise
Finally, the proposal is still on the front line item after exercise caching layer dawdle out, applications and back-end load and the problems that may arise, highly available for preview in advance, identify problems early.
Cache penetration
caching refers to penetrate a certain query data does not exist, due to the cache miss, and for fault-tolerant considerations, finding out if the database data is not written to the cache, which will lead to the absence of each database requests to search the database, it lost the meaning of the cache.
Cache penetration refers to a query data does not exist, for example: no redis hits from the cache, you need to query from the mysql database, finding the data is not written to the cache, which will result in a time that does not exist in the data request should be go to the database query result caching penetration.
Solution:
Since the parameters of the request is not legitimate (every request parameter does not exist). So we can use compressed Bloom filter or filter ahead of interception. Do not let illegal requests to the database layer.
When we could not find the database data, we will empty the cache object is set to go, and then request the next time, you can get the inside from the cache.
In this case we will generally empty object set a short expiration time.
If you query the database also empty directly set a default value stored in the cache, so the second time to get there is value in the cache, but will not continue to access the database, you can set some formatting rules to set up a key expired, then query prior to filter out does not comply with the rules of Key *
Cache Concurrency
Concurrent herein refers to a plurality of client redis simultaneously set.
concurrency issues key cause, in fact, redis itself is single-threaded, the principle of first come, first installed to perform concurrent operation of multiple client, first come, first implemented, the rest of the obstruction, of course, another solution is placed in a queue to redis.set serialize it must be executed one by one.
Cache warming
After preheating means cache line system, the load directly to the relevant cache data cache system.
This can be avoided when the user requests, first query the database, then the data cache issue, so that the user direct access to the cached data to be preheated.
Solutions
1, direct write cache to refresh the page, under the manual on-line time;
2, the amount of data can be loaded automatically when the project starts;
purpose is at the front of the line system, data is loaded into the cache.
