The giant brother talks about the use of cache in high concurrency scenarios

cache coherency issues

When the data timeliness requirements are very high, it is necessary to ensure that the data in the cache is consistent with that in the database, and it is necessary to ensure that the data in the cache node and the replica are also consistent, and differences cannot occur. This is more dependent on the expiration and update strategy of the cache. Generally, when the data changes, the data in the cache is actively updated or the corresponding cache is removed.

cache concurrency problem

After the cache expires, an attempt will be made to fetch data from the backend database, which is a plausible process. However, in a high concurrency scenario, it is possible for multiple requests to obtain data from the database concurrently, which has a great impact on the back-end database, and even leads to an "avalanche" phenomenon. In addition, when a cache key is updated, it may also be fetched by a large number of requests, which will also lead to consistency problems. So how to avoid similar problems? We will think of a mechanism similar to "lock". In the case of cache update or expiration, try to acquire the lock first, and release the lock when the update or acquisition from the database is completed. Other requests only need to sacrifice a certain waiting time. Continue fetching data directly from the cache.

cache penetration problem

Cache penetration is also called "breakdown" in some places. Many friends' understanding of cache penetration is that due to cache failure or cache expiration, a large number of requests penetrate to the back-end database server, which has a huge impact on the database.

This is actually a misunderstanding. The real cache penetration should look like this:

In a high concurrency scenario, if a key is accessed with high concurrent access and is not hit, for fault tolerance, it will try to obtain it from the back-end database, resulting in a large number of requests reaching the database. When the data itself is empty, this leads to the concurrent execution of many unnecessary query operations in the database, resulting in huge impact and pressure.

There are several common ways to avoid caching legacy issues:

Cache empty objects

Objects with empty query results are also cached. If it is a collection, an empty collection (non-null) can be cached. If a single object is cached, it can be distinguished by the field identifier. This prevents requests from penetrating to the backend database. At the same time, it is also necessary to ensure the timeliness of cached data. This method is less expensive to implement, and is more suitable for data that has low hits but may be updated frequently.

separate filter processing

All keys whose corresponding data may be empty are stored uniformly, and intercepted before the request, so as to prevent the request from penetrating into the back-end database. This method is relatively complex to implement, and is more suitable for data with low hits but infrequent updates.

cache thrashing problem

The cache thrashing problem, which may be called "cache thrashing" in some places, can be seen as a lesser failure than "avalanche", but it can also cause shock and performance impact to the system for a period of time. Usually caused by cache node failure. The recommended practice in the industry is to solve it through the consistent Hash algorithm. I won't go into too much detail here, you can refer to other chapters

Cached Avalanche

Cache avalanche means that due to the cache, a large number of requests reach the back-end database, resulting in the collapse of the database, the collapse of the entire system, and a disaster. There are many reasons for this phenomenon. The above-mentioned issues such as "cache concurrency", "cache penetration", and "cache thrashing" may actually cause cache avalanches. These issues may also be exploited by malicious attackers. In another situation, for example, at a certain point in time, the caches preloaded by the system are periodically invalidated, which may also lead to an avalanche. In order to avoid this periodic invalidation, the cache expiration can be staggered by setting different expiration times, thereby avoiding centralized cache invalidation.

From the perspective of application architecture, we can reduce the impact by means of current limiting, downgrade, and circuit breaker, and we can also avoid such disasters through multi-level caching.

In addition, from the perspective of the entire R&D system process, stress testing should be strengthened to simulate real scenarios as much as possible, and to expose problems as soon as possible to prevent them.

I specially sorted out the above technologies. There are many technologies that can’t be explained clearly in a few words, so I just asked a friend to record some videos. The answers to many questions are actually very simple, but the thinking and logic behind them are not simple. If you know it, you also need to know why. If you want to learn Java engineering, high performance and distributed, explain the profound things in simple language. Friends of microservices, Spring, MyBatis, and Netty source code analysis can add my Java advanced group: 433540541. In the group, there are Ali Daniel live-broadcasting technology and Java large-scale Internet technology videos for free to share with you.

Cache bottomless phenomenon

This problem was raised by the staff of facebook. Facebook had 3,000 memcached nodes around 2010, caching thousands of gigabytes of content.

They found a problem---memcached connection frequency, efficiency decreased, so they added memcached nodes,

After adding it, it was found that the problem caused by the connection frequency still exists and has not improved, which is called "bottomless pit phenomenon".

At present, the mainstream technology stacks such as database, cache, Nosql, and search middleware all support "sharding" technology to meet the requirements of "high performance, high concurrency, high availability, and scalability". Some are mapped to different instances by Hash modulo (or consistent hash) on the client side, and some are mapped by range values on the client side. Of course, some are done on the server side. However, each operation may require network communication with different nodes to complete. The more instance nodes, the greater the overhead and the greater the impact on performance.

It can be avoided and optimized mainly from the following aspects:

data distribution

Some business data may be suitable for Hash distribution, while others are suitable for range distribution, which can avoid network IO overhead to a certain extent.

IO optimization

You can make full use of connection pooling, NIO and other technologies to reduce connection overhead as much as possible and enhance concurrent connection capabilities.

data access method

Obtaining a large data set at one time will have less network IO overhead than obtaining a small data set multiple times.

Of course, the phenomenon of cache bottomless is not common. Probably not at all in most companies