Server application multi-level cache architecture solution | JD Cloud technical team

One: scene

In the scenario of 20w QPS, how should the server architecture be designed?

Two: conventional solutions

Distributed cache can be used to resist, such as redis cluster, 6 masters and 6 slaves, the master provides read and write, and the slave serves as the backup, and does not provide read and write services. One machine can resist 3w concurrency on average, and it can still resist. If the QPS reaches 100w, by increasing the number of machines in the redis cluster, the cache capacity and concurrent read and write capabilities can be expanded. At the same time, the cached data is shared by the application, and the master-slave architecture achieves high availability.

Three: How to solve the cache hotspot (hot key) problem

However, if there are cache hotspots, such as 100,000 traffic coming from the same key and hitting the same redis instance, then the CPU may be full. This increase in the number of redis clusters cannot solve the problem.

The local cache can solve the hot key problem. The main reason is that the local cache can avoid the high load of a single redis cache server. By replicating multiple cache copies and distributing requests to multiple cache servers, the pressure on a single cache server caused by cache hotspots can be reduced. In addition, local memory caching also has faster access speeds because the data is stored in the application's memory without the need to transfer data across the network.

Four: General multi-level caching scheme

The request is first sent to the local cache of the application. If the local cache does not exist, it will be pulled from the r2m (redis) cluster and cached locally at the same time.

Five: Multi-level cache synchronization scheme

1 The operation background saves the data, writes it into the r2m cache, and publishes the message through the publish and subscribe function of redis

2 The local application cluster, as a message subscriber, deletes the local cache after receiving the message. When the C-side traffic request comes in, if the local cache does not exist, the cache in r2m is loaded into the local cache.

3 The timing task is to prevent the r2m cache from being invalidated in extreme cases, and reload the data to the r2m cache.

Six: Cache synchronization component selection

Publish and subscribe using redis.

The publish-subscribe mode of Redis is push mode. In Redis, the SUBSCRIBE command is used to subscribe to one or more channels to receive notifications when messages are published to those channels. The PUBLISH command is used to publish a message to one or more channels. When a message is published to a channel, all clients subscribed to the channel will receive the message. In the push mode, each channel maintains a list of clients, and when sending a message, it traverses the list and pushes the message to all subscribers. The pull mode is the opposite. The sender puts the message in a mailbox, and all clients who subscribe to this mailbox can receive it at any time. Make sure all clients have successfully retrieved the complete message before deleting the message.

Redis publish-subscribe is asynchronous. When a message is published to a channel, Redis will asynchronously push the message to all clients subscribed to the channel. This means that the client will not block waiting for the message, but continue to perform other tasks, and will not get it until it needs to receive the message. This asynchronous approach can improve the concurrency and efficiency of the system.

Seven: Precautions for using local cache

1 The local cache occupies the jvm memory space of the java process, so it cannot store a large amount of data, and needs to evaluate the cache size.

2 The business can accept the inconsistency of short-lived data, which is more suitable for reading scenarios.

3 Cache update strategy, active update and passive update, the local cache must set the validity period

4 Timed task synchronization cache mechanism, considering extreme data loss according to business conditions

5 RPC calls avoid local cache pollution, which can be solved by deep copying.

6 The local cache becomes invalid as the application restarts, pay attention to the timing of loading the distributed cache

7 The pub and sub mode of redis update the cache strategy (delete the local cache key to avoid passing large values in the pub and sub mode, the pub and sub mode will not persist the message data, causing the consumer's corresponding redis buffer to exceed the limit, thus result in data loss), when the local cache fails, the lock is synchronized, and one thread loads the r2m cache to avoid concurrent updates.

Remarks: The bottom layer of r2m is implemented by redis.

Author: JD Technology Zhang Shilei

Source: JD Cloud Developer Community