Cache penetration, avalanche, breakdown:
Cache penetration of redis:
refers to the fact that the data sent by the front end does not exist in redis or the database. If a large number of such requests are repeatedly received, performance will be wasted.
There are two solutions:
1. Cache empty objects and add an expiration time, which is relatively simple to implement and easy to maintain, but there will be additional memory loss, and it may cause short-term data inconsistencies (for example, cache an id with 1, this data does not really exist, but a data with an id of 1 is actually added later, which will cause data inconsistency)2. Using the Bolong filter, its memory footprint is small, but the implementation is more complicated, and there is still the possibility of misjudgment
Cache Avalanche:
It means that a large number of cache keys fail at the same time or the redis service is down, resulting in a large number of requests to directly access the database, which puts a lot of pressure on the database Solution
: add different expiration times to different keys, and then build a redis cluster , add multi-level cache to the business or limit traffic
Cache breakdown:
It refers to the sudden failure of a key that is highly concurrently accessed and the reconstruction business of the cache is relatively complex. A large number of requests will bring great pressure to the database in an instant.
Solution:
Mutex lock principle:
If the query cache data does not exist, first acquire the mutex lock, then query the database to write the found data into the cache and release the mutex lock. For example, thread A is a query request, and the query data is in If there is no cache, then acquire the mutex first, and then query the database. If the cache is not written and the lock is not released, thread B comes in, and thread B is also a query request. If the query data is not in the cache, it will Acquire the mutex, but it will fail when acquiring the mutex, because thread A has not released the lock, and then waits for a while to query the cache again. Assuming that thread A has written the cache successfully, then thread B will query the cache successfully. Return the data, otherwise it will be retriedLogical expiration principle:
For example, if thread A finds that the physical expiration time of the cache has arrived, it will acquire a mutex, then start a new thread B, and hand over the cache reconstruction work to thread B, and then thread A will directly Return the old cached data. When thread B has not written new cached data and released the lock, thread C comes. Thread C finds that the cached data is still expired and fails to acquire the lock, which means that there is a thread executing the cache After reconstruction, the old data will be returned directly. When thread B successfully reconstructs the cached data, the lock will be released.
Ensure data consistency between redis and database:
Redis data update: first operate the database, then delete the cache, and then add the cache expiration time to get the bottom line
1. If the cache is deleted first, the problem will be caused when the database is operated:
Normal situation:
thread A deletes the cache, and then updates the database. After the update is completed, thread B queries the data in the cache, finds that there is no database in the cache, and then reads Database, the data read at this time is the updated data, and then write the data into the cache
Abnormal situation:
thread A deletes the cache first, and then when it is about to update the database, thread B comes in. Thread B is a query request. It first queries the cached data, finds that there is no data in the cache, and then reads the database. The received data is written into the cache, and after thread B finishes executing, thread A finishes updating the database, which will lead to data inconsistency. Fast, so in the case of concurrency, the error of this strategy is relatively large
2. Operate the database first, and then delete the cache:
Normal situation:
After thread A finishes updating the database, delete the cache, and then thread B comes in. Thread B is a query request. After querying that there is no data in the cache, it will read The updated data is written to the cache
Abnormal situation:
Thread A is a query request. Assuming that there is no data in the cache, then thread A will query the database after querying the cache and then write the data into the cache. When writing cached data, thread B comes in Update the database. After the update is completed, the write cache operation of thread A is completed. At this time, the data will be inconsistent. data, but also to ensure that the operation of updating the database is faster than the operation of writing to the cache, but in general, the speed of redis is faster than that of the database, so the probability of this situation will be relatively small
However, the above two situations will lead to data inconsistency, so when adding cached data, an expiration time should be added to the cached data, so as to ensure the consistency of the final data
type of data:
Redis has five common data structures: string, list, hashmap, set, and zset
Master-slave replication:
One master node and multiple slave nodes. The master node can perform read and write operations, and the slave node can only be responsible for read operations. When the master node executes a write command, it will synchronize the data to the slave node. When reading data, Go to the slave node to read, so that you can share the read pressure of the master node. The operation of configuring master-slave replication is on the slave node, use the slaveof command, and then keep up with the ip and port number of the master node
Master-slave principle:
When the master-slave server is connected, full synchronization is performed first, and incremental synchronization will be performed after the slave node is reconnected.
Full synchronization process :
When the slave node is connected to the master node, it will send the psync command. After the master node receives the psync command, it will execute the bgsave command to generate the RDB snapshot file and use the cache area to record all write commands executed thereafter. When the bgsave command is completed, it will send all The slave node sends the RDB snapshot file. After receiving the RDB snapshot file, the slave node will write the file to the hard disk, then clear all the old data, and then load the received snapshot file from the local hard disk into the memory (based on the old data at the same time) Version provides external services), after the master node sends the RDB snapshot file, it continues to send the write command in the buffer to the slave node, and then the slave node receives and executes the command.
Incremental synchronization :
When the slave node reconnects, it will send its own reply and offset. After the master node receives it, it will obtain the subsequent data according to the offset, and then send it to the slave node. After the slave node receives it, it will start to synchronize
Sentinel mode:
Sentinel mode is an improved version of master-slave replication, because after the master node in master-slave mode hangs up, you need to manually change the slave node to the master node, and during this time, only read operations can be performed. This mode is efficient It is relatively low, so I joined the sentinel to monitor the master-slave server. When the master node is found to be down, it will automatically turn the slave node into the master node, and then notify other servers through the publish-subscribe mode, modify the configuration file, and replace the host. Moreover, clusters can also be made between sentries to prevent the sentry mode from being unable to be maintained if one sentry hangs up.
Sentinel mode principle:
Sentry mode has three timed tasks:
1. Send a request to the master node every 10s to obtain the latest information of all slave nodes
2. Broadcast the judgment of the current sentinel on the master node and the information of the current sentry node every 2s. The broadcast here is realized through the publish and subscribe mode of redis
3 .Ping commands will be sent to master-slave nodes and other sentinels every 1s for heartbeat detection.
Assuming that the master node is down, Sentinel 1 detects this result first, but Sentinel 1 does not immediately perform a failover operation, but subjectively believes that the master node is unavailable. When other sentinels also detect that the master node is unavailable, and the number When it reaches a certain value, a vote will be conducted among the Sentinels, and a Sentinel representative will be elected as a result of the voting, and the Sentinel representative will perform the failover operation. After the switch is successful, each sentinel will switch the host from the slave node it monitors through the publish and subscribe mode. This process is called objective offline. If there are multiple sentries, not only will each sentry listen to the master-slave node, but the sentries will also listen to each other
Clustering mode:
Whether it is master-slave or sentinel mode, the slave nodes still save all the data, which is a waste of memory space, and when the amount of data in the memory is too large, the performance of redis write operations will be affected, plus The write operation in the sentinel or master-slave mode is a stand-alone mode, which will be affected by the performance of the stand-alone, so the cluster mode appears.
Principle: There are 16384 built-in hash slots in the redis cluster. When a write command needs to be executed in the Redis cluster, redis first performs an algorithm processing on the key, and then calculates the remainder of the result to 16384, so that each key will correspond to a For the hash slots numbered between 0-16383, redis will evenly map the hash slots to different master nodes according to the number of master nodes, so as to achieve the effect of clustering