【Moving Brick-4】Distributed Cache

Redis、Memcached

Redis

1. Definition

Key-Value type memory database, much like memcached, the entire database is loaded in memory for operation.

Can be used as a message queue: List is used as a FIFO doubly linked list to realize a lightweight high-performance message queue service
Can be used as a tag system: SortedSet can be used as a high-performance tag system

2. Data Type

String, set, ordered set, list, hash

3. Disadvantages

Due to the limitations of physical memory, it cannot be used for high-performance reading and writing of massive data. Therefore, the suitable scenarios for Redis are mainly limited to high-performance operations and calculations with a small amount of data.

4. Cache avalanche

definition

The data in the cache has the same expiration time, all of which are invalid at a certain time, and all data goes to the database
Redis all hang up, all data goes to the database

Solution

Advance:
1. Analyze user behavior and make the failure time discrete
2. Use locks or single-threaded write cache to avoid a large number of concurrent requests falling on the database
3. Let redis data never expire
4. Redis is highly available, master-slave + sentinel, redis cluster, to avoid total crash
In the event:
1. Local cache Guaua, Caffeine
2. Hystrix current limit, downgrade of failed requests, return to default value, blank page, etc.
afterwards:
1. Redis is persistent. Once restarted, it will automatically load data from the disk and quickly restore cached data.

4. Cache penetration

definition

A malicious attack by hackers deliberately requests data that does not exist in the database, that is, it does not exist in the cache.
For example, the database id starts from 1, as a result, the request id sent by the hacker is all negative. In this case, it will not be in the cache, and every time the request is "seeing the cache as nothing", it directly queries the database. The cache penetration of this malicious attack scenario will directly kill the database.

Solution

Every time system A does not find it from the database, it writes a null value to the cache, such as set -999 UNKNOWN. Then set an expiration time, so that when the same key is accessed next time, the data can be directly retrieved from the cache before the cache expires.

5. Cache breakdown

definition

A certain key is very hot, frequently accessed, and is in a situation of centralized high concurrent access. When the key fails, a large number of requests break through the cache and directly request the database, just like breaking through a barrier A hole.

Solution

Set hotspot data to never expire;
Realize distributed lock based on redis, wait for the first request to build the cache, then release the lock, and then other requests can access data through the key.
Setnx means set if not exist. If the key does not exist, set the value of the key, and the expiration time is expire_secs. Return 1 means that the key does not exist and the setting is successful.

public String get(key) {
    
    
    String value = redis.get(key);
    if (value == null) {
    
     //代表缓存值过期
        //设置3min的超时，防止del操作失败的时候，下次缓存过期一直不能load db
        if (redis.setnx(key_mutex, 1, 3 * 60) == 1) {
    
      //代表设置成功
            value = db.get(key);
            redis.set(key, value, expire_secs);
            redis.del(key_mutex);
       	} else {
    
      //这个时候代表同时候的其他线程已经load db并回设到缓存了，这时候重试获取缓存值即可
               sleep(50);
               get(key);  //重试
       	}
    } else {
    
    
        return value;      
    }
}

5. Why fast

1) Redis is a pure memory database
2) Redis uses non-blocking IO, IO multiplexing, using a single thread to poll the descriptor, turning the database on, off, reading, and writing into events, reducing Context switching and competition during thread switching.
3) Redis uses a single-threaded model.
4) The data structure also helps a lot. Redis uses the hash structure throughout the process, which is fast to read

6. Usage scenarios

1) Session cache

Realize the session function

2) Full page cache

Based on the persistence implementation, even if Redis restarts, the user will not notice it, and it is stored in String format

3) Queue

Because it provides list and set operations, it is used in elk to avoid node log loss. When input is filebeat, the output is logstash.
It can also provide list paging functions, such as blog list paging

4) Leaderboard/Counter

Redis implements the operation of incrementing or decrementing numbers in memory very well. Set (Set) and ordered set (SorteSet) also make it very simple for us to perform these operations, such as getting the top 10 users from the sorted set

5) Publish/Subscribe

7. What are the client implementations of Redis java

Redisson, Jedis, lettuce, etc., Redisson is officially recommended

8. Endurance

Persistence is to write memory data to disk to prevent memory data loss when the service is down.

RDB（Redis DataBase)
AOF (Append Only File)
RDB is an RDB file that snapshots data to disk at intervals.
AOF continuously records the user's read and write operations into the AOF file. After Redis restarts, it executes it again to copy user operations.
Both of the above two modes can update RDB and AOF files regularly.
https://www.cnblogs.com/chenliangcl/p/7240350.html

9. Redid vs Memcached difference

Redis single thread, Memcached multi thread
Redis value max 1G, Memcached max 1M
Redis supports string, hash, set, sortedset, list, Memcached only supports string
Redis supports persistence, Memcached does not

10. If there are 100 million keys in Redis, 10w of them start with a fixed known prefix, how to find them all?

Use the keys command keys start* to find

11. If this Redis is providing services to online businesses, what are the problems with using the keys command?

Redis is single-threaded. The keys instruction will cause the thread to block for a period of time, and the online service will pause. The service cannot be restored until the instruction is executed. At this time, you can use the scan command. The scan command can extract the key list of the specified mode without blocking, but there will be a certain probability of repetition. It is enough to do the deduplication once on the client side, but the overall time will be more than direct use The keys instruction is long.

12. How to choose between RDB and AOF?

Using RDB alone will lose a lot of data. Using AOF alone will cause data recovery to be faster than RDB. When something really goes wrong, use RDB to recover as soon as possible, and then AOF can do data completion.

13. The master-slave mode of redis?

The master machine writes, and the data is synchronized to other slave machines. They all read it, distribute a large number of requests, and can easily achieve horizontal expansion when expanding.

14. How to synchronize the master and slave of redis?

When starting a slave, he will send a psync command to the master. If this slave connects to the master for the first time, he will trigger a full replication. The master will start a thread, generate an RDB snapshot, and cache all new write requests in memory. After the RDB file is generated, the master will send the RDB to the slave. The first thing the slave does after getting it is Write it to the local disk and load it into the memory. Then the master will send all the new commands cached in the memory to the slave.