What is Redis?
Redis is a non-relational database. The data is in memory and the read and write speed is very fast. Redis is a single-threaded architecture, so it is thread-safe and uses IO multiplexing internally. Contains our commonly used data structures, while supporting persistence and cluster mode. It can also be used as a distributed lock or message middleware.
Redis data types
Redis supports commonly used data structures such as String, Hash, List, Set, Sorted Set. Redis uses a redisObject object to represent all key-values as shown below:
Type indicates which data type the data is, and encoding indicates how the data is stored inside redis. As the length of the data increases, redis may compress the data.
String: The string type is the most basic data type of Redis. First of all, keys are strings, and several other data structures are built on the basis of string types. The value of String can be a string (simple string, complex JSON, XML), number, or even binary (picture, audio, video), but the maximum value cannot exceed 512M.
Hash: A hash is a key-value collection. The hash of redis is a key-value mapping table of String
List: The list type is used to store multiple ordered strings. The elements in the list are ordered, which means that a certain element or a certain range of elements can be obtained by index subscript. Redis's list increment implementation is a doubly linked list, you can add an element to the head or tail of the list, or you can use the message queue.
Set: The Set type is also used to store multiple string elements. Unlike lists, duplicate elements are allowed in the set, and the set is unordered and must be able to get elements through the index. The bottom layer is implemented by hashtable. And set provides a very useful way to judge whether a member is in a set collection.
ZSet: The ordered set retains the feature that the set does not allow duplicate members, while supporting sorting. Provide each element score as a sorting basis. So an ordered collection can obtain queries with specified scores and element ranges.
The ordered set uses HashMap and jump table to ensure the storage and order of data. Hash stores the mapping of members to score, and the jump table stores all members. The sorting basis is based on the mapping of member to score stored in the HashMap, and the structure of the skip table can be used to obtain higher query efficiency.
Why is Redis so fast
Redis uses the C language. Generally speaking, programs implemented in C language are closer to the operating system and execute relatively faster.
Redis is completely based on memory operations, so the CPU will not become the bottleneck of Redis. Only the memory size and network bandwidth of the machine can become the bottleneck. Multithreading is to better squeeze the CPU. Since the CPU will not become the bottleneck of Redis, the single-threaded structure reduces unnecessary context switching and competition conditions. There is also no problem of locking in a multithreaded environment. Adopt non-blocking IO multiplexing model.
Redis's persistence mechanism
In order to ensure the efficiency of Redis, the data is stored in the memory, but when the redis hangs or the server restarts, the data in the memory will be lost when restarted. Therefore, redis will periodically persist data to disk to ensure data persistence. Currently there are two persistence strategies:
- RDB: RDB persistence is the process of generating snapshot files of current process data and saving them to disk
Advantages: RDB is a compact compressed binary file, very suitable for backup and full copy scenarios, and recovery is relatively fast.
Disadvantages: There is no way to achieve real-time persistence. When persisting, the redis process will execute fork to create a child process. The rbd persistence process is responsible for the child process and automatically ends after completion. The fork creation of the child process is a heavyweight operation that is frequently blocked. The implementation cost is relatively high. rdb uses binary storage in a specific format. There are multiple versions of reb files during the evolution of redis versions, and there will be a problem that the old version cannot be compatible with the new version of the rdb format
- AOP: Record each specified command in an independent log, and execute the command in the AOP file when restarting to achieve the command to restore data
Redis defaults to the persistence method of snapshot RDB, but restarting will give priority to using AOF files to restore data. Because the data set saved in the AOF file is usually more complete than the data set saved in the RDB file.
AOF can be persisted throughout the entire process, and AOF is enabled in the configuration. In this way, redis does not execute a command to modify the data, it will be appended to the AOF file. We can set up different synchronization measurements according to requirements, for example, synchronization once per second, so that even if there is a downtime, at most one second of data will be lost after restart. Therefore, AOF files are usually larger than RDB files. Therefore, recovering data based on AOF return visits may be slower than RDB.
Redis supports two persistence methods at the same time, so we can choose between AOF and RDB. Use AOF to ensure that data is not lost, as the first choice for recovery; use RDB for different degrees of cold backup, so that when AOF files are lost or damaged or unavailable, RDB can also be used for fast data recovery.
What exactly is cache penetration, cache avalanche, and cache breakdown
https://blog.csdn.net/qq_25448409/article/details/86566859
Cache and database consistency issues
In a distributed environment, data inconsistencies between the cache and the database are very easy to occur. If the project requires strong consistency in the cache, do not use the cache for a long time. In other words, adopt appropriate strategies to reduce the probability of inconsistency between the cache and the database, including appropriate update strategies, timely update of the cache after updating the database, and increased retry mechanisms for cache failures.
Talk about the cluster solution of redis
The redis cluster solution mainly includes Redis Sentinal and Redis Cluster
- Redis Sentinal: The sentinel mode focuses on high availability.
In Redis master-slave mode, although the master node can send data to the slave nodes, once the master node is down. At this time, it is also necessary to artificially modify the address of the master node placed in the application, and order all slave nodes to replicate the new master node. The whole process needs to be considered very troublesome. In order to make everything do not need to think intervention, so there is the Redis Sentinel architecture. Sentinel mainly includes liveness detection and automatic failover. The application side obtains the redis master node information from the sentinel cluster and then connects. When the master node of redis goes down, the sentinel cluster will vote to upgrade one slave node to the master node and point the other slave nodes to the new master node. The client application connects to the sentinel node collection during initialization and obtains the master node information from it.
https://blog.csdn.net/qq_25448409/article/details/106985362
- Redis Cluster: Focusing on scalability, when a single redis is insufficient in memory, Cluster is used for shard storage.
https://blog.csdn.net/qq_25448409/article/details/86013885
Redis Cluster uses virtual slot partitioning. In fact, I feel that it is similar to the consistent hash algorithm. It maps all keys to integer slots from 0 to 16383 according to the hash function. Each node maintains a part of the slot and the key value data of the slot mapping.
Talk about Redis distributed locks
If Redis stand-alone condition, we only need to use String data type nx, px to realize a simple distributed lock
SET resource_name my_random_value NX PX 30000- NX : It means that the setting will be successful only when the key does not exist. (If this key exists in redis at this time, then the setting fails and returns nil )
- PX 30000 : It means that the lock is automatically released after 30s. When someone else creates it, if they find it already exists, they can't lock it.
In order to prevent the client from acquiring the lock, some operations were blocked and the time exceeded the effective time of the lock, and then the lock acquired by another client was deleted. We can use atomic lua scripts. Set an expected value when locking, and only the value stored in the key is the one we set when locking can be released.
if redis.call("get",KEYS[1]) == ARGV[1] then
return redis.call("del",KEYS[1])
else
return 0
endThe above implementations are all based on a single Redis. If we build a cluster, the keys will be scattered in different machines. What should we do at this time? The official solution for multi-node Redis became RedLock. After reading it, I found it was useless and there were still many problems. If your Redis has multiple nodes, use Zookeeper directly for distributed locks. This is the optimal solution for distributed locks.