1. What is Redis? What is its main use?
Answer: Redis is an open source in-memory data structure storage system that can be used as a database, cache, and message broker. It supports various data structures such as strings, lists, hash tables, sets, and sorted sets. The main uses of Redis include caching, session storage, leaderboards, counters, and publish/subscribe patterns, among others.
2. What are the advantages of Redis?
Answer: The advantages of Redis include high performance, scalability, data persistence, rich data types, support for transactions and Lua scripts, and flexible configuration options. Additionally, Redis supports multiple programming languages such as Python, Java, and Ruby, allowing for easy integration with applications.
3. How does Redis implement data persistence?
Answer:
Redis can achieve data persistence by saving data to files on disk.
Redis provides a variety of persistence methods, including RDB and AOF.
RDB saves Redis data to snapshot files on disk, while AOF records Redis write operations and saves them to log files on disk. At server startup, Redis can restore data from snapshot files or log files.
4. How does Redis handle concurrent access?
Answer: Redis is single-threaded, but it uses asynchronous I/O and multiplexing to handle multiple client requests at the same time. In addition, Redis uses an optimistic locking mechanism to avoid concurrent access problems. If two clients try to modify the value of the same key at the same time, Redis will perform different operations according to the type of operation (such as SET, INCRBY).
5. What are the data structures of Redis?
Answer: Redis supports a variety of data structures, including strings, lists, hash tables, sets, and sorted sets. Strings are the simplest data structure used to store text or binary data. A list is an ordered list of strings that can be used to implement queues, stacks, and publish/subscribe patterns, etc. Hash tables are used to store key-value pairs, similar to associative arrays. A set is an unordered collection of strings that can be used to implement intersection, union, and difference. An ordered set is an ordered collection of strings, and each string has a score associated with it, which can be used to implement leaderboards, counters, etc.
6. What is the master-slave replication of Redis?
Answer: The master-slave replication of Redis refers to copying the data of one Redis server to other Redis servers, so that each server can independently provide read operation services. Master-slave replication can be divided into the following processes:
-
Primary server persistence operation: The primary server records write commands into AOF or RDB files.
-
Slave server synchronization operation: the slave server establishes a connection with the master server, receives and executes the write command of the master server.
-
Online replication from the server: the master server is disconnected from the slave server, and the slave server becomes an independent Redis database that can accept read operation requests.
7. What is the expiration strategy of Redis?
Answer: Redis implements the expiration strategy through the expiration time. There are two main expiration strategies:
-
Periodic deletion: Redis traverses the entire data set at regular intervals and deletes expired key-value pairs.
-
Lazy deletion: When Redis reads and writes key-value pairs, it will check the expiration time and delete it immediately if it expires.
8. What is the difference between Redis and Memcached?
Answer: Both Redis and Memcached are memory-based caching systems, but Redis supports more data structures than Memcached, such as hashes, lists, sets, ordered sets, etc., and can also persist data to disk. In addition, Redis also provides more functions, such as publish subscription, Lua script, transaction and so on.
9. What is the architectural design of Redis?
Answer: The architectural design of Redis adopts a single-threaded approach, that is, all requests are processed by one thread. This design can avoid the competition problem of multithreading and improve performance and stability. At the same time, Redis also supports concurrent access by multiple clients.
10. What advanced functions does Redis support?
Answer: Redis supports a variety of advanced functions, such as publish and subscribe, Lua scripts, transactions, etc. Publish-subscribe is a messaging pattern used to broadcast messages to multiple clients. Lua script is an embedded scripting language that can be executed in Redis. A transaction is a collection of commands whose atomicity can be guaranteed.
11. What kinds of data elimination strategies does Redis have?
- noeviction: returns an error when the memory limit is reached and the client tries to execute a command that would cause more memory to be used (most
write commands, but DEL and a few exceptions) - allkeys-lru: Attempts to reclaim the least used keys (LRU) to make room for newly added data.
volatile-lru: Attempts to reclaim the least used keys (LRU), but only for keys in expired collections, so that there is room for newly added data .- allkeys-random: Recycle random keys to make room for newly added data.
- volatile-random: Recycle random keys to make room for newly added data, but only for keys in expired collections.
- volatile-ttl: Recycle the keys in the expired collection, and give priority to recycle the keys with a shorter time-to-live (TTL), so that the newly added data
has space to store.
12. What should be done in the Redis cluster solution? What are the plans?
- twemproxy, the general concept is that it is similar to a proxy method, and its usage is no different from that
of ordinary redis. After setting up multiple redis instances under it, when using it, connect to twemproxy at the place where redis needs to be connected, and it
will Receive the request as a proxy and use the consistent hash algorithm to transfer the request to specific redis, and return the result to
twemproxy. Easy to use (relative to redis only need to modify the connection port), the first choice for old project expansion. Problem : The pressure of twemproxy's own single-port instance, after using consistent hash, the data cannot be automatically moved to the new node
when the calculated value changes when the number of redis nodes changes . - Codis, currently the most used cluster solution, basically has the same effect as twemproxy, but it supports that when the number of nodes
changes, the old node data can be restored to the new hash node. - The cluster that comes with redis cluster3.0 is characterized in that its distributed algorithm is not a consistent hash, but the concept of hash slots
, and it supports nodes to set slave nodes. See the official documentation for details. - Implemented at the business code layer, start several unrelated redis instances, and at the code layer, perform hash calculation on the key, and then
go to the corresponding redis instance to operate the data. This method has relatively high requirements for the hash layer code, and considerations include
alternative algorithm solutions after node failures, automatic script recovery after data shocks, instance monitoring, and so on.
13. Talk about cache penetration, cache breakdown and cache avalanche, and the solutions?
-
Cache penetration
problem: A large number of concurrent queries do not exist in the KEY, which does not exist in the cache and database, and puts pressure on the cache and database at the same time.
Reason: Generally speaking, there are two possibilities for cache penetration: business data is deleted by mistake, resulting in no data in the cache and database. Malicious ddos attack.
Analysis: Why is it transparently transmitted multiple times? It is always empty if it does not exist. It is necessary to pay attention to let the cache distinguish between the absence of the KEY and the query of a null value.
Solution: Cache the KEY with null value, so that it will be loaded and recorded if it does not exist for the first time, and it will be saved next time (the asynchronous thread is responsible for maintaining the cached data, triggering updates periodically or according to conditions), so that it will not Trigger an update. -
Cache breakdown
problem: When a KEY becomes invalid, there are a large number of concurrent requests to access this KEY.
Analysis: It is actually very similar to penetration, and it is quite accidental.
Solution: The update operation of KEY adds a global mutex. It is completely based on the cache, and the strategy of delaying asynchronous loading is used to transmit the traffic pressure to the database, resulting in excessive pressure on the database or even downtime. -
Cache avalanche
Problem: When a large-scale cache failure occurs at a certain moment, a large number of requests cannot obtain data, so KEY. Bloom filtering or RoaingBitmap judges whether the KEY exists. If the data is not found in the Bloom filter, it will not be checked in the database. Add a malicious request check before processing the request, if a malicious attack is detected, the service will be denied. It is completely based on the cache and uses a delayed asynchronous loading strategy (asynchronous threads are responsible for maintaining cached data, triggering updates periodically or according to conditions), so that updates will not be triggered.
Reason: Generally speaking, there are two possibilities for cache avalanche: A large amount of data fails at the same time: For example, data with strong business relationships requires simultaneous failure Redis downtime
analysis: Generally speaking, due to update strategies, or data hotspots, cache Due to reasons such as service downtime, the cached data may be unavailable on a large scale at the same time, or all of them may be updated. Therefore, our update strategy needs to be appropriate in time, the data should be shared evenly, and multiple cache servers should be highly available.
Solution: The update strategy should be relatively average in time. If the data needs to expire at the same time, you can add some random values to this batch of data, so that this batch of data will not expire at the same time, reducing the pressure on the database. The hot data used should be distributed to different machines as much as possible. Multiple machines act as master-slave replication or multiple copies to achieve high availability. Do a good job in master-slave deployment. When the master node hangs up, the slave node can be quickly used on top. Realize the fuse current limiting mechanism and control the load capacity of the system. For businesses with non-core functions, their requests are rejected, and only core function businesses are allowed to access the database to obtain data. Service price reduction: provide a default return value, or a simple prompt message.
14. The difference between Redis data structure compression list and jump table
- A compressed list (ziplist) is essentially a byte array. It is a linear data structure designed by Redis to save memory. It can contain multiple elements, and each element can be a byte array or an integer.
- A skip list (skiplist) is an ordered data structure, which achieves the purpose of quickly accessing nodes by maintaining multiple pointers to other nodes in each node. The jump table supports node lookup with average O(logN) and worst O(N) complexity, and can also process nodes in batches through sequential operations.
15. What is the maximum capacity that a string type value can store?
Answer: 512M
16. Tell me about the common data structures of Redis? respective usage scenarios?
- String
introduction: The string data structure is a simple key-value type.
Usage scenario: It is generally used in scenarios that need to be counted, such as the number of user visits, the number of likes and reposts of popular articles, and so on. - List
introduction: list is a linked list
Usage scenario: publish and subscribe or message queue, slow query. - Hash
introduction: hash is similar to HashMap before JDK1.8, and the internal implementation is similar (array + linked list).
Usage scenario: storage of object data in the system. - Set
introduction: set is similar to HashSet in Java. The set type in Redis is an unordered collection, and the elements in the collection have no order.
Set is a good choice when you need to store a list of data and do not want duplicate data, and set provides an important interface for judging whether a member is in a set collection, which list cannot provide.
The operations of intersection, union, and difference can be easily realized based on set.
Scenarios: The data that needs to be stored cannot be repeated, and the intersection and union of multiple data sources need to be obtained. - Introduction to sorted set
: Compared with set, sorted set adds a weight parameter score, so that the elements in the set can be arranged in order according to score, and the list of elements can also be obtained through the range of score. It's a bit like a combination of HashMap and TreeSet in Java.
Usage scenario: A scenario where data needs to be sorted according to a certain weight. For example, in the live broadcast system, the real-time ranking information includes the list of online users in the live broadcast room, various gift rankings, barrage messages (which can be understood as message rankings by message dimension) and other information. - Bitmap
introduction: Bitmap stores continuous binary numbers (0 and 1). Through bitmap, only one bit is needed to represent the value or state corresponding to an element, and the key is the corresponding element itself. We know that 8 bits can form a byte, so the bitmap itself will greatly save storage space. .
Usage scenario: It is suitable for scenarios that need to save status information (such as whether to sign in, whether to log in...) and further analyze this information. For example, user check-in status, active user status, and user behavior statistics (such as whether you have liked a certain video).
17. Redis common performance problems and solutions:
- It is best not to write a memory snapshot for the master. If the master writes a memory snapshot, the save command schedules the rdbSave function, which will block the work of the main thread. When the snapshot is relatively large, the performance impact will be very large, and the service will be suspended intermittently
- If the data is more important, a Slave turns on AOF backup data, and the policy is set to synchronize once per second
- For the speed of master-slave replication and the stability of the connection, it is best for Master and Slave to be in the same LAN
- Try to avoid adding slave libraries on the main library under high pressure
- Do not use a graph structure for master-slave replication. It is more stable to use a one-way linked list structure, namely: Master <- Slave1<-Slave2 <- Slave3... This structure is convenient to solve the single point of failure problem and realize the replacement of Slave to Master. If the Master hangs up, you can immediately enable Slave1 to be the Master, and the others remain unchanged.
18. Why does Redis need to put all data in memory?
Answer: In order to achieve the fastest read and write speed, Redis reads all data into memory and writes data to disk asynchronously.
So redis has the characteristics of fast and data persistence. If the data is not placed in memory, the disk I/O speed will seriously affect the performance of redis. Today, when memory is getting cheaper and cheaper, redis will become more and more popular. If the maximum memory used is set, new values cannot be inserted after the number of existing data records reaches the memory limit.
19. Do you understand the synchronization mechanism of Redis?
Answer: Redis can use master-slave synchronization and slave-slave synchronization.
During the first synchronization, the master node performs a bgsave, and records the subsequent modification operations to the memory buffer at the same time. After the completion, the rdb file is fully synchronized to the replication node. After the replication node accepts it, the rdb image is loaded into the memory.
After the loading is complete, notify the master node to synchronize the modified operation records to the copy node for replay to complete the synchronization process.
20. What are the benefits of Pipeline, why use pipeline?
Answer: The time for multiple IO round trips can be reduced to one, provided that there is no causal correlation between the instructions executed by the pipeline. When using redis-benchmark for pressure testing, it can be found that an important factor affecting the QPS peak value of redis is the number of pipeline batch instructions.
21. Have you ever used a Redis cluster? What is the principle of the cluster?
- Redis Sentinal focuses on high availability. When the master is down, it will automatically promote the slave to master and continue to provide services.
- Redis Cluster focuses on scalability, and uses Cluster for shard storage when a single redis memory is insufficient.
22. What are the Java clients supported by Redis? Which one is officially recommended?
Answer: Redisson, Jedis, lettuce, etc. are officially recommended to use Redisson.
23. What are the advantages and disadvantages of Jedis and Redisson?
Answer: Jedis is the client of Redis implemented in Java, and its API provides comprehensive support for Redis commands;
Redisson implements a distributed and scalable Java data structure. Compared with Jedis, its functions are simpler and it does not support strings. Operations do not support Redis features such as sorting, transactions, pipelines, and partitions.
The purpose of Redisson is to promote the separation of users' concerns about Redis, so that users can focus more on processing business logic.
24. How to set password and verify password in Redis?
设置密码：config set requirepass 123456
授权密码：auth 123456
25. Tell me about the concept of Redis hash slot?
Answer: Redis cluster does not use consistent hash, but introduces the concept of hash slots. Redis cluster has 16384 hash slots. A node is responsible for a part of the hash slots.
26. What is the master-slave replication model of Redis cluster?
Answer: In order to make the cluster still available when some nodes fail or most nodes cannot communicate, the cluster uses a master-slave replication model, and each node will have N-1 replicas.
27. Will the write operation be lost in the Redis cluster? Why?
Answer: Redis cannot guarantee the strong consistency of data, which means that in practice, the cluster may lose write operations under certain conditions.
28. How are Redis clusters replicated?
Answer: Asynchronous replication
29. What is the maximum number of nodes in a Redis cluster?
Answer: 16384.
30. How does the Redis cluster choose a database?
Answer: Redis cluster currently cannot select databases, and the default is 0 databases.
31. What are the commands related to Redis transactions?
答：MULTI、EXEC、DISCARD、WATCH
32. How to set the expiration time and permanent validity of Redis key?
Answer: EXPIRE and PERSIST commands.
33. Have you ever used Redis distributed locks? What is it about?
Answer: Use setnx to grab the lock first, and then use expire to add an expiration time to the lock to prevent the lock from being forgotten to be released. When the process crashes unexpectedly after setnx and before expire, or needs to be restarted for maintenance, the set command has very complicated parameters, and setnx and expire can be combined into one command to use at the same time!
34. What is the most suitable scenario for Redis?
-
One of the most commonly used scenarios for using Redis is session cache .
The advantage of using Redis to cache sessions over other storage (such as Memcached) is that Redis provides persistence. When maintaining a cache that is not strictly consistent, most people would be unhappy if all of the user's shopping cart information was lost. Now, would they? Fortunately, as Redis has improved over the years, it's easy to find documentation on how to properly use Redis for session caching. Even the well-known business platform Magento offers a plugin for Redis. -
Full Page Cache (FPC)
In addition to the basic session token, Redis also provides a very simple FPC platform. Going back to the issue of consistency, even if the Redis instance is restarted, users will not see a drop in page loading speed due to disk persistence. This is a great improvement, similar to PHP's local FPC. Taking Magento as an example again, Magento provides a plugin to use Redis as a full page cache backend. In addition, for WordPress users, Pantheon has a very good plugin wp-redis, which can help you load the pages you have visited as fast as possible. -
Queue
One of the advantages of Reids in the field of memory storage engines is that it provides list and set operations, which makes Redis a good message queuing platform to use. The operation that Redis uses as a queue is similar to the push/pop operation of a local programming language (such as Python) on a list. If you do a quick Google search for "Redis queues", you'll immediately find a ton of open source projects that aim to use Redis to create really nice backend tools for a variety of queuing needs. For example, Celery has a background that uses Redis as a broker, you can check it from here. -
Leaderboards/Counters
Redis does a great job of incrementing or decrementing numbers in memory. Sets and Sorted Sets also make it very simple for us to perform these operations, and Redis just provides these two data structures. So, we want to get the top 10 users from the sorted set - let's call it "user_scores", we just need to do it like this: Of course, this assumes that you are doing it based on your users' scores Incremental sort. If you want to return the user and the user's score, you need to execute it like this: ZRANGE user_scores 0 10 WITHSCORES Agora Games is a good example, implemented in Ruby, its leaderboard uses Redis to store data, you can find it here See. -
Publish/Subscribe
Last (but certainly not least) is Redis' publish/subscribe functionality. There are indeed many use cases for publish/subscribe. I've seen people use it in social networking connections, as triggers for pub/sub based scripts, and even use Redis' pub/sub functionality to build chat systems!
