.mysql and redis database types
MySQL is a relational database, mainly used to store persistent data. It stores data on the hard disk and has a slow reading speed.
Redis is a NOSQL, which is a non-relational database and a cache database. It stores data in the cache. The cache has fast reading speed and can greatly improve operating efficiency, but the storage time is limited.
2.Mysql operating mechanism
As a relational database with persistent storage, the relative weakness of mysql is that every time a request is made to access the database, there are I/O operations. If the database is accessed repeatedly and frequently. First: it will take a lot of time to repeatedly connect to the database, resulting in too slow operating efficiency; second: repeated access to the database will also cause the database load to be too high, so the concept of caching is derived at this time.
3. Caching
The cache is the buffer (cache) for data exchange. When the browser performs a request, it will first search the cache. If it exists, it will be obtained; otherwise, the database will be accessed.
The advantage of caching is that it reads quickly
4.redis database
The redis database is a cache database used to store frequently used data, thus reducing the number of database accesses and improving operating efficiency.
5. Summary of the differences between redis and mysql
(1) Type
In terms of type, mysql is a relational database and redis is a cache database.
(2) Functionally
mysql is used to store data persistently on the hard disk. It is powerful and slow. It is based on disk. The read and write speed is not as fast as Redis, but it is not limited by space capacity and is cost-effective.
Redis is used to store frequently used data in the cache. It has fast reading speed. It is based on memory and has fast reading and writing speed. It can also be persisted. However, the memory space is limited. When the amount of data exceeds the memory space, the memory needs to be expanded. However, Memory is expensive
(3) On demand
Mysql and redis are generally used together due to different needs.
Use Redis where high performance is required, and use MySQL where high performance is not required. Stored data is synchronized between MySQL and Redis.
Redis persistence
Since Redis data is stored in memory, if persistence is not configured, all data will be lost after redis restarts. Therefore, it is necessary to enable the persistence function of redis and save the data to the disk. When redis restarts, it can be retrieved from the disk. Data recovery. Redis provides two methods for persistence, one is RDB persistence (the principle is to periodically dump Reids database records in memory to RDB persistence on disk), and the other is AOF (append only file) persistence ( The principle is to write Reids' operation log to the file in an appending manner).
RDB
RDB persistence refers to writing a snapshot of the data set in memory to disk within a specified time interval. The actual operation process is to fork a child process, first write the data set to a temporary file, and then replace the previous file after the writing is successful. , stored using binary compression.
advantage
Stored files are compact
Suitable for backup and easy to restore different versions of data
Suitable for disaster recovery, backup files can be restored on other servers
Maximizes the performance of Redis. When backing up, the child thread is started, and the parent process does not need to perform IO operations.
Data saving is faster than AOF
shortcoming
If Redis stops working because it is not shut down properly, the data between the last save point will be lost.
Since it is necessary to frequently fork sub-threads to perform backup operations, if the amount of data is large, fork is time-consuming. If the CPU performance is not enough, the server may be stuck. When the amount of data is large, do not deploy multiple Redis services on one server.
There are five ways to create a snapshot:
1. The client sends the BGSAVE command, and the server will fork a sub-thread to write the snapshot to the disk.
2. The client sends the SAVE command, and the server performs the writing action on the main thread. Generally not used often, usually used when there is not enough memory to execute BGSVAE.
3. If the SAVE configuration item is set, such as SAVE 300 100, then when "there are 100 writes within 300 seconds", Redus will automatically trigger the BGSAVE command. If there are multiple configuration items and any one of them is satisfied, a backup will be triggered.
4. When Redis receives a request to shut down the server or a TERM signal through the SHUTDOWN command, it will execute the SAVE command. At this time, all clients will be blocked and no commands sent by the clients will be executed.
5. When a Redis server connects to another Redis server and sends a SYNC command to the other party to start a replication operation, if the main server is not currently executing the BGSAVE operation, or the main server has just completed the execution, then the main server will execute the GBSAVE operation.
AOF
AOF persistence records every write and delete operation processed by the server in the form of a log. Query operations are not recorded, but are recorded in text. You can open the file to see detailed operation records.
AOF records all write operations of the server. When the server restarts, all write operations will be performed again to achieve data backup. When the write operation set is too large (larger than the original data set), Redis will rewrite the write operation set.
advantage
Using AOF mode is more flexible because different fsync strategies can be used
AOF is a log append file. It does not require positioning. Even if the power is cut off, there will be no damage problem. Even if there is a half-written command at the end of the file, the redus-check-aof tool can easily repair it.
When the AOF file is large, Redis will automatically rewrite it in the background. Overwriting is absolutely safe, because Redis continues to append to the old file, using the minimum set of operations required to create the current data set to create a brand new file. Once the creation is completed, Redis will switch to the new file and start Append to new file
AOF contains one operation command after another, which is easy to understand and parse.
shortcoming
For the same data set, AOF files are usually larger than RDB files.
AOF may be slower than RDB, depending on the fsync strategy. Usually fsync is set to once per second and the performance is still very high. If sfync is turned off, it is as fast as RDB even under high load. However, RDB can still provide good maximum latency guarantees even under heavy write loads.
AOF updates data incrementally, while RDB snapshots are created from scratch. RDB will be more robust and stable (so suitable for backup)
Redis persistence strategy
RDB (data snapshot mode), stored regularly, saves the data itself, and the storage files are compact
AOF (append mode), every time the data is modified, it is synchronized to the hard disk (write operation log), and the data change record is saved.
If you only want the data to be kept in memory, both strategies can be turned off.
You can also enable both strategies at the same time. When Redis restarts, the AOF file will be used to reconstruct the original data.
Redis implements scheduled tasks
Publish / Subscribe
Redis launched the Pub/Sub command after 2.0.0, which basically means that one side sends a message to a specific channel of Redis, and the other side gets the value from the specific channel of Redis - forming a simple message queue.
Redis Keyspace Notifications
There are some events in Redis, such as key expiration, key deletion, etc. Then we can configure something to let Redis push a message to a specific Channel once these events are triggered. Note: Redis's publish/subscribe elapsed time does not support persistence, so if the client disconnects and reconnects, the messages during this period will be lost!
Redis distributed lock
First of all, in order to ensure that the distributed lock is available, we must at least ensure that the lock implementation meets the following four conditions at the same time:
Mutual exclusivity. At any time, only one client can hold the lock.
No deadlock will occur. Even if a client crashes while holding the lock without actively unlocking it, it is guaranteed that other clients can subsequently lock it.
Be fault tolerant. As long as most Redis nodes are running normally, the client can lock and unlock.
The trouble should end it. The locking and unlocking must be done by the same client. The client itself cannot unlock the lock added by others. The Redis distributed lock process is as follows:
Several key points to note when implementing:
1. The lock information must expire and time out, and a thread cannot be allowed to occupy a lock for a long time and cause deadlock;
2. Only one thread can obtain the lock at the same time.
Write more about Redis
illustrate:
The component is called YeeRedisGroup. There are four main basic services. When data arrives, two Redis services will be inserted respectively. These two Redis services use a remote active-active solution. When one of the Redis services hangs up, it will Remove this Redis service from the available queue and put it in the retry queue, and the other Redis will continue to be used. Similarly, when reading Redis, only the first Redis service will be read from the available queue to continue reading.
Redis timeout deletion
· Scheduled deletion: While setting the expiration time of the key, create a timer (timer) so that the timer can immediately perform the deletion operation on the key when the expiration time of the key comes.
·Lazy deletion: Let the key expire regardless, but every time you get the key from the key space, check whether the obtained key has expired. If it has expired, delete the key; if it has not expired, return the key.
·Regular deletion: Every once in a while, the program checks the database and deletes the expired keys in it. It's up to the algorithm to decide how many expired keys to delete and how many databases to check.
Among these three strategies, the first and third are active deletion strategies, while the second is a passive deletion strategy.
Expiration strategy adopted by Redis: lazy deletion + regular deletion
Lazy deletion process
When performing operations such as get or setnx, first check whether the key has expired.
If it expires, delete the key and then perform the corresponding operations;
If it has not expired, perform the corresponding operation directly.
Periodic deletion process (simply put, randomly delete less than or equal to the specified number of expired keys from each of the specified number of libraries)
Traverse each database (that is, the number of "databases" configured in redis.conf, the default is 16)
Check the specified number of keys in the current library (the default is to check 20 keys for each library. Note that it is equivalent to executing the loop 20 times. The loop body is described below)
If no key in the current library has an expiration time set, directly execute the traversal of the next library.
Randomly obtain a key with an expiration time set, check whether the key has expired, and if it has expired, delete the key
Determine whether the periodic deletion operation has reached the specified time. If it has reached the specified time, exit the periodic deletion directly.
RDB processing of expired keys
Expired keys have no impact on RDB
Persist data from in-memory database to RDB file
Before persisting the key, it will check whether it has expired. Expired keys will not be entered into the RDB file.
Recover data from RDB file to in-memory database
Before data is loaded into the database, the key will be checked for expiration. If it expires, it will not be imported into the database (in the case of the main database)
AOF's handling of expired keys
Expired keys have no impact on AOF
Persist data from in-memory database to AOF file:
When the key expires and has not been deleted, the persistence operation is performed at this time (the key will not enter the aof file because no modification command has occurred)
When the key expires and a deletion operation occurs, the program will append a del command to the aof file (the expired key will be deleted when the aof file is used to restore data in the future)