Original link:
http://www.cnblogs.com/xiang9286/p/11056654.html
Pictures from "Redis Design and Implementation"
Redis database
redis.server redisServer.dbnum for the generation number database initialization, default
16
months,
the Select
command to select the database
redis.client in redis.db pointer to the database currently in use
First, the expiration policy
Set expiration time commands
expire
as expire <key> <ttl>
redisDb mainly by the dictionary dict and expires two components:
dict dictionary space is a bond, all of the keys stored in the database objects; expires expiration parameter called the dictionary, each key corresponding to the stored time expires
expires key is a pointer to the key of a key in the object space; long long integer value type, storage expiration time (ms UNIX timestamp (Chuo))
Two steps to determine whether the key expired :
(1) Check whether a given key dictionary is present in the expired; is present, the corresponding time stamp is obtained
(2) to get the current UNIX timestamp, compare the two, you can know whether expired
Delete expired three key strategies:
1.
regularly delete
: Use a timer (time-based events, disorderly list, complexity O (N), can not efficiently handle a lot of time event), regularly check whether the key expiration policy for friendly memory
2.
inert Delete
: use key to check if the key has expired, for CPU-friendly policies
3.
regularly delete
: compromise strategy, random inspection expiration time part of the key and delete outdated key difficulty is to determine the duration and frequency of the deletion execution
Redis in inert delete and delete regularly used in conjunction with
Since both expired still key to delete the policy could lead to a large number of expired keys accumulate in memory, for memory Redis elimination mechanism introduced (out of a total of six kinds of strategies. The most important thing is allkeys-lru)
Expired keys to RDB, AOF persistence and replication of influence
RDB: RDB when persistent, it checks whether the key expired; when RDB is loaded, the server checks the master key has expired, omitted from the server, waiting for the main server synchronization processing
AOF: When AOF file written ignored until the expiration key is deleted, it appends a command to delete expired key file AOF
AOF when overwriting files, it checks whether the key expired
Copy: each primary server delete an expired key, to display all of the DEL command is sent from a server, the notification is removed from the server to the key expired
(This is actually copied from the master synchronization process?)
Second, endurance of
Redis persistence divided into two types: RDB persistence and persistence AOF
RDB persistence:
A compressed binary file through; key-value pairs are stored in a database of information, different keys for different ways to save
The difference between the two commands: the SAVE, bgsave
save:
blocking
RDB persistence. RDB server thread performs persistence operations, the command can not process any requests during the
bgsave:
non-blocking
RDB persistence.
Creates a child process for executing RDB persistence operations
, server thread is still able to handle the request command
RDB file to load
No command is performed automatically when the server starts; if the server is turned AOF persistence feature, will give priority to the use of AOF file to restore the database state
RDB automatic intermittent saved
Can configure the server save option, every so often automatically execute a command bgsave
The default meet one of three conditions is automatically executed once bgsave command. Condition stored in the parameter saveparams
In addition to saveparams array, also stores
dirty counter
and
LASTSAVE attributes
; dirty for calculating the number of accumulation operation, and for LASTSAVE current timestamp, and calculates elapsed time
Three conditions:
9001; 30010; 6,010,000
Redis periodic function serverCron default operation is performed once every 100ms intervals for maintenance of the server is running, including checking parameter condition is satisfied
AOF persistence:
To save the database information by saving Redis commands
Redis command to save the request protocol format, Append Only File
Three Steps to AOF persistence:
(1) command is added: each execute a command, the command will be added to the
aof_buf buffer
end (64 size constant value)
(2) command to write: a server before the end of each event loop will call flushAppendOnlyFile function, consider whether to buffer the data file is written AOF
(3) command synchronization: with flushAppendOnlyFile function
appendfsync parameter
decision
Always
: write buffer and synchronize files to AOF
everysec
: sync once per second
NO
: write only decided by the operating system synchronization
AOF load reduction:
(1) create a pseudo client without network connection (for execution AOF file Redis command as command can be executed in the client)
(2) analysis AOF file and a command read
(3) Run the pseudo client
(4) performing cyclic 2,3; AOF file until the command is complete
AOF rewrite files : bgrewriteaof
Why: To solve the problem of expansion and growth over time operations, AOF file, the file command redundant processing AOF
Realized concept: instead of reading the original AOF file content, but directly read the current state of the database, and then use one (or more, as little as possible) command to record key-value pairs
Implementation process:
1. Create a
child process to rewrite the background
at --- avoid the use of locks, to ensure data security; and do not block the server process
2. Use the AOF rewrite buffer, each server execute a command, the command will be simultaneously added to the AOF AOF rewrite buffer and buffer
3. After the rewrite is completed, the child process returns a signal, the parent calls the signal handler
(1) The AOF rewrite the contents of the buffer to write new AOF file, the new file to the database state is consistent AOF
(2) the new file renamed AOF, atomically overwrite the existing file AOF, completed the old and new files are alternately AOF
Note: Call the signal handler phase, the server process blocked. (
Blocking the process of rewriting on this point
)
Compare RDB and AOF two kinds of persistence methods
(1) RDB file, quick recovery; but can not save data after the last snapshot that will lose this part of the data
(2) AOF high readability of documents, data are not readily lost; large file size, however, a long recovery time
Redis 4.0 RDB and AOF began to support mixed persistence
Third, the event
Redis server is an event-driven program
Single-threaded
operation,
the I / O multiplexing
to monitor multiple sockets; two events (file events and time of the event)
File event
: a socket abstraction operate; there is an event readable, writable events; file event file event has a processor based on the Reactor pattern.
Event Processor File: Socket + I / O multiplexing program event dispatcher + + file event handler
Common Event Processor: connect an answering processor, the command processor request, reply command processor. (In order to connect command is received, processing command reply)
Time events : a given abstract operation time execution class ; the timed event, a periodic event
Time Event achieve: Redis all time events are placed in an unordered list (not sorted by size when property), look through the entire list by the time the event has arrived, and calls the appropriate event handler.
Time Event: three parameters (id, when, timeProc); typical events: serverCron
Event scheduling and execution
def aeProcessEvents(): # Acquisition time event arrival time from the current time closest time_event = aeSearchNearestTimer() # Calculate the distance to the nearest time event, how many milliseconds remaind_ms = time_event.when - unix_ts_now() # If the event has been reached, then the value of remaind_ms may be negative, it is set to 0 IF remaind_ms < 0 : remaind_ms = 0 # Remaind_ms depending on the value of creating timeval timeval = create_timeval_with_ms(remaind_ms) # Obstruction and wait for the file to produce the event, the maximum blocking time from the incoming timeval decision aeApiPoll(timeval) # Handle all file events have generated procesFileEvents() # Deal with all the time the event has arrived processTimeEvents()
Reproduced in: https: //www.cnblogs.com/xiang9286/p/11056654.html