Redis Series - Part IV persistence and transactions

One, endurance of

Redis is a memory database, in order to ensure data persistence, persistence it provides two options:

• RDB mode (default)

• AOF way

Persistence function effectively avoid data loss caused by the process of withdrawal problems persisted when the next restart before the file can be realized using the number of
data recovery.

1.RDB

1.1 Introduction

• RDB is Redis default persistence methods used.

• RDB is through snapshots (snapshotting) completed, when certain conditions are met Redis will automatically data in memory when the snapshot and persisted to the hard disk.

• Redis will be in specified circumstances trigger a snapshot at (divided into manual and automatic trigger trigger)

  • 1. in line with custom configuration snapshots rule (automatic, rules that are configured in redis.conf)
  • 2. The execution of the save command or bgsave (manual)
  • 3. execute flushall command (manual)
  • 4. If the total amount of the copy operation performed from the node, the master node performs bgsave automatically generated and sent from node RDB (Auto)
  • 5. When the shutdown command, if there is no persistence opening AOF bgsave function is automatically performed by default (automatic)
  • 6 When you execute debug reload command to reload Redis, it will automatically trigger the save operation (automatic)

save command: Redis server is currently blocked until the process is complete RDB, for example relatively large memory for a long time can cause obstruction, online environment is not recommended.

bgsave command: Redis process of implementation fork operation to create a child process, RDB lasting process responsible for the child, after the completion of automatic> End. Blocking occurs only in the fork phase, generally a very short time.

In redis.conf set a custom rule Snapshot

1. RDB endurance conditions

Format: the Save

Example:

save 900 1: represents a bond of at least one snapshot has been changed is performed for 15 minutes (900 seconds).

save 300 10: at least 10 key is represented by the snapshot changed within 5 minutes (300 seconds).

save 60 10000: 10000 represents at least the snapshot button is changed within one minute.

​

You can configure the plurality of conditions (conditions a per line configuration), the relationship between each condition "or."

2. Configure dir specify the location of the snapshot file rdb

dir   ./   

3. Configure the name dbfilename specify the snapshot file rdb

dump.rdb

Special Note:

• After the RDB Redis starts, it reads the snapshot file, the data is loaded from the hard disk into memory.

• The amount of data with different structures and server performance, this time is different. Typically ten million records key string type, a size of 1GB snapshot file is loaded into memory takes 20 to 30 seconds.

The principle 1.2 snapshots

• Snapshot process

1. redis using the fork function copy of a current process copy (the child)
2. The parent process continues to receive and process commands sent by the client, and the child began to write the data in memory of the temporary files on your hard disk.
3. When the child process all the data written to the RDB will replace the old file with the temporary file, so far, a snapshot operation is complete.

• Precautions

1.redis making process snapshot will not modify RDB file , only after the end of the snapshot will replace the old file into the new, that is to say at any time RDB files are complete.

2. This allows us to redis database backup of files through a scheduled backup RDB, RDB file is a compressed binary file , space is often less than the data in memory, more conducive to transmission.

1.3RDB advantages and disadvantages

RDB disadvantages:

• RDB way no way to do real-time data persistence / second-level persistence. Bgsave have to perform the operation because the fork to create a child process, belongs to the heavyweights operations, frequently performed high cost of each run.
• RDB file saved using a specific binary format, Redis version of evolution have multiple versions of RDB format, there is not compatible with older versions of the new version of Redis service RDB format problems. RDB is not suitable for real-time persistent problem, Redis provides a way to solve the AOF persistence.

RDB advantages:

• RDB is a compact compressed binary file, on behalf of Redis data snapshot at a certain point in time. Ideal for backup, the whole amount of replication scenarios. For example bgsave backup performed every 6 hours, and copy the file to the remote machine or RDB file system (e.g. HDFS), for disaster recovery.
• Redis load RDB to recover data much faster than AOF way.

2.AOF

2.1 Introduction

• Redis is not turned on by default persistence AOF (append only file) mode

• After opening AOF each perform a persistence will change the data in Redis commands , Redis command will write the AOF file on your hard disk, this process obviously reduces the performance of Redis , but in most cases this influence is capable of acceptable, so further with faster disk can improve the performance of AOF .

• Redis.conf you can modify the configuration file appendonly open parameters

appendonly yes

• AOF save location and file location of the file RDB same, are set by the parameter dir

dir ./

• The default file name is appendonly.aof, can be modified by appendfilename parameters:

appendfilename appendonly.aof

2.2 Principle rewrite

• AOF Redis may become too large file size , the AOF of the background automatically rewrites

• AOF rewrite new file contains the current data set required for recovery of a minimum set of commands .

• Rewrite the entire operation is absolutely safe because Redis creating a new file AOF process, will continue to command appended to the existing file AOF inside, even during downtime rewrite existing AOF files will not be lost . Once the new AOF file is created, Redis will be from AOF file to switch to the new AOF file ** and begin a new AOF file is additionally operate.

• AOF file rewritten Why can become smaller:

  • In-process has timed the data is no longer written to the file.

  • The old AOF file contains an invalid command

  • Multiple write commands can be combined into a

• AOF rewrite process can be triggered manually and automatically triggers:

  • Manual trigger: a direct call bgrewriteaof command.

• Auto: The auto-aof-rewrite-min- size and auto-aof-rewrite-percentage parameters
  determined automatically trigger the timing number.

Parameter Description

auto-aof-rewrite-percentage 100 represents more than once on how many percent of the aof file size will be rewritten when the current aof file size. If you do not rewritten before, when to start aof file size prevail

auto-aof-rewrite-min- size 64mb restriction allows rewriting aof minimum file size, the file size is less than 64mb time, you do not need to be optimized

2.3 synchronous disk data

Redis data each time you change the time, the mechanism will command aof aof record to file, but is actually due to the operating system's caching mechanism , data is not written to the hard disk in real time , but into the hard disk cache. Then flushed to go through the hard disk saved to a file caching mechanism.

Parameter Description:

Configuration values	Explanation
always	Each time a write will be synchronized, but this is the safest way is relatively low efficiency
everysec	Every second execution
no	Do not take the initiative to synchronous operation, to be performed by the operating system, this is the fastest but least secure way

2.4 restart load

2.5 AOF file how to repair after damage

The server may shut down when the program is being AOF file for writing, if downtime caused AOF file error (corrupt), then restart Redis in AOF will refuse to load this file, ensuring consistency of the data will not be destroyed.

When this happens, you can use the following methods to repair the mistakes of AOF documents:

• AOF create a file for an existing backup .

• Redis using the supplied redis-check-aof program, AOF file to the original repair. redis-check-aof --fix readonly.aof

• Redis server restart, wait AOF file server load repair and data recovery.

How to choose RDB and 2.6 AOF

• In general, if the data security requirements are very high, you should use both persistent feature.
• If you can afford the loss of data within a few minutes, you can only use RDB persistence.
• AOF many users use only persistent, but does not recommend this approach: since the timing generating RDB Snapshot (Snapshot) is very easy database backup and recovery speed RDB data set than AOF is also faster recovery.
• Two kinds of persistence strategies can be used simultaneously, you can use one of them. If you use it, when you restart Redis, it will give priority to the use of AOF file to restore data

Second, the transaction

1.Redis Affairs introduced

In order to ensure atomicity combination of multiple commands, Redis provides a simple transactional capabilities and integrated Lua script to solve this problem. Transaction represents a set of actions, either all executed or not executed all.

• Redis transaction by MULTI, EXEC, DISCARD and WATCH these four commands to complete.
• Redis single command is atomic , so here is to ensure transactional objects set of commands .
• Redis commands and ensure that the set of sequences in the same transaction command set of continuous and interrupted execution
• Redis does not support rollback

2. Basic Commands

• MULTI

  For starting a transaction tag block .

  Redis subsequent commands will be placed in the queue one by one, and then atomically executing the command sequence the EXEC command.

  Syntax: multi

127.0.0.1:6379> MULTI 
OK
127.0.0.1:6379> set s1 11
QUEUED
127.0.0.1:6379> set s2 s2
QUEUED

You can see the set command returns the result in this case is QUEUED, on behalf of the command does not really perform, but temporarily stored in Redis execute the following command if results are as follows:

127.0.0.1:6379> EXISTS s1
(integer) 0
127.0.0.1:6379> get s1
(nil)

Only when the implementation of the exec, will really add to the redis.

• EXEC

  All the commands in the queue in a previous transaction , and then return to normal connection state

  Syntax: exec

127.0.0.1:6379> MULTI 
OK
127.0.0.1:6379> set s1 11
QUEUED
127.0.0.1:6379> set s2 s2
QUEUED
127.0.0.1:6379> EXEC
1) OK
2) OK

Set command because the execution of the two, so the two return exec execute data.

• DISCARD

  Clear all the commands in the queue in a previous transaction , and then return to normal connection status.

  Syntax: discard

127.0.0.1:6379> MULTI 
OK
127.0.0.1:6379> set s3 333
QUEUED
127.0.0.1:6379> set s4 444
QUEUED
127.0.0.1:6379> DISCARD
OK

3. Transaction processing failed

3.1 Redis syntax error (can be understood as a compile-time error)

127.0.0.1:6379> MULTI
OK
127.0.0.1:6379> set s3 333
QUEUED
127.0.0.1:6379> set s4
(error) ERR wrong number of arguments for 'set' command
127.0.0.1:6379> EXEC
(error) EXECABORT Transaction discarded because of previous errors.

Perform a little above the set command value, resulting in a transaction can not be committed.

3.2 Redis type of error (can be understood as runtime errors)

This command is run, because the syntax is correct.

127.0.0.1:6379> MULTI
OK
127.0.0.1:6379> set s3 333
QUEUED
127.0.0.1:6379> LPUSH s3 444 555
QUEUED
127.0.0.1:6379> EXEC
1) OK
2) (error) WRONGTYPE Operation against a key holding the wrong kind of value

As can be seen from the above, Redis transaction rollback is not supported . exec execute and return an OK, an exception. Description set s3 333 has performed successful.

127.0.0.1:6379> get s3
"333"

4. Monitoring command

• WATCH

  When a transaction requires a conditional execution time, it is necessary to use this command given keys to the monitored state.

  Syntax: watch key [key ...]

  Note: Use this command to achieve the redis optimistic locking .

• UNWATCH
  clear all previous transaction monitoring as a key.
  Syntax: unwatch

Some applications require the scene before the transaction, the transaction is key to ensure that no other client is modified, the transaction was executed, and otherwise does (similar optimistic locking). Redis provides the watch command to resolve such problems.

Client 1	Client 2

After the client 1 after executing MULTI command APPED s5 world order.

The client then 2, in the implementation of APPED s5 java command.

Then the client 1, in the implementation of APPED s5 vue command.

Then the client 1, in the EXEC command.

Because the client value 1 watch has changed, resulting in a transaction not commit, returns nil.

Redis provides a simple transaction, the reason for it is simple, mainly because it does not roll back the transaction characteristics support

Micro-channel public number

JAVA program ape growth path
to share resources, loggers ape growing little by little. Focus on Java, Spring, SpringBoot, SpringCloud, distributed, slightly services.