Detailed Redis optimized configuration

1. Redis configuration strategy

1.1 Redis configuration file

• Configuration parameters (/etc/redis/6379.conf)

bind：监听的主机地址
port：端口（默认6379）
daemonize yes：启用守护进程
pidfile：指定PID文件
loglevel notice：日志级别
logfile：指定日志文件

1.2 Redis database commonly used commands

• redis-cli command line tool

连接本地数据库
/usr/local/redis/bin/redis-cli
127.0.0.1:6379>

连接远程数据库
redis-cli -h 192.168.140.10 -p 6379
 192.168.140.10>

1.3 Redis command tool

1.3.1 Introduction to Command Tool

• redis-server: a tool for starting Redis
• redis-benchmark: Used to detect the operating efficiency of Redis on this machine
• redis-check-aof: Repair AOF persistent files
• redis-check-rdb: Repair RDB persistent files
• redis-cli: Redis command line tool
• redis-sentinel: is a tool to start sentinel mode

1.3.2 redis-benchmark (test tool) command line options

• The basic test syntax is

redis-benchmark [option] [option value]

-h: 指定服务器主机名
-p：指定服务器端口
-s：指定服务器socket
-c：指定并发连接数
-n：指定请求数
-d：以字节的形式指定SET/GET值的数据大小
-k：1=keep alive  0=reconnect
-r：SET/GET/INCR使用随机key，SADD使用随机值
-P：通过管道传输<numreq>请求
-q：强制退出redis，仅显示query/sec值
---csv：以CSV格式输出
-l：生成循环，永久执行测试
-t：仅运行以逗号分隔的测试命令列表
-I：Idle模式，仅打开N个idle 连接并等待

• Application example

[root@server ~]# redis-benchmark -h 192.168.140.20 -p 6379 -c 100 -n 10000
//向IP地址为192.168.140.20，端口为6379的Redis服务器发送100个并发连接与10000个请求测试性能

2. Redis multi-database operation

2.1 Database analysis

• Redis supports multiple databases, 16 databases are supported by default, named 0-15;

• Multiple databases are independent of each other and do not interfere with each other;

• Commonly used commands for
  multiple databases : Switch between
  multiple databases Move data between multiple databases
  Clear data in the database

2.2 Command analysis

2.2.1 Switch between multiple databases

• Use the Select command to switch between multiple Redis databases
• Command format

select index
//其中index 表示数据库的序号，而使用redis-cli 连接Redis数据库后，默认使用的是序号为0的数据库

• Application example

127.0.0.1:6379> select 2      //此时表示从数据库0切换到数据库2
OK
127.0.0.1:6379[2]> select 10  //切换到数据库10
OK
127.0.0.1:6379[10]> 

2.2.2 Moving data between multiple databases

• Redis database provides a move command, which can move data from multiple databases
• Basic syntax format of the command

move key dbindex
//其中“key“表示当前数据库的目标键，"dbindex"表示目标数据库的序号

• Application example

127.0.0.1:6379> move q 10     //移动q到数据库10
(integer) 1
127.0.0.1:6379> get q   
(nil)
127.0.0.1:6379> select 10     //切换到数据库10
OK
127.0.0.1:6379[10]> keys *    列出key中的所有关键词
1) "q"

2.2.3 Clear data in the database

• The entire database data deletion of the Redis database is mainly divided into two parts:
  clear the current database data, use the FLUSHDB command to
  clear all database data, use the FLUSHALL command to achieve (generally not)

• Application example

127.0.0.1:6379> select 10    //切换到数据库10
OK
127.0.0.1:6379[10]> keys *   //列出key中的所有关键词
1) "q"
127.0.0.1:6379[10]> get q    //输出q键值
"10"
127.0.0.1:6379[10]> flushdb  //清除当前数据库中的数据
OK
127.0.0.1:6379[10]> keys *   //列出key中的所有关键词
(empty list or set)

2.3 Redis database commonly used commands

2.3.1 key related commands

• keys: Get a list of key values ​​that meet the rule
• exists: Determine whether the key value exists
• del: delete the specified key of the current database
• type: Get the value type corresponding to the key
• rename (overwrite)/renamenx (not overwrite): rename the existing key
• dbsize: View the number of keys in the current database

2.3.2 Application examples of commonly used commands

• Usage example

[root@server1 ~]# redis-cli     //连接数据库

• View beginning with s

192.168.140.20:6379> keys s*                 
1) "set1"
2) "set2"

• Check whether the keyword set exists (1 means it exists, 0 means it does not exist)

192.168.140.20:6379> exists set3
(integer) 0
192.168.140.20:6379> exists set1
(integer) 1

• Delete keywords, the operation is successful output 1

192.168.140.20:6379> del set1
(integer) 1

• Get the value type corresponding to the key

192.168.140.20:6379> set string1 77
OK
192.168.140.20:6379> type string1
string
192.168.140.20:6379> type hash1   //获取hash1对应的vlane值类型
hash

• The rename command renames an existing key

rename命令进行重命名时，无论目标key是否存在都进行重命名，且源key的值会覆盖目标key 的值。
在实际使用过程中，建议先用exists命令查看目标 key是否存在，然后再决定是否执行rename命令，以避免覆盖重要数据
命令格式为 rename 源key 目标key

192.168.140.20:6379> set bw 1           //添加键值
OK
192.168.140.20:6379> set cw 2
OK
192.168.140.20:6379> rename cw bw   //重命名cw覆盖bw键值
OK
192.168.140.20:6379> get bw
"2"

• The renamenx command renames the existing key and checks whether the new name exists

其命令格式与 rename 的命令格式除命令关键字不同外基本相同，renamenx源key目标key。
使用renamenx命令进行重命名时，如果目标key存在则不进行重命名
命令格式：renamenx 源key 目标key

192.168.140.20:6379> set bw 1
OK
192.168.140.20:6379> set cw 2
OK
192.168.140.20:6379> renamenx cw bw		//重命名cw覆盖bw键值，重命名失败
(integer) 0

3. Analysis of Redis configuration file

3.1 Overview of persistence

Overview:

• Redis is running in memory, and the data in the memory is lost when power is off
• In order to be able to reuse Redis data or prevent system failures, you need to
• The data in Redis is written to the disk space, that is, persistence

Persistence classification:

• RDB method: Create a snapshot to obtain a copy of all data in Redis at a certain time
• AOF method: write the executed write command to the end of the file, and record the data changes in a log

3.2 RDB endurance

3.2.1 Overview

• Redis's default persistence method, the default file name is dump.rdb.

3.2.2 Trigger conditions

• In the specified time interval, execute the specified number of write operations (configuration file control)
• Execute save or bgsave (asynchronous) command
• Execute flushall command to clear all data in the database
• Execute the shutdown command to ensure that the server shuts down normally without losing any data

3.2.3 Advantages and disadvantages

• Suitable for large-scale data recovery
• If the business does not require high data integrity and consistency, RDB is a good choice
• Data integrity and consistency are not high
• Occupy memory during backup

3.2.4 Recover data through RDB file

• Copy the dump.rdb file to the bin directory of the redis installation directory and restart the redis service

[root@localhost ~]# vim /etc/redis/6379.conf

#RDB核心规则配置 save <指定时间间隔> <执行指定次数更新操作>，满足条件就将内存中的数据同步到硬盘
中。官方出厂配置默认是 900秒内有1个更改，300秒内有10个更改以及60秒内有10000个更改，则将内存中的
数据快照写入磁盘。
若不想用RDB方案，可以把 save "" 的注释打开，下面三个注释
#save ""
save 900 1					"这三项默认设置"
save 300 10
save 60 10000
dbfilename dump.rdb				//RDB文件名称
dir /var/lib/redis/6379			//RDB文件路径
rdbcompression yes				//是否进行压缩
 
#当RDB持久化出现错误后，是否依然进行继续进行工作，yes：不能进行工作，no：可以继续进行工作，可以通过info中的rdb_last_bgsave_status了解RDB持久化是否有错误
stop-writes-on-bgsave-error yes
 
#配置存储至本地数据库时是否压缩数据，默认为yes。Redis采用LZF压缩方式，但占用了一点CPU的时间。若关闭该选项，但会导致数据库文件变的巨大。建议开启。
rdbcompression yes
 
#是否校验rdb文件;从rdb格式的第五个版本开始，在rdb文件的末尾会带上CRC64的校验和。这跟有利于文件的容错性，但是在保存rdb文件的时候，会有大概10%的性能损耗，所以如果你追求高性能，可以关闭该配置
rdbchecksum yes
 
#指定本地数据库文件名，一般采用默认的dump.rdb
dbfilename dump.rdb
 
#数据目录，数据库会写入这个目录。rdb、aof文件也会写在这个目录
dir /var/lib/redis/6379

3.3 AOF persistence

• Redis is not turned on by default;

• Make up for the shortcomings of RDB (data inconsistency);

• Use a log to record each write operation and append it to the file;

• Redis restart will execute the write command from front to back according to the content of the log file to complete the data recovery work;

• AOF is not turned on by default, it must be turned on manually. RDB is turned on by default.

[root@localhost ~]# vim /etc/redis/6379.conf
 
'//Redis 默认不开启。它的出现是为了弥补RDB的不足（数据的不一致性），所以它采用日志的形式来记录每个写操作，并追加到文件中。
appendonly no
 
appendfilename "appendonly.aof"		//指定本地数据库文件名，默认值为 appendonly.aof
  
#aof持久化策略的配置
#no表示不执行fsync，由操作系统保证数据同步到磁盘，速度最快
	always表示每次写入都执行fsync，以保证数据同步到磁盘
	everysec表示每秒执行一次fsync，可能会导致丢失这1s数据
# appendfsync always		'//always:数据要求高'
appendfsync everysec 	'//everysec:一般情况使用 '
# appendfsync no			

no-appendfsync-on-rewrite no
 
auto-aof-rewrite-percentage 100
 
'//设置允许重写的最小aof文件大小，避免了达到约定百分比但尺寸仍然很小的情况还要重写'
auto-aof-rewrite-min-size 64mb
 
aof-load-truncated yes
 
'//加载redis时，可以识别AOF文件以“redis”开头'
'//字符串并加载带前缀的RDB文件，然后继续加载AOF尾巴'
aof-use-rdb-preamble yes

Detailed configuration commands

• When Redis restarts, it will execute the write instruction from front to back according to the content of the log file to complete the data recovery work. By default, redis uses the RDB method of persistence, which is sufficient in many applications. However, if redis crashes in the middle, it will cause data loss for several minutes. According to the save strategy for persistence, Append Only File is another persistence method that can provide better persistence characteristics. Redis will write the data written every time into appendonly.aof file after receiving. Redis will first read the data of this file into memory every time it starts, and ignore the RDB file first. If rdb is turned on, change no to yes

appendonly no

• When AOF rewrites or writes RDB files, a lot of IO will be performed. At this time, for the AOF mode of Everysec and Always, the execution of fsync will cause blocking for too long, and the no-appendfsync-on-rewrite field is set to the default setting Is no. For applications with high latency requirements, this field can be set to yes, otherwise it is still set to no, which is a safer choice for persistence features. Set to yes to indicate that the new write operation will not be fsynced during rewrite, and it will temporarily be stored in memory, and will be written after the rewrite is completed. The default is no, yes is recommended. The default fsync policy of Linux is 30 seconds. May lose 30 seconds of data

no-appendfsync-on-rewrite no

• Aof automatically rewrites the configuration. When the current AOF file size exceeds the last rewritten AOF file size, rewrite is performed, that is, when the AOF file grows to a certain size, Redis can call bgrewriteaof to rewrite the log file. When the current AOF file size is twice the size of the AOF file obtained from the last log rewrite (set to 100), a new log rewriting process is automatically started

auto-aof-rewrite-percentage 100

• The aof file may be incomplete at the end. When redis is started, the data of the aof file is loaded into the memory. The restart may occur after the host operating system where redis is down, especially if the data=ordered option is not added to the ext4 file system (redis down or abnormal termination will not cause incomplete tails.) If this phenomenon occurs, you can choose to let Redis exits, or import as much data as possible. If you select yes, when the truncated aof file is imported, a log will be automatically posted to the client and then loaded. If it is no, the user must manually redis-check-aof to repair the AOF file.

aof-load-truncated yes