Redis configuration and introduction

1. Overview of redis

Redis (Remote Dictionary Server, remote dictionary service) is a non-relational ( NoSQL ) database. It is an open source key-value storage system written in C language (different from MySQL's two-dimensional table storage)

Redis will periodically write updated data to disk or write modification operations to additional record files to achieve data persistence

Redis official website address:

https://redis.io/

Chinese website
http://www.redis.cn/

Redis has the following characteristics:

• High read and write performance: Redis read speed is 110000 times/s, write speed is 81000 times/s
• Atomicity: All Redis operations are atomic. At the same time, Redis also supports atomic execution of several operations.
• Support multiple data structures: string (string); list (list); hash (hash), set (collection); zset (ordered collection)
• Stability: master-slave structure
• Support affairs
• Support expiration time
• Support news subscription

2. Configure redis

2.1 Download the redis installation package and unzip

# 下载安装包
cd /export/softwares
wget http://download.redis.io/releases/redis-3.2.8.tar.gz

# 解压
cd /export/softwares
tar -zxvf redis-3.2.8.tar.gz -C ../servers/

2.2 Install C program operating environment

yum -y install gcc-c++

2.3 Install tcl

Tcl (tool command language) is a simple scripting language , mainly used to issue commands to some interactive programs such as text editors , debuggers and shells . It has a simple syntax and strong extensibility. Tcl can create new procedures to enhance its built-in command capabilities.

Secondly, Tcl is a library package that can be embedded in applications . The Tcl library contains an analyzer , routines for executing built-in commands, and library functions that allow you to expand (define new procedures) . The application program can generate and execute Tcl commands. The commands can be generated by the user or read from an input in the user interface (button or menu, etc.). But after receiving the command, the Tcl library decomposes it and executes the built-in commands, which often produces recursive calls

yum  -y  install  tcl

2.4 Edit redis

# 编译
make MALLOC=libc 或 make
# 测试编译是否完成
make test 或者 make install

2.5 Modify the configuration file

First create the logs and redisdata folders

cd /export/servers/redis-3.2.8/
mkdir -p /export/servers/redis-3.2.8/logs
mkdir -p /export/servers/redis-3.2.8/redisdata
# 修改redis.conf
vim redis.conf

Need to modify these places

bind hadoop1
# 后台服务运行
daemonize yes
pidfile /var/run/redis_6379.pid
# 日志文件路径
logfile "/export/servers/redis-3.2.8/logs/redis.log"
# 数据存放
dir /export/servers/redis-3.2.8/redisdata

2.5.1 Implementation of master-slave replication architecture

# 在配置hadoop2和hadoop3时还需要加下面这条指令
slaveof hadoop1 6379

2.4.2 Implement Sentinel Architecture

Sentinel (Sentinel) is a high-availability solution for Redis: The Sentinel system consisting of one or more Sentinel instances can monitor any number of master servers, as well as all slave servers under these master servers, and enter the monitored master server When offline, a slave server under the offline master server is automatically upgraded to a new master server.

The specific mechanism is shown in the figure below:

Modify sentinel.conffile

bind hadoop1
# 后台服务运行
daemonize yes
# 修改三台机器监控的主节点,hadoop1是主节点，有两台主机监控
sentinel monitor mymaster hadoop1 6379 2

2.6 Start redis

cd  /export/servers/redis-3.2.8/
src/redis-server redis.conf

There is no reminder when redis is started, you can use it to ps -ef | grep redischeck whether it is started

2.7 Connect to redis

src/redis-cli -h hadoop1

3. Persistence of redis

Redis persistence is to save data from memory to disk. Persistence in redis is divided into RDB and AOF .

3.1 RDB persistence method

Redis enables RDB snapshots by default, and saves data to a rbd file periodically. You can also trigger RDB snapshots to save manually through sava or bgsave (recommended)

• SAVE directly calls rdbSave to block the main Redis process until the save is complete. During the blocking of the main process, the server cannot process any requests from the client.

• BGSAVE forks a child process , and the child process is responsible for calling rdbSave, and sends a signal to the main process after the save is completed, notifying that the save has been completed. The Redis server can continue to process client requests during the execution of BGSAVE .

Advantages of RDB solution

1. Minimal impact on performance. Redis forks out child processes when saving RDB snapshots, which hardly affects the efficiency of Redis processing client requests.

2. Each snapshot will generate a complete data snapshot file, so it can be supplemented by other means to save snapshots at multiple points in time (for example, backup the snapshot at 0 o'clock every day to other storage media) as a very reliable disaster recovery method.

3. Data recovery using RDB files is much faster than using AOF

Disadvantages of RDB solution

1. Snapshots are generated regularly, so some data will be lost in Redis crashes.

2. If the data set is very large and the CPU is not strong enough (such as a single-core CPU), Redis may consume a relatively long time when fork child processes, which affects Redis's ability to provide external services.

The original configuration in redis is shown in the figure below, you can modify it yourself

You need to restart the redis service after modifying the configuration

# 查看redis进程
ps -ef | grep redis
# kill进程

# 启动redis
cd  /export/servers/redis-3.2.8/
src/redis-server redis.conf

3.2 AOF persistence method

When using AOF persistence, Redis will record every write request in a log file. When Redis restarts, all the write operations recorded in the AOF file will be executed sequentially to ensure that the data is restored to the latest.

AOF has three fsync configurations: always/everysec/no

• always: Perform an fsync operation every time a log is written, with the highest data security
• everysec: fsync once every second
• no: Do ​​not perform fsync operation, let os automatically flush

Advantages of AOF:

1. The safest, when appendfsync always is enabled, any data that has been written will not be lost, and when appendfsync everysec is enabled, only 1 second of data will be lost.

2. The AOF file will not be damaged in the event of a power failure. Even if a log is only half written, it can be easily repaired with the redis-check-aof tool.

3. The AOF file is easy to read and can be modified. After some wrong data removal operations, as long as the AOF file is not rewrite, you can back up the AOF file, delete the wrong command, and then restore the data.

Disadvantages of AOF:

1. AOF files are usually larger than RDB files

2. Performance consumption is higher than RDB

3. Data recovery speed is slower than RDB

In redis, AOF is turned off by default. The settings are shown in the figure below and can be changed

In order to prevent the AOF file from being too large and making the recovery time too long, redis provides the AOF rewrite function, which can rewrite the AOF file, and only retains the minimum set of write operations that can restore the data to the latest state.

AOF rewrite can be triggered by the BGREWRITEAOF command, and Redis can also be configured to automatically perform periodically