NoSQL Redis installation configuration and optimization of (+ Practice theory)

Relational databases and non-relational databases

Relational Database

A structured database, created in the relational model, based on - as for the record
Including Oracle, MySQL, SQL Server, Microsoft Access, DB2, etc.

Non-relational databases

In addition to the database outside of the mainstream relational database, it is considered non-relational
Including Redis, MongBD, Hbase, CouhDB etc.

Non-relational database background

High performance-- high demand for concurrent read and write database
Huge Storage-- efficient storage and access requirements for massive data
High Scalability && High Availability-- database high scalability and high availability requirements

Redis Introduction

Redis-based memory to run and support persistence
Using the key-value (key-value pair) is stored in the form of
advantage
- With a high speed data read and write
- Support for rich data types
- Persistence support data
- Atomicity
- It supports data backup

Redis configuration file

Configuration parameters (/etc/redis/6379.conf)

bind: listening host address
port: Port
daemonize yes: Enable daemon
pidfile: Specifies the PID file
loglevel notice: Log Level
logfile: specified log file

Service set up practice

Installation services environment components and mount the archive, compile and install redis

[root@localhost ~]# yum install gcc gcc-c++ make -y         //安装环境组件
[root@localhost ~]# mount.cifs //192.168.100.3/LNMP-C7 /mnt/     //挂载软件包
Password for root@//192.168.100.8/LNMP-C7:  
[root@localhost ~]# cd /mnt/
[root@localhost mnt]# tar zxvf redis-5.0.7.tar.gz -C /opt/      //解压
[root@localhost mnt]# cd /opt/redis-5.0.7/
[root@localhost redis-5.0.7]# make                     //编译
[root@localhost redis-5.0.7]# make PREFIX=/usr/local/redis/ install     //安装

Redis to perform configuration profile scripts, and configure

[root@localhost redis-5.0.7]# cd utils/
[root@localhost utils]# ./install_server.sh       //执行脚本进行配置
Welcome to the redis service installer
This script will help you easily set up a running redis server

Please select the redis port for this instance: [6379]       //默认端口号
Selecting default: 6379
Please select the redis config file name [/etc/redis/6379.conf]     //配置文件存放位置
Selected default - /etc/redis/6379.conf
Please select the redis log file name [/var/log/redis_6379.log]       //日志文件存放位置
Selected default - /var/log/redis_6379.log
Please select the data directory for this instance [/var/lib/redis/6379]      //数据文件存放位置
Selected default - /var/lib/redis/6379
Please select the redis executable path [] /usr/local/redis/bin/redis-server     //可执行文件路径
[root@localhost utils]# ln -s /usr/local/redis/bin/* /usr/local/bin/       //制作链接文件便于系统识别
[root@localhost utils]# netstat -ntap | grep 6379                    //查看端口是否开启
tcp        0      0 127.0.0.1:6379          0.0.0.0:*               LISTEN      44510/redis-server  
[root@localhost utils]# /etc/init.d/redis_6379 stop              //关闭redis命令
Stopping ...
Redis stopped
[root@localhost utils]# /etc/init.d/redis_6379 start                //开启redis命令
Starting Redis server...
[root@localhost utils]# vim /etc/redis/6379.conf       //修改配置文件
bind 127.0.0.1 192.168.144.128                         //配置监听地址
[root@localhost utils]# /etc/init.d/redis_6379 restart      //重启redis服务
Stopping ...
Redis stopped
Starting Redis server...

Redis database commonly used commands

redis-cli command-line tool

Local database connection

[root@localhost utils]# /usr/local/redis/bin/redis-cli
127.0.0.1:6379>

Connect to a remote database

[root@localhost utils]# redis-cli -h 192.168.144.128 -p 6379
192.168.144.128:6379>

Get Command Help

192.168.144.128:6379> help set  ##help帮助

SET key value [expiration EX seconds|PX milliseconds] [NX|XX]
summary: Set the string value of a key
since: 1.0.0
group: string

set: storing data

192.168.144.128:6379> set teacher zhangsan
OK
192.168.144.128:6379> set tea red
OK

get: Obtains data

192.168.144.128:6379> get tea   ##查看键的值
"red"

key related commands

keys: get in line with the rules of the list of keys

127.0.0.1:6379> keys *       //查看当前数据库中所有的键
127.0.0.1:6379> keys V*      //查看当前数据库中以v开头的键
127.0.0.1:6379> keys v?      //查看当前数据库中以v开头后面包含任意一个字符的键
127.0.0.1:6379> keys v??     //查看当前数据库中以v开头后面包含任意二个字符的键

192.168.144.128:6379> KEYS *   ##查看所有的键
1) "teacher"
2) "tea"
192.168.144.128:6379> keys t??  ##查看键是t开头后面是两个字符的
1) "tea"

exists: determining whether there is a key

192.168.144.128:6379> EXISTS tea
(integer) 1                  //1是存在
192.168.144.128:6379> EXISTS teas
(integer) 0                  //0是不存在

del: delete the current designated key database

192.168.144.128:6379> del teacher  ##删除键
(integer) 1
192.168.144.128:6379> KEYS *
1) "tea"

type: Get value corresponding to the key value type

192.168.144.128:6379> type tea   ##查看键的类型
string

rename (cover) / renamenx (not covered): the existing key rename

192.168.144.128:6379> rename tea t1   ##给键重命名
OK
192.168.144.128:6379> keys *
1) "t1"
192.168.144.128:6379> get t1
"red"

dbsize: View the number of key current database
```
192.168.144.128:6379> dbsize
(integer) 1
```
redis-benchmark testing tools
- -h: Specifies the server host name
- -p: Specifies the server port
- -c: Specifies the number of concurrent connections
- -n: Specifies the number of requests
- -d: specifies the data size SET / GET value in bytes
- -q: Forced launched redis. Display only query / sec value

[root@localhost utils]# redis-benchmark -h 192.168.144.128 -p 6379 -c 100 -n 100000     //并发100，100000个请求
====== SET ======
100000 requests completed in 1.14 seconds        //请求花费的时间
100 parallel clients
3 bytes payload
keep alive: 1

84.66% <= 1 milliseconds
98.48% <= 2 milliseconds
99.69% <= 3 milliseconds
99.90% <= 18 milliseconds
100.00% <= 18 milliseconds
87642.41 requests per second

====== GET ======
100000 requests completed in 1.144 seconds
100 parallel clients
3 bytes payload
keep alive: 1
[root@localhost utils]# redis-benchmark -h 192.168.144.128 -p 6379 -q -d 100        //以字节形式指定set/get值的数据大小
SET: 90497.73 requests per second
GET: 90991.81 requests per second

Redis multi-database operations

Redis supports multiple databases, 16 databases supported by default, named 0-15
Multi-database independent, non-interfering

Multi-database commonly used commands

Switching between multiple databases

192.168.144.128:6379> select 10       //进入第11个库
OK
192.168.144.128:6379[10]> keys *
(empty list or set)
192.168.144.128:6379[10]> select 0         //进入第1个库
OK

Move data between multiple databases

192.168.144.128:6379> move t1 10       //移动键值对到第11个库
(integer) 1
192.168.144.128:6379> select 10        //进入第11个库
OK
192.168.144.128:6379[10]> keys *       //查看键
1) "t1"
192.168.144.128:6379[10]> get t1
"red"

Clear the data in the database

192.168.144.128:6379[10]> flushdb          //清除库中数据
OK
192.168.144.128:6379[10]> keys *            //查看所有键
(empty list or set)

Redis persistence

Persistence Overview

Redis is running in memory, the data in memory is lost off
In order to be able to reuse Redis data, or to prevent system failures, we need to write data to disk space Redis, that persistence

Persistence of classification

RDB way: create a snapshot of the way to get a copy of all the data in a time Redis
- Pros: Low resource consumption
- Disadvantages: can not guarantee real-time data is saved
AOF way: Write command will do at the end of the file is written to a log way to record data changes
- Advantages: real time data
- Disadvantages: large resource depletion, loss of data synchronization time is too long
  RDB endurance of
The default Redis persistence mode
The default file name dump.rdb
Triggering conditions
- Within a specified time interval, performing a specified number of write operations (Profile control)
- Execution save or bgsave (asynchronous) command
- Flushall command execution, emptying all the data in the database (not recommended)
- Shutdown command to ensure normal server shut down without loss of any data
Advantages and disadvantages
- Suitable for large-scale data recovery
- If the business of data integrity and consistency do not ask, RDB is a good choice
- Data integrity and consistency is not high
- When the memory for backup

RDB data file recovery

Copy the file to the bin directory dump.rdb redis installation directory, and restart the service to redis

Profile Options

[root@localhost utils]# vim /etc/redis/6379.conf 
save 900 1                    //900秒之内至少一次写操作
save 300 10                   //300秒之内至少发生10次写操作
save 60 10000                 //60秒之内发生至少10000次写操作;只要满足其一都会触发快照操作,注释所有的save项表示关闭RDB
dbfilename dump.rdb           //备份文件名称
dir /var/lib/redis/6379       //备份文件保存目录
rdbcompression yes            //开启压缩

AOF persistence

Redis is not turned on by default
Compensate for the lack of RDB (data inconsistency)
In the form of log records to each write operation, and appended to the file
Redis will restart the write command based on the contents of the log file to perform recovery work once in the past in order to complete the data to back

According to the data file recovery AOF

Copy the file to the bin directory appendonly.aof redis installation directory, you can restart the service redis

AOF persistence configuration

[root@localhost utils]# vim /etc/redis/6379.conf 
appendonly yes          //开启AOF持久化
appendfilename "appendonly.aof"       //AOF文件名称
# appendfsync always                //always:同步持久化，每次发生数据变化会立刻写入磁盘
appendfsync everysec                //everysec:默认推荐，每秒异步记录次(默认值)
# appendfsync no                    //no:不同步，交给操作系统决定如何同步
aof-load-truncated yes              //忽略最后一条可能存在问题的指令

AOF rewrite mechanism

AOF works is to write to append to the file, the contents of the file redundancy will be more and more
When AOF file size exceeds the set threshold, Redis will be the content of AOF file compression

AOF rewrite principle

Redis will fork out a new process to read the data in memory (not to read the old files), and re-written to a temporary file, and finally replace the old aof file

AOF rewrite the configuration

[root@localhost utils]# vim /etc/redis/6379.conf 
no-appendfsync-on-rewrite no        
//在日志进行BGREWRITEAOF时， 如果设置为yes表示新写操作不进行同步fsync,只暂存在缓冲区里，避免造成磁盘I0操作冲突，等重写完成后在写入。redis中默认为no
auto-aof-rewrite-percentage 100     
//当前AOF文件大小是上次日志重写时AOF文件大小两倍时，发生BGREWRITEAOF操作
auto-aof-rewrite-min-size 64mb   
//当前AOF文件执行BGREWRITEAOF命令的最小值，避免刚开始启动Reids时由于文件尺寸较小导致频繁的BGREWRITEAOF

Redis Performance Management

Redis View Memory Usage

[root@localhost utils]# /usr/local/redis/bin/redis-cli
127.0.0.1:6379> info memory

Memory fragmentation rate

Operating system allocates memory value is divided by redis memory used used_memory_rss value calculated used_memory
Memory fragmentation is inefficient dispensing OS / recovery due to the physical memory
- Discontinuous physical memory allocation
Tracking memory fragmentation rate of understanding redis instance resource performance is very important
- Memory fragmentation rate slightly greater than 1 is reasonable, this value indicates the memory fragmentation rate is relatively low
- Memory fragmentation ratio exceeds 1.5, described redis consume 150% of the actual needs of physical memory, wherein the memory fragmentation rate is 50%
- Memory fragmentation rate of less than 1, indicating Redis memory allocation exceeds the physical memory, the operating system being swapping
Memory Usage
- Redis memory usage exceeds the available examples of maximum memory, the operating system will start
- Memory and swap space swap
Avoid memory swapping
- For data cache size selection
- Use Hash data structure as much as possible
- Set the expiration time of key
Recovery key
- Redis ensure the rational allocation of limited memory resources
- When the maximum memory usage reaches the set threshold, need to choose a strategy for key recovery
- By default recovery policy is prohibited deleted
- redis.conf profile modification maxmemory-policy attribute value
- volatile-lru: LRU algorithm using data from a set-out expiration time of the data collection
- volatile-ttl: selection of expiring data from the phase-out has set the expiration time of data collection (recommended)
- volatile-random: randomly selected from a set of data-out expiration time of the data collection
- allkeys-lru: LRU algorithm using data from all the data set out in
- allkeys-random: selecting data from the data set out any
- no-enviction: prohibit out of data