table of Contents
One, relational database and non-relational database
2.1 Relational database
- A structured database, created on the basis of a relational model
- Generally for records, mainly including
Oracle, MySQL, SQL Server, MIicrosoft Access, DB2, etc.
2.2 Non-relational database
- Except for mainstream relational databases, all databases are considered non-relational databases
- Mainly include Redis, MongBD, Hbase, CouhDB
- The background of non-relational databases
High performance————High concurrent read and write requirements for the database
Huge Storage————The demand for funny storage and access to massive data
High Scalability && High Availability————High scalability and high for the database Available demand
Second, Redis introduction and installation
2.1 Introduction to Redis
- Redis runs on memory and supports persistence
- Use key-value (key-value pair) storage format
- Advantages
Very high data read and write speed
Supports rich data types
Supports data persistence
Atomicity
Supports data backup
2.2 Redis installation and deployment
[root@localhost ~]# yum install gcc gcc-c++ -y
准备安装包
redis-5.0.7.tar.gz
[root@localhost ~]# tar zxvf redis-5.0.7.tar.gz -C /opt/
[root@localhost ~]# cd /opt/redis-5.0.7/
[root@localhost ~]# make
[root@localhost redis-5.0.7]# make PREFIX=/usr/local/redis install
[root@localhost redis-5.0.7]# ln -s /usr/local/redis/bin/* /usr/local/bin/ ## 建立软连接让系统能识别redis命令
[root@localhost redis-5.0.7]# cd /opt/redis-5.0.7/utils/
[root@localhost utils]# ./install_server.sh
………………一直回车
Port : 6379
Config file : /etc/redis/6379.conf
Log file : /var/log/redis_6379.log
Data dir : /var/lib/redis/6379
Executable : /usr/local/bin/redis-server
Cli Executable : /usr/local/bin/redis-cli
Is this ok? Then press ENTER to go on or Ctrl-C to abort.
[root@localhost utils]# netstat -lnupt | grep redis
tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN 18495/redis-server
[root@localhost ~]# /etc/init.d/redis_6379 stop ## 停止
Stopping ...
Redis stopped
[root@localhost ~]# /etc/init.d/redis_6379 start ## 开启
Starting Redis server...
[root@localhost ~]# /etc/init.d/redis_6379 restart ## 重启
[root@localhost ~]# /etc/init.d/redis_6379 status ## 查看状态
2.3 Redis configuration file modification
[root@localhost ~]# vim /etc/redis/6379.conf
bind 127.0.0.1 192.168.233.127 ## 70行 添加本机的IP地址
port 6379 ##93 端口号6379
daemonize yes ## 137 开启守护进程
pidfile /var/run/redis_6379.pid ## 159 指定PID文件
loglevel notice ## 167 日志级别
logfile /var/log/redis_6379.log ## 172 指定日志文件
Three, Redis database commonly used commands
3.1 Connect to the database
- Connect to local database
[root@localhost ~]# /usr/local/redis/bin/redis-cli
127.0.0.1:6379>
- Connect to remote database
[root@localhost ~]# /usr/local/redis/bin/redis-cli -h 192.168.233.127 -p 6379
192.168.233.127:6379>
3.2 Get help commands
help @<group>: 获取<group>中的命令列表
help <command>: 获取某个命令的帮助
help <tab> : 获取可能帮助的主题列表
127.0.0.1:6379> help set
SET key value [expiration EX seconds|PX milliseconds] [NX|XX]
summary: Set the string value of a key
since: 1.0.0
group: string
127.0.0.1:6379> help @list
BLPOP key [key ...] timeout
summary: Remove and get the first element in a list, or block until one is available
since: 2.0.0
BRPOP key [key ...] timeout
summary: Remove and get the last element in a list, or block until one is available
since: 2.0.0
BRPOPLPUSH source destination timeout
summary: Pop a value from a list, push it to another list and return it; or block until one is available
since: 2.2.0
3.3 Store and get data
- get Get data
- set to store data
127.0.0.1:6379> set name zhangsan
OK
127.0.0.1:6379> get name
"zhangsan"
3.4 key related commands
| Statement | effect |
|---|---|
| keys | Get a part-time job that meets the rules |
| exists | Determine whether the key value exists |
| of the | Delete the specified key of the current database |
| type | Get the value type corresponding to the key |
| rename (cover)/renamex (not cover) | Rename the existing key |
| dbsize | View the number of keys in the current database |
3.5 redis-benchmark
- The basic test syntax is redis-benchmark [option] [option value]
-h:指定服务器主机
-p:指定服务器端口
-s:指定服务器 socket
-c：指定并发连接数
-n: 指定请求数
-d：以字节的形式指定SET/GET 值的数据大小
-k： 1=keep alive 0=reconnect
-r： SET/GET/INCR 使用随机key ，SADD使用随机值
-q： 强制退出 Redis 仅显示query/sec值
[root@localhost ~]# redis-benchmark -h 192.168.233.127 -p 6379 -c 100 -n 100000 -q
## 向IP地址为192.168.233.127 端口为6379 的Redis服务器 发送100个并发连接与100000个请求测试性能 只显示query/sec值
[root@localhost ~]# redis-benchmark -h 192.168.233.127 -p 6379 -q -d 100
## 测试 存取大小为100字节的数据包的性能
[root@localhost ~]# redis-benchmark -h 192.168.233.127 -p 6379 -q -d 100
3.6 Redis multi-database operations
- Redis supports most databases, data supports 16 databases, 0-15 commands
- Multiple databases are independent of each other and do not interfere with each other
- Common commands for
multiple databases Switch between
multiple databases Move data in multiple databases
Clear data in the database
Fourth, Redis persistence
4.1 Overview of persistence
- Redis runs in memory, and the data in the memory is lost when power is off
- In order to be able to reuse Redis data or prevent system failures, the data in Redis needs to be written to disk space, that is, persistence.
4.2 Persistent classification
RDB method: Create a snapshot to obtain a copy of all data in Redis at a certain time.
AOF mode: write the executed write command to the end of the file, and record the data changes in a log mode
4.3 RDB endurance
-
Redis default persistence method RDB
default file name dump.rdb ## /var/lib/log/redis/6379
Trigger condition:
within a specified time interval, perform a specified number of write operations (configuration file control)
execute save or yes bgsave (asynchronous) command
Execute flushall command (use with caution) to clear all data in the database
Execute shutdown command to ensure that the server is shut down normally without losing any data -
Pros and cons
-
Suitable for large-scale data recovery
-
If the business does not require high data integrity and consistency, RDB is a good choice
-
Data integrity and consistency are not high
-
Occupy memory during backup
4.4 Recover data through RDB file
- Copy the dump.rdb file to the bin directory of the redis installation directory and restart the redis service
- Configuration file options
[root@localhost ~]# vim /etc/redis/6379.conf
219 save 900 1
220 save 300 10
221 save 60 10000
## 900秒之内至少一次写操作、300秒之内至少发生10次写操作、60秒之内至少10000次写操作，只要满足其一都会触发快照操作，注释所有的save项表示关闭 RDB
254 dbfilename dump.rdb ## RDB文件名称
264 dir /var/lib/redis/6379 ## RDB文件路径
242 rdbcompression yes ## 是否进行压缩
4.5 AOF persistence
- AOF persistence is not turned on by default
- Make up for the shortcomings of RDB
- Use a log to record each operation and append it to the file
- Redis restart will execute the write instruction from front to back according to the content of the log file to complete the data recovery work
4.6 Restore data based on AOF file
- Restore data according to the AOF file.
Copy the appendonly.aof file to the bin directory of the Redis installation directory, restart the Redis service to
configure the file options
[root@localhost ~]# vim /etc/redis/6379.conf
700 appendonly yes ##开启AOF持久化
704 appendfilename "appendonly.aof" ## AOF文件名称
729 # appendfsync always ## 同步持久化，每次发生数据变化会立刻写入磁盘
730 appendfsync everysec ## 默认推荐，每秒异步记录一次（默认值）
731 # appendfsync no ## 不同步 交给操作系统决定如何同步
796 aof-load-truncated yes ## 忽略最后一条可能存在问题的命令 例如执行到一半中断了的命令
4.7 Rewriting mechanism and principle of AOF
AOF rewrite mechanism
- The working principle of AOF is to append write operations to the file, and the redundant content of the file will become more and more
- When the size of the AOF file exceeds the set threshold, Redis will compress the content of the AOF file.
The principle of AOF rewriting - Redis will fork a new process, read the data in the memory (not read the old file), and rewrite it to a temporary file, and finally replace the old aof file
4.8 AOF rewrite configuration
vim /etc/redis/6379.conf
#在日志进行BGREWRITEAOF时，如果设置为yes表示新写操作不进
行同步fsync，只是暂存在缓冲区里，避免造成磁盘IO操作冲突，等重
写完成后在写入。Redis中默认为no
no-appendfsync-on-rewrite no
#当前AOF文件大小是上次日志重写时AOF文件大小两倍时，发生
BGREWRITEAOF操作
auto-aof-rewrite-percentage 100
#当前AOF文件执行BGREWRITEAOF命令的最小值，避免刚开始启动
Reids时由于文件尺寸较小导致频繁的BGREWRITEAOF
auto-aof-rewrite- -min-size 64mb
Five, redis performance management
5.1 View Redis memory usage
[root@localhost ~]# /usr/local/redis/bin/redis-cli -h 192.168.233.127 -p 6379
192.168.233.127:6379> info memory
used_memory:11767376 ## 内存使用总量
used_memory_human:11.22M
……………… 省略一部分
allocator_rss_ratio:1.58 ## 内存碎片率
5.2 Memory fragmentation rate
- The memory fragmentation rate
is calculated by dividing the memory value used_memory_rss allocated by the operating system by the memory value used_memory used by Redis - Memory fragmentation is caused by the inefficient allocation/reclamation of physical memory by the operating system.
Discontinuous physical memory allocation - Tracking the memory fragmentation rate is very important to understand the resource performance of the Redis instance. It
is reasonable that the memory fragmentation rate is slightly greater than 1. This value indicates that the memory fragmentation rate is relatively low. The
memory fragmentation rate exceeds 1.5, indicating that Redis consumes the actual physical memory of Xu Yaping 150%, of which 50% is the memory fragmentation rate. The
memory fragmentation rate is less than 1, indicating that the Redis memory allocation exceeds the physical memory, and the operating system is in the process of memory swapping.
Memory fragmentation around 1 is the best
5.3 Memory usage
The memory usage of the redis instance exceeds the maximum available memory, and the operating system will begin to exchange memory and swap space
- Avoid memory swapping
Select the size of cached data
Use Hash data structure as much as possible
Set key expiration time
5.4 Recover key
- Ensure reasonable allocation of redis's limited memory resources
- When the set maximum threshold is reached, a key recovery strategy needs to be selected
默认情况下回收策略是禁止删除
redis.conf配置文件中修改 maxmemory-policy属性值
volatile-lru:使用LRU算法从已设置过期时间的数据集合中淘汰数据
volatile-ttl:从已设置过期时间的数据集合中挑选即将过期的数据淘汰
volatile-random; 从已设置过期时间的数据集合中随机挑选数据淘汰
allkeys-lru:使用L RU算法从所有数据集合中淘汰数据
allkeys-random:从数据集合中任意选择数据淘汰
no-enviction: 禁止淘汰数据