(1) Introduction, Installation, Persistence, Data Types of Redis

9. Introduction to redis

• Redis is similar to Memcached and also belongs to kv data storage, but its functions and operability are much better than Memcached.
• Redis official website redis.io, the current latest stable version 4.0.9, supports more value types, in addition to string, also supports hash, lists (linked list), sets (collection) and sorted sets (ordered collection)
• Redis uses two file formats: full data (RDB) and incremental requests (aof).
• The full data format is to write the data in the memory to the disk, so that it is convenient to read the file for loading next time.
• The incremental request file serializes the data in the memory into an operation request, which is used to read the file for replay to obtain the data, which is similar to mysql binlog.
• In order to save resources, when we manually save the full amount of data once, we can delete all the current incremental data. Some incremental data has already expired, and we can also use scripts to do some optimizations regularly.
• The storage of redis is divided into three parts : memory storage, disk storage and log file

10. redis installation

1 download

官网地址 redis.io
[root@yt-01 ~]# cd /usr/local/src
[root@yt-01 src]# wget http://download.redis.io/releases/redis-4.0.9.tar.gz

2 Installation

[root@yt-01 src]# tar zxvf redis-4.0.9.tar.gz   //解压
[root@yt-01 src]# cd redis-4.0.9                    //进入redis目录
[root@yt-01 redis-4.0.9]# make && make install    //和其他软件不同，redis安装直接就可以编译
[root@yt-01 redis-4.0.9]# echo $?      //确认编译是否成功
0
[root@yt-01 redis-4.0.9]# redis-      //用tab查看redis的命令，确认是否安装成功
redis-benchmark redis-check-aof redis-check-rdb redis-cli redis-sentinel redis-server
[root@yt-01 redis-4.0.9]# which redis-cli    //查看redis的启动文件地址
/usr/local/bin/redis-cli
[root@yt-01 redis-4.0.9]# ls    //查看redis安装后的文件
00-RELEASENOTES BUGS CONTRIBUTING COPYING deps INSTALL Makefile MANIFESTO README.md redis.conf runtest runtest-cluster runtest-sentinel sentinel.conf src tests utils

3 Configuration files

[root@yt-01 redis-4.0.9]# cp redis.conf /etc/redis.conf
[root@yt-01 redis-4.0.9]# vim /etc/redis.conf    //修改配置文件
……
daemonize yes     #yes表示后台启动；no表示前台启动
pidfile /var/run/redis_6379.pid      #pid存放位置
loglevel notice
#指定日志级别：debug（调试，排错）、verbose（冗长的）、notice、warning
#debug适合排查错误时使用，错误排查完毕后要换到notice，避免产生大量日志浪费系统空间
logfile "/var/log/redis.log" #定义日志存放路径
databases 16
#Redis有库的概念，默认在0库
dir /data/redis #定义rdb、aof文件的存放位置
appendonly yes #开启aof日志，开启后会在dir定义的目录生成appendonly.aof文件
appendfsync everysec #指定记录日志的规则：always（只要有变动就记录）、everysec（每秒记录一次）、no（不记录）

创建rdb、aof文件的存放文件夹
[root@yt-01 redis-4.0.9]# mkdir /data/redis

启动redis
[root@yt-01 redis-4.0.9]# redis-server /etc/redis.conf

查看redis进程（因为虚拟机还运行着gitlab网站服务，所以进程会多几个）
[root@yt-01 redis-4.0.9]# ps aux |grep redis
root 5526 0.0 0.0 4184 16 ? Ss 10:10 0:00 runsv redis
root 5527 0.0 0.0 4328 56 ? S 10:10 0:00 svlogd -tt /var/log/gitlab/redis
gitlab-+ 5528 0.2 0.1 41484 3740 ? Ssl 10:10 0:36 /opt/gitlab/embedded/bin/redis-server 127.0.0.1:0
root 6041 0.0 0.0 4184 0 ? Ss 10:11 0:00 runsv redis-exporter
root 6042 0.0 0.0 4328 36 ? S 10:11 0:00 svlogd -tt /var/log/gitlab/redis-exporter
gitlab-+ 6043 0.0 0.4 53424 7976 ? Ssl 10:11 0:04 /opt/gitlab/embedded/bin/redis_exporter -web.listen-address=localhost:9121 -redis.addr=unix:///var/opt/gitlab/redis/redis.socket
root 36317 0.0 0.1 145260 2184 ? Ssl 13:33 0:00 redis-server 127.0.0.1:6379               //默认监听端口是6379

查看redis日志
[root@yt-01 redis-4.0.9]# less /var/log/redis.log
# 2条warning，提醒我们有2个内核参数需要调整
……
36317:M 06 Apr 13:33:51.922 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
36317:M 06 Apr 13:33:51.922 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
36317:M 06 Apr 13:33:51.922 * Ready to accept connections
/var/log/redis.log (END)

配置内核参数
[root@yt-01 redis-4.0.9]# sysctl vm.overcommit_memory=1
[root@yt-01 redis-4.0.9]# echo never > /sys/kernel/mm/transparent_hugepage/enabled

加入开机启动
[root@yt-01 redis-4.0.9]# vim /etc/rc.local
sysctl vm.overcommit_memory=1
echo never > /sys/kernel/mm/transparent_hugepage/enabled

启动
[root@yt-01 redis-4.0.9]# redis-server /etc/redis.conf

4 Methods of shutting down the redis service

• Method 1: Query the PID, kill -9 pid. This method is equivalent to power off, abnormal shutdown, generally not used, will cause data loss

• Method 2

[root@yt-01 redis-4.0.9]# redis-cli shutdown

11. redis persistence

Like the Memcached service, if the RDB and aof data are closed, they will be stored in memory, and the stored data will be lost when the service is restarted or the machine is restarted. If the data is important, we need to do persistence.

• Redis provides two persistence methods, namely RDB (Redis DataBase) and AOF (Append Only File)
• RDB, in short, is to generate snapshots of data stored in redis at different points in time and store them on media such as disks.
• AOF is a different perspective to achieve persistence, that is, to record all the write instructions executed by redis. When redis restarts next time, as long as these write instructions are repeated from the front to the beginning, the data can be realized. back to normal.
• In fact, RDB and AOF can also be used at the same time. In this case, if redis restarts, the AOF method will be preferred for data recovery, because the data recovery integrity of the AOF method is higher.
• If you don't need data persistence, you can turn off RDB and AOF, so redis will become a pure memory database, just like memcache.

1 Redis persistence related parameters

save 900 1          //表示每15分钟且至少有1个key改变，就触发一次持久化
save 300 10        //表示每5分钟且至少有10个key改变，就触发一次持久化
save 60 10000    //表示每60秒至少有10000个key改变，就触发一次持久
save " "                //这样可以禁用rdb持久化
appendonly yes   //如果是yes，则开启aof持久化
appendfilename “appendonly.aof”   //指定aof文件名字
appendfsync everysec  //每一秒写入aof文件，并完成磁盘同步。
# 指定fsync()调用模式，有三种
# no(不调用fsync),always(每次写都会调用fsync),everysec(每秒钟调用一次fsync)。
# 第一种最快，第二种数据最安全，但会占用磁盘I/O会差一些，第三种为这种方案最恰当，默认选第三种。

2 Turn persistence on and off

开启
[root@yt-01 ~]# vim /etc/redis.conf
……
# save " "
save 900 1
save 300 10
save 60 10000

关闭
[root@yt-01 ~]# vim /etc/redis.conf
……
save " "
# save 900 1
# save 300 10
# save 60 10000

设置完后，重启服务
# redis-cli -p 127.0.0.1 shutdown 
# redis-server  /etc/redis.conf

12. redis data type

string
list
set
sort set
hash

1 Redis data type - string

String is the simplest type, the same type as Memcached. One key corresponds to one value. The supported operations are similar to those of Memcached, and its functions are more abundant. Sets the object that can store binary.

[root@yt-01 ~]# redis-cli
127.0.0.1:6379> set mykey 123
OK
127.0.0.1:6379> get mykey
"123"
127.0.0.1:6379> mset k1 a k2 b k3 c
OK
127.0.0.1:6379> mget k1 k2 k3
1) "a"
2) "b"
3) "c"
127.0.0.1:6379> quit

2 Redis data types - list

• List is a linked list structure, the main functions are push, pop, get all the values ​​of a range and so on. In the operation, the key is understood as the name of the linked list.

• Using the list structure, we can easily implement the latest news ranking and other functions (such as Sina Weibo's TimeLine). Another application of list is the message queue. You can use the push operation of the list to store the task in the list, and then the worker thread uses the pop operation to take out the task for execution.

[root@yt-01 redis-4.0.9]# redis-cli
# 添加一条list数据
127.0.0.1:6379> lpush list1 "yuntai"
(integer) 1
127.0.0.1:6379> lpush list1 "yuntai1"
(integer) 2
127.0.0.1:6379> lpush list1 "yuntai2"
(integer) 3

# 查看指定list的数据
127.0.0.1:6379> lrange list1 0 -1
1) "yuntai2"
2) "yuntai1"
3) "yuntai"

# 取出（pop挤出）一条数据，挤出就没有了
127.0.0.1:6379> lpop list1
"yuntai2"
# 取出数据list 从0到1
127.0.0.1:6379> lrange list1 0 -1
1) "yuntai1"
2) "yuntai"
127.0.0.1:6379> quit

3 Redis data types - set

• A set is a set, which is similar to the concept of a set in our mathematics. The operations on the set include adding and deleting elements, and there are operations such as the intersection and difference of multiple sets. The key in the operation is understood as the name of the collection. For example, in a Weibo application, all followers of a user can be stored in a set, and all followers of a user can be stored in a set. Because Redis provides collections with operations such as intersection, union, and difference in a very user-friendly manner, it is very convenient to implement functions such as common attention, common preferences, and second-degree friends. For all the above set operations, you can also Different commands can be used to choose whether to return the result to the client or save it to a new collection.

[root@yt-01 redis-4.0.9]# redis-cli
# 添加一条set数据
127.0.0.1:6379> SADD set1 a
(integer) 1
127.0.0.1:6379> SADD set1 b
(integer) 1
127.0.0.1:6379> SADD set1 c
(integer) 1
127.0.0.1:6379> SADD set1 d
(integer) 1

# 查看set数据
127.0.0.1:6379> SMEMBERS set1
1) "d"
2) "c"
3) "a"
4) "b"
127.0.0.1:6379> SREM set1 c
(integer) 1
127.0.0.1:6379> SADD set2 a 2 b
(integer) 3
127.0.0.1:6379> SINTER set1 set2
1) "a"
2) "b"
127.0.0.1:6379> SUNION set1 set2
1) "d"
2) "a"
3) "2"
4) "b"
127.0.0.1:6379> SDIFF set1 set2
1) "d"
127.0.0.1:6379> quit

4 Redis data types - sort set

• Sorted set is an ordered set. It has one more weight parameter score than set, so that the elements in the set can be arranged in order by score. For example, a Sorted Sets that stores the grades of the whole class, the set value can be the student number of the classmate. , and the score can be its test score, so that when the data is inserted into the set, it has already been sorted naturally.

[root@yt-01 redis-4.0.9]# redis-cli
# 添加sort set数据
127.0.0.1:6379> ZADD set3 12 abc
(integer) 1
127.0.0.1:6379> ZADD set3 2 "cde 123"
(integer) 1
127.0.0.1:6379> ZADD set3 24 "123-aaa"
(integer) 1
127.0.0.1:6379> ZADD set3 4 "a123a"
(integer) 1

# 查看数据set3
127.0.0.1:6379> ZRANGE set3 0 -1
1) "cde 123"
2) "a123a"
3) "abc"
4) "123-aaa"
#数据的value根据score大小排序

# 倒序查看数据set3
127.0.0.1:6379> ZREVRANGE set3 0 -1
1) "123-aaa"
2) "abc"
3) "a123a"
4) "cde 123"
127.0.0.1:6379> quit

5 Redis data type - hash

In Memcached, we often package some structured information into a hashmap, which is serialized on the client and stored as a string value (usually in JSON format), such as the user's nickname, age, gender, points, etc.

[root@yt-01 redis-4.0.9]# redis-cli
# 添加一条hash数据hash1 k值为name，v值为yuntai
127.0.0.1:6379> hset hash1 name yuntai
(integer) 1

# 查询数据hash1 k值为name的v值
127.0.0.1:6379> hget hash1 name
"yuntai"

127.0.0.1:6379> hset hash1 age 27
(integer) 1
127.0.0.1:6379> hget hash1 age
"27"

# 查询hash1的所有值
127.0.0.1:6379> hgetall hash1
1) "name"
2) "yuntai"
3) "age"
4) "27"
127.0.0.1:6379> quit