Comprehensive analysis of redis

1. What is redis?

Redis is a key-value noSql database based on memory and can be persisted to the hard disk. It supports five data types such as String, List, Set, ZSet, and Hash. It is single-threaded, single-process, and does not support concurrent operations, because the speed is very fast (Redis read speed is 110,000 times/s, write speed is 81000 times/s), so it can also be called macro-parallel and micro-serial.

2. Stand-alone and cluster construction

2.1 Stand-alone version (only for introductory learning and use)

Environmental preparation: centos7, redis-4.0.10.tar.gz ①Turn
off the firewall

systemctl stop firewalld
systemctl disable firewalld

②Install redis environment dependency

yum install gcc

③Unzip the package

tar -zxf  redis-4.0.10.tar.gz

④Enter the decompressed folder and compile

[root@localhost redis-4.0.10]# make

⑤Modify the redis.conf file

[root@localhost redis-4.0.10]# vim redis.conf 
daemonize yes  #开启后台运行模式
protected-mode no  #开启远程访问权限
bind 0.0.0.0  #或者将这一行注释掉

⑥Start redis

[root@localhost src]# ./redis-server ../redis.conf 
# 因为前面配置文件修改为后台运行模式，所以验证是否成功查看redis进行号
[root@localhost src]# ps aux | grep "redis"
root      1490  0.0  1.0 145312  7664 ?        Ssl  12:23   0:00 ./redis-server *:6379
root      1495  0.0  0.1 112724   996 pts/0    R+   12:23   0:00 grep --color=auto redis

⑦Start the client, the default port is 6379, you can also modify the port number in the redis.conf file, the startup command is like: ./redis-cli -p 7000

[root@localhost src]# ./redis-cli

2.2 Redis cluster version (local simulation, using three-master and three-slave mode)

①Turn off the firewall

systemctl stop firewalld
systemctl disable firewalld

②Install the redis cluster environment

yum install ruby
gem install redis-3.2.1.gem

③Unzip

tar -zxf  redis-4.0.10.tar.gz

④Compile

[root@localhost redis-4.0.10]# make

⑤simulate different ports, so we create a folder in the decompression directory to simulate, prevent the redis.conf file under each file, modify the configuration file

cluster-enabled  yes 	        	//开启集群模式
cluster-config-file  nodes-7000.conf 		//集群节点配置文件
cluster-node-timeout  5000      	//集群节点超时时间
appendonly  yes 

⑥Start node 7000~7005

[root@localhost src]# ./redis-server ../7000/redis.conf

⑦ Cover all hash slots

./redis-trib.rb create --replicas 1 192.168.139.172:7000 192.168.139.172:7001 192.168.139.172:7002 192.168.139.172:7003 192.168.139.172:7004 192.168.139.172:7005

⑧View the status of cluster nodes

./redis-trib.rb check 192.168.139.172:7000

⑨Start the client

./redis-cli -c -p 7000

3. Redis persistence mechanism

Redis provides two different persistence mechanisms, namely snapshot (snapShotting) and AOF append file (append only file) .

3.1 Snapshot (snapShotting)

This method can write all data at a certain moment to the hard disk. Of course, this is also the default persistence method of redis, because the saved file is a file ending in .rdb format, so this method is also called RDB mode .

In redis.conf, we can see the file name of .rdb and

the save path of the file:

there are two ways to create a snapshot, namelyBGSAVEwithSAVE
(A) SAVE
can use the SAVE command to create a snapshot. The redis server that receives the SAVE command will no longer respond to any other commands before the snapshot is created, which is what we call the blocking state. When redis receives a request to shut down the server through the shutdown command, it executes a save command, blocks all clients, no longer executes any commands sent by the client, and shuts down the server after the save command is executed.

(B) BGSAVE
can use the BGSAVE command to create a snapshot. When receiving the BGSAVE command from the client, redis will call fork to create a child process, and then the child process is responsible for writing the snapshot to the disk, while the parent process continues to process the command request.

Fork function: create parent and child threads. When there is a client call, the parent and child threads use their own memory separately. When the client does not call, the child thread occupies the parent thread's memory processing snapshot.

You can set the execution cycle of the snapshot in redis.confZ. The default configuration as shown in the figure means: it is changed 10,000 times in 1 minute, or changed 10 times in 5 minutes, or changed once in 15 minutes.

Analysis of the shortcomings of the snapshot method : In the case of an abnormal exit such as the redis process being killed directly, the snapshot will cause data loss before it has time.

3.2 AOF append file (append only file)

In the default configuration of redis, the AOF persistence mechanism is not enabled. AOF persistence will write the executed write command to the end of the AOF file to record the changes in the data, so as long as redis executes AOF once from start to end All the write commands contained in the file can restore the recorded data set of the AOF file.

Turn on the AOF persistence mechanism. In redis.conf, change appendonly to yes

or specify the name of the generated aof file.

Specify the frequency of log update

Options	Sync frequency
always	Each redis write command must be written to the hard disk synchronously, which seriously reduces the speed of redis
everysec	Perform synchronization once per second to explicitly synchronize multiple write commands to disk
no	The operating system decides when to synchronize

Analysis : if usedalwaysOption, then every redis write command will be written to the hard disk, and the data loss that occurs when the system crashes is minimized. Unfortunately, because this synchronization strategy requires a large number of write operations on the hard disk, redis processing The speed of the command will be limited by the performance of the hard disk.
Note: Turntable hard disks are about 200 commands/s at this frequency; solid state disks (SSD) are several million commands/s;
warning: users using SSDs should be cautious to use the always option, this mode continuously writes a small amount of data The practice may cause serious write amplification problems, which will reduce the life of the solid state drive from the original several years to several months.

In order to take into account data security and write performance, you can consider usingeverysecOption to enable redis to synchronize AOF files at a frequency of once per second. When redis synchronizes AOF files once per second, the performance is almost the same as when it does not use any persistence features. By synchronizing AOF files once per second, redis can guarantee even The system crashes, and the data generated within one second is lost at most (this method is recommended).

Finally, if the no option is used, the operating system will determine when to synchronize the AOF log file. This option will not affect the performance of redis, but when the system crashes, an indefinite amount of data will be lost. In addition, if the user's hard disk is not fast enough to process write operations If the buffer is filled up with data waiting to be written to the hard disk, redis will be in a blocked state, and the speed of redis processing command requests will slow down (not recommended)

AOF file rewriting

The aof method also brings another problem, the persistent file will become bigger and bigger. For example, if we call the incr test command 100 times, all 100 commands must be saved in the file. In fact, 99 of them are redundant. Because to restore the state of the database, it is enough to save a set test 100 in the file. In order to compress the persistent files of AOF, Redis provides two AOF rewriting mechanisms, namely:BGREWRITEAOFwithAuto-aof-rewrite-percentage in the configuration file redis.conf

(1) Execute the BGREWRITEAOF command : execute this command directly on the client.
(2) The description of auto-aof-rewrite-percentage in redis.conf

: This action automatically executes BGREWRITEAOF., as shown in the figure, the value of auto-aof-rewrite-percentage is 100 and auto-aof-rewrite-min-size 64mb, and it is enabled When the AOF is persisted, when the volume of the AOF file is greater than 64M, and the volume of the AOF file is at least double (100%) larger than the volume after the last rewrite, it will automatically trigger. If the rewrite is too frequent, you can consider changing auto-aof-rewrite-percentage is set to greater.

AOF rewriting process
① redis calls fork. Now there are two processes: father and son. The child process writes a command to rebuild the database state into a temporary file according to the database snapshot in memory.
② The parent process continues to process client requests, except for writing the write command to the original aof file, and at the same time buffering the received write command, so that it can ensure that there will be no problems if the child process fails to rewrite.
③When the child process writes the snapshot content into a temporary file in a commanded manner, the child process sends a signal to the parent process, and then the parent process also writes the cached write command to the temporary file.
④Now the parent process can use the temporary file to replace the old aof file and rename it, and the write commands received later will also start to append to the new aof file.

4. Redis distributed cache

Disadvantages of the existing mybatis cache:
①Mybatis belongs to the local cache, and the memory of the server will be occupied when the project is placed in the server.
②In a distributed environment, cache sharing cannot be achieved.

mbatis will automatically clear the cache when operating additions, deletions and changes. Redis is
used for distributed cache code, and Java language describes
tool classes. Because of the cache we implemented by ourselves, SpringBoot cannot scan the classes we implemented by ourselves, and automatic injection cannot be used.

package com.chinaTelecom.cache;

import org.springframework.beans.BeansException;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.stereotype.Component;

@Component
public class SpringContextUtil implements ApplicationContextAware {
    
    

   private static ApplicationContext context;
   @Override
   public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
    
    
       context = applicationContext;
   }

   //获取整个工厂对象
   public static ApplicationContext getContext(){
    
    
       return context;
   }

   //根据名称获取指定bean
   public static Object getBean(String name){
    
    
       return context.getBean(name);
   }

   //根据类型获取指定bean
   public static Object getBean(Class clazz){
    
    
       return context.getBean(clazz);
   }

   //根据名称和类型获取
   public static Object getBean(String name,Class clazz){
    
    
       return context.getBean(name,clazz);
   }
}

Function implementation code

package com.chinaTelecom.cache;


import org.apache.ibatis.cache.Cache;
import org.springframework.data.redis.core.RedisTemplate;

import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;

public class MybatisRedisCache implements Cache {
    
    

    private final String id; //id: 包名.DAO名，即mapper文件的namespace
    private final ReentrantReadWriteLock reentrantReadWriteLock = new ReentrantReadWriteLock();
    public MybatisRedisCache(String id) {
    
    
        this.id = id;
    }

    @Override
    public String getId() {
    
    
        return null;
    }

    /**
     * @param key : 唯一值，代表方法唯一
     * @param value： 结果
     */
    @Override
    public void putObject(Object key, Object value) {
    
    
        RedisTemplate redisTemplate = (RedisTemplate)SpringContextUtil.getBean("redisTemplate");
        redisTemplate.opsForHash().put(id,key,value);
    }

    @Override
    public Object getObject(Object key) {
    
    
        RedisTemplate redisTemplate = (RedisTemplate)SpringContextUtil.getBean("redisTemplate");
        return redisTemplate.opsForHash().get(id,key);
    }

    @Override
    public Object removeObject(Object key) {
    
    
        RedisTemplate redisTemplate = (RedisTemplate)SpringContextUtil.getBean("redisTemplate");
        return redisTemplate.opsForHash().delete(id,key);
    }

    @Override
    public void clear() {
    
    
        RedisTemplate redisTemplate = (RedisTemplate)SpringContextUtil.getBean("redisTemplate");
        redisTemplate.opsForHash().delete(id);
    }

    @Override
    public int getSize() {
    
    
        RedisTemplate redisTemplate = (RedisTemplate)SpringContextUtil.getBean("redisTemplate");
        return redisTemplate.opsForHash().size(id).intValue();
    }

    @Override
    public ReadWriteLock getReadWriteLock() {
    
    
        return reentrantReadWriteLock;
    }
}