Redis not only supports decentralized cluster mode, but can also implement master-slave replication clusters. Below I will build a simple master-slave replication cluster step by step to help everyone understand master-slave replication and sentinel mode.
Before starting the text, let me first explain the several concepts involved to save you time in Baidu.
A cluster refers to multiple instances of the same component forming a logical whole.
Master-slave replication is a data backup solution. To put it simply, two or more identical databases are used, one database is used as the master database, and the other database is used as the slave database. When performing corresponding operations in the master database, the slave database records all operations of the master database so that they are exactly the same.
Separation of reading and writing is a way to use the database to make the database more stable.
It is used when there is a slave database. When the master database adds, deletes, or modifies data, that is, writes, the query task is handed over to the slave database. The separation of read operations and write operations can greatly increase the concurrent load capacity of the database.
1. Download and install Redis
Please refer to: The whole process of installing, configuring and starting Redis on Linux (centos7) and solving the make error problem
2. Master-slave replication model
I drew a sketch. This is a simple cluster model with one master and two slaves. The master is used for writing and the slave is used for reading data to achieve separation of reading and writing. Our next step is to implement such a cluster system.
3. Set up cluster environment
I used a centos7 virtual machine to build this demo. The required environment is:
JDK, linux, and Redis.
Copy multiple redis.conf files and start the redis server using different configuration files to simulate multiple services.
1. Start your redis service and enterinfo replicationCommand to view information about the current library
127.0.0.1:6379> info replication
# Replication
role:master //当前库为master
connected_slaves:0 //连接当前master的从机数目为0
master_replid:7b7cb8aa6a54e4fb2b717e59d6d626578c7b7f3f
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
After the cluster is deployed, you can use this command to view the information.
2. Create a new conf directory and copy the redis configuration file three times into the conf directory.
[root@localhost bin]# mkdir conf
[root@localhost bin]# cp redis.conf conf/redis6380.conf
[root@localhost bin]# cp redis.conf conf/redis6381.conf
[root@localhost bin]# cp redis.conf conf/redis6382.conf
In this way we get three configuration files. Let's make a few changes to them.
3. Change the host configuration file
We configure redis6380.conf as the host (master)
and enter the conf directory.vim redis6380.confEdit configuration file
- First we need to change the port number to correspond to the configuration file name
- Remember to change it to yes when running in the background
- Change the pid file name to avoid duplication
- Change the log file name to avoid duplication
- Change rdb file name to avoid duplication
4. To change the slave configuration file, refer to the host modification method.
The changes to the host are all the same. We can set the ports to 6381 and 6382 respectively. Change the log file and rdb file similarly to avoid duplication of names.
If you want to configure master-slave replication directly through the configuration file, then you should add a configuration when changing the slave configuration file:slaveof 127.0.0.1 6380, specifying their host server address.
In this way, they will start directly as slaves. At this time, please ignore the input slaveof step below.
5. Start separately
We can start multiple command terminals to open them separately, or use Xshell to multiple tab pages.
Execute the following commands for each one (note changing the port number):
After all are started, check the process information and confirm againps -ef|grep redis:
The above picture shows that all services have been successfully started and the cluster environment has been set up.
4. Configure master-slave replication (slaveof)
Now we can enter in each serviceinfo replication, you will find that the three redis are now hosts, and there is no connection between them. We need to configure it manually.
- Open the 6381 client and useslaveof 127.0.0.1 6380command, specifying the host address for it. Theninfo replicationYou will find that its identity becomes a slave machine.
Similarly, specify 6380 as the host for 6382, which basically forms a master and two slave situation, in which 6380 is the common host and 6381/6382 are slaves.
You can read and write some data on separate clients. You will find that the host can read and write, the slave can only read, and the data is shared between them. For example, you can set several variables on the master, and then use the slave client to access these variables.
Replenish
redis cascade master-slave replication
Redis can not help but support master-slave replication of one master and multiple slaves, and also supports the cascade mode shown in the figure below: The
implementation mode is similar to the one master and multiple slaves mentioned above, that is, the slave service can also be used as the master of others, using the slaveof command Just baby.
Configure master-slave replication through configuration files
To use the configuration file to configure the master-slave replication service, you only need to add a configuration to the slave server configuration file. The master service does not need to make any changes. The configuration is as follows:
slaveof [host] [port]
where host in the above formula is the master server address. port is the main server port, example:
slaveof 127.0.0.1 6379
Redis master-slave replication principle
The Redis master-slave replication synchronization process is divided into two types: full synchronization and incremental synchronization. When the master-slave is just connected, full synchronization is performed; after full synchronization is completed, incremental synchronization is performed. Of course, if necessary, the slave can initiate full synchronization at any time.
So what are full synchronization and incremental synchronization? In fact, in layman's terms, the process of synchronizing all data from the master server to the slave server is full synchronization. As you can imagine, this process should be time-consuming and labor-intensive; while incremental synchronization means that the slave server only synchronizes the latest data written and changed. Data is sent to each extension every time the main server executes a write command to achieve incremental replication, which is very fast.
The redis strategy is that, no matter what, incremental synchronization will be attempted first, and if unsuccessful, the slave machine will be required to perform full synchronization.
The following are the detailed concepts of full synchronization and incremental synchronization:
Full synchronization
Redis full copy generally occurs during the Slave initialization phase. At this time, the Slave needs to copy all the data on the Master. Specific steps are as follows:
- The slave server connects to the master server and sends the SYNC command;
- After receiving the SYNC naming, the main server starts executing the BGSAVE command to generate an RDB file and uses the buffer to record all write commands executed thereafter;
- After the master server BGSAVE is executed, it sends snapshot files to all slave servers and continues to record the executed write commands during the sending period;
- After receiving the snapshot file from the server, discard all old data and load the received snapshot;
- After the master server snapshot is sent, it starts sending the write command in the buffer to the slave server;
- The slave server completes loading the snapshot, starts receiving command requests, and executes write commands from the master server buffer;
After completing the above steps, all operations of data initialization from the slave server are completed. The slave server can now receive read requests from users.
Incremental synchronization
Redis incremental replication refers to the process in which write operations that occur on the master server are synchronized to the slave server when the slave is initialized and starts to work normally.
The process of incremental replication is mainly that every time the master server executes a write command, it will send the same write command to the slave server, and the slave server receives and executes the received write command.
Note:
If multiple slaves are disconnected and need to be restarted, as long as the slave starts, it will send a sync request and fully synchronize with the host. When multiple slaves appear at the same time, it may cause a sharp increase in Master IO downtime.
Expand knowledge
The configuration of Redis master-slave replication is very simple, it can make the slave server a complete copy of the master server. There are several important points that need to be understood about Redis master-slave replication:
1) Redis uses asynchronous replication. But starting with Redis 2.8, the slave server will periodically respond to the amount of data processed from the replication stream.
2) A master server can have multiple slave servers.
3) The slave server can also accept connections from other slave servers. In addition to multiple slave servers connected to a master server, multiple slave servers can also be connected to a slave server to form a
graph-like structure.
4) Redis master-slave replication does not block the master server. That is to say, when several slave servers are performing initial synchronization, the master server can still process requests.
5) Master-slave replication does not block the slave server side. When the slave server performs the initial synchronization, it uses the old version of the data to respond to query requests, assuming you configured this in the redis.conf configuration file. Otherwise, you can configure the slave to return an error to the client when the replication stream is closed. However, when the initial synchronization is completed, the old data set needs to be deleted and the new data set needs to be loaded. During this short period of time, the slave server will block incoming requests.
6) Master-slave replication can be used to enhance scalability, using multiple slave servers to handle read-only requests (for example, heavy sorting operations can be done on the slave servers), or it can be simply used for data redundancy.
7) Using master-slave replication can save the master server from the consumption of writing data to disk: configure "Avoid saving" in the redis.conf file of the master server (comment out all "save" commands), and then connect a server configured as "Proceed" Just click "Save" on the slave server. But this configuration must ensure that the main server will not automatically restart
In the next article, we will configure the sentinel mechanism for them to prepare for the special situation of host failure.