https://www.jianshu.com/p/faf0127f1cb2
https://www.cnblogs.com/xyhero/p/9343747.html ( method of setting master-slave replication with or without locking tables in mysql )
https://www.iteye.com/blog/yunnick-1845301
https://www.jianshu.com/p/0a83d4f077c1
★Preliminary knowledge:
1. Dual machine hot standby
For the concept of dual-machine hot backup, I searched a lot of information, and finally, according to most of the information, it is divided into two meanings, broad sense and narrow sense.
In a broad sense, for important services, two servers are used to back up each other and jointly execute the same service. When one server fails, another server can assume the service task, so that the system can automatically ensure that the system can continue to provide services without manual intervention.
In a narrow sense, dual-machine hot standby is to use two servers that are mutually backed up to jointly perform the same service, one of which is the working machine (Primary Server), and the other is the backup host (Standby Server). When the system is normal, the working machine provides services for the application system, and the backup machine monitors the operation of the working machine (usually through heartbeat diagnosis, and the working machine is also checking whether the backup machine is normal). When the working machine is abnormal, it cannot support the application. When the system is in operation, the backup machine actively takes over the work of the working machine, continues to support key application services, and ensures the uninterrupted operation of the system. Dual-system hot backup is a high-availability solution for failures of IT core servers, storage, and network routing and switching.
2. Why do you need to perform dual-system hot backup?
The dual-system hot standby service is aimed at server failures. The failure of the server may be caused by various reasons, such as equipment failure, operating system failure, software system failure and so on. Generally speaking, it may take 10 minutes, hours or even days to restore the server to normal when the technician is on site. From practical experience, unless the server is simply restarted (maybe hidden dangers still exist), it often takes more than a few hours. And if the technician is not on site, the time to restore service is even longer.
For some important systems, it is difficult for users to endure such a long service interruption. Therefore, it is necessary to use dual-machine hot backup service to avoid long-term service interruption and ensure long-term and reliable service of the system.
Of course, the correct way to decide whether to use dual-machine hot backup is to analyze the importance of the system and the tolerance for service interruption, so as to decide whether to use dual-machine hot backup. In other words, how long can your users tolerate restoring service, and how much impact will it cause if service cannot be restored.
When considering dual-system hot backup, it should be noted that in the general sense, dual-system hot backup will have a switching process, and this switching process may take about one minute. During the switching process, the service may be interrupted for a short time. However, when the switchover is complete, service will resume normally. Therefore, dual-machine hot backup is not seamless and uninterrupted, but it can ensure that normal services can be restored quickly in the event of a system failure, and the business will not be affected. However, if there is no dual-machine hot backup, once a server fails, the service may be interrupted for several hours, and the impact on the business may cause serious losses.
3. The conceptual difference between dual-machine hot backup technology and backup
Hot backup refers to: High Available (HA), which means high availability, and backup refers to Backup, which is a type of data backup. These are two different concepts, and the corresponding products are also two completely different products in terms of functions. Hot backup mainly guarantees the continuity of the business, and the method of realization is the transfer of the point of failure. The main purpose of backup is to make a copy to prevent data loss, so backup emphasizes data recovery rather than application failover.
4. There are two main ways to build the dual-system hot backup solution
When discussing the dual-system hot backup solution, many factors must be considered, and special attention should be paid to it when it is applied in various environments. Of course, there are still two main ways that can be used for reference.
The first one, dual-machine hot standby, works by using two servers, one as the main server (Active), running the application system to provide services. The other is used as a backup machine, with exactly the same application system installed, but in standby mode (Standby). When the Active server fails, the Standby machine is activated through software diagnosis to ensure that the application can be restored to normal use in a short time.
The second method, the dual-machine mutual backup method, is based on the dual-machine hot backup. Two relatively independent applications run on the two machines at the same time, but they are both set as backup machines. When a server fails, Another server can take over the application of the failed server in a short time, thus ensuring the continuity of the application. This method is actually an application of the dual-machine hot backup solution.
But currently the most used is the master-slave mode dual-machine hot backup solution. Its rough representation can be shown in the figure below:
Currently, dual-system hot backup based on storage sharing is the most standard solution for dual-system hot backup. For this approach, two servers are used and shared storage devices (disk array cabinets or storage area network SAN) are used. The two servers can adopt different methods such as master-slave and mutual backup. During the working process, the two servers will provide external services with the next virtual IP address. Depending on the working method, the service request will be sent to one of the servers. At the same time, the server detects the working status of another server through the heartbeat line (currently, the method of establishing a private network is often used).
The following figure is the general status of dual-machine hot backup work, as shown in the figure below:
Dual-system hot backup solution When one server fails, the other server will make a judgment based on the heartbeat detection and switch to take over the service. For users, this process is fully automatic and completed in a short period of time, so that it will not have a great impact on the business. Due to the use of shared storage devices, the two servers actually use the same data, which is managed by dual-machine or cluster software.
5. Schematic diagram of Mysql dual-system hot backup implementation
With the above explanation of dual-machine hot backup knowledge, we have a deep understanding of its implementation principle, then it is much clearer that we need to do dual-machine hot backup of mysql database.
The topology structure of the Mysql dual-system hot standby system is shown in the following figure:
The two servers are connected to the network through Ethernet, provide external services and communicate with each other through the network.
The two servers are connected directly through the com port, and the dual-system hot backup software uses this connection to perform communication, monitoring and control related to the dual-system hot backup.
The two servers are connected to the FC network through the HBA card, access the common disk array, and realize the necessary disks of the dual-system hot backup system.
6. Configuration of Mysql dual-machine hot standby implementation
For data security, the customer has two machines as mutual backup, and when one machine fails, it will automatically switch to another server. Most of the software is implemented through LifeKeeper, but Mysql dual-machine backup is not implemented in LifeKeeper, so I can only manually implement Mysql dual-machine backup.
In fact, there is a very simple third-party software for Mysql dual-machine backup, which is SQLyog. It has a function called sja (SQLyog Job Agent) that can be easily implemented, but it has a shortcoming, that is, the Mysql table must There is a primary key, that is, the primary key value. If not, this table cannot be implemented with sja.
The second method is to use Mysql's own Replication mechanism to achieve. But this function is only available in versions above Mysql 3.23.
Let me explain here first, since I have not passed the actual application example to test whether this dual-machine hot backup method can pass, so I will continue to write about the implementation of Mysql dual-machine hot backup after I implement it through an example configuration section. Now this part mainly talks about the realization principle and significance of dual-system hot backup.
★Implementation of mysql dual machine hot backup
Continuing from the previous article on the analysis of the principle of mysql dual-machine hot backup, this article has been carefully considered and tested in different ways many times. Finally, in this article, a small example is used to complete the implementation of mysql dual-system hot backup.
Mysql database does not have an incremental backup mechanism, and backup is a big problem when the amount of data is too large. Fortunately, the mysql database provides a master-slave backup mechanism. In fact, all the data in the master database is written to the backup database at the same time. Realize hot backup of mysql database.
In order to realize the hot standby of two machines, we must first understand the version requirements of the master-slave database server. To achieve hot standby mysql version is higher than 3.2. Another basic principle is that the data version of the slave database can be higher than that of the master server database, but not lower than that of the master server.
Of course, to achieve mysql dual-machine hot backup, in addition to the REPLICATION function that comes with mysql itself, it can also be realized with Heartbeat, an open source software. However, this article mainly talks about how to use the REPLICATION that comes with mysql to realize the function of mysql dual-system hot backup.
1. Prepare the server
Since the (binary log) binlog format between different versions of Mysql may be different, the best combination is that the Mysql version of the master (Master) server is the same or lower than the version of the slave (Slave) server, and the version of the master server It must not be higher than the slave server version.
The two server versions I used for testing this time are both Mysql-5.5.17.
2. Mysql establishes master-slave server dual-system hot standby configuration steps
2.1 Environment description
A server (master server): 59.151.15.36
B server (slave server): 218.206.70.146
The Mysql versions of the master and slave servers are both 5.5.17
Linux environment
Make a backup copy of the database content that needs to be synchronized on the master server, and upload it to the slave server to ensure that the database content in the two servers is consistent at the beginning.
| Mysqldump application case: 1. First 2. Back up data Back up the remote database locally: mysqldump -uroot -p -h 192.168.220.10 -P3306 --databases test > /data/mysql20210112.bak.SQL 3. Unlock the main library 4. Use the scp command to move the file to the slave library 5. If the slave has been opened from the library, it must be closed first. 6. Import data |
But let me explain here, because I used the database test that Mysql had after installation for testing, so no tables were created in the two servers, but the same empty table tb_mobile was created in the test respectively;
The Sql statement is as follows:
mysql> create table tb_mobile( mobile VARCHAR(20) comment'mobile number', time timestamp DEFAULT now() comment'time' );
2.2 Master server configuration
2.2.1 Create a synchronization user
Enter the mysql operation interface, and establish a connection account for the slave server on the master server. This account must grant REPLICATION SLAVE permission. Because from mysql version 3.2 onwards, it is possible to perform dual-machine hot backup function operations through REPLICATION.
The operation instructions are as follows:
mysql> grant replication slave on *.* to 'replicate'@'218.206.70.146' identified by '123456';
mysql> flush privileges;
Note: 218.206.70.146 here can be replaced with %, which means that any ip can connect to the main server.
After creating the synchronous connection account, we can use the replicat account on the slave server (Slave) to access the master server (Master) database to see if the connection is successful.
Enter the following command on the slave server (Slave):
[root@YD146 ~]# mysql -h59.151.15.36 -ureplicate -p123456
If the following results appear, it means that the login is successful, indicating that the two servers can be operated in dual-system hot backup.
2.2.2 Modify the mysql configuration file
If the above preparations are done well, then we can modify the mysql configuration file. First, find all the mysql configuration files in the directory. Generally, after the mysql service is installed, the configuration files will be copied one by one and placed in / ect directory, and the configuration file is named: my.cnf. That is, the exact directory of the configuration file is /etc/my.cnf
After finding the configuration file my.cnf and opening it, modify it under [mysqld]:
[mysqld]
server-id = 1
log-bin=mysql-bin
log-err=mysql-err
binlog-do-db = test
binlog-ignore-db = mysql
2.2.3 Restart mysql service
After modifying the configuration file, save it and restart the mysql service. If it succeeds, there is no problem.
2.2.4 View the status of the main server
After entering the mysql service, you can view the status of the Master through the command, enter the following command: show master status
Pay attention to the parameters inside, especially the first two File and Position, which will be useful in configuring the master-slave relationship on the slave server (Slave).
Note: The lock table is used here to prevent new data from entering the environment, so that the slave server can locate the synchronization position. After the initial synchronization is completed, remember to unlock it.
2.3 Slave configuration from the server
2.3.1 Modify the configuration file
Because the mysql dual-machine hot backup is implemented in a master-slave manner, there is no need to create a synchronization account on the slave server. You can directly open the configuration file my.cnf and modify it. The reason is the same as modifying the master server. , but the parameters that need to be modified are different. as follows:
[mysqld]
server-id = 2
log-bin=mysql-bin
replicate-do-db = test
replicate-ignore-db = mysql,information_schema,performance_schema
2.3.2 Restart mysql service
After modifying the configuration file, save it and restart the mysql service. If it succeeds, there is no problem.
2.3.3 Use the change master statement to specify the synchronization position
This step is the most critical step. After entering the mysql operation interface, enter the following command:
mysql>stop slave; //Stop the slave service thread first, this is very important, otherwise the following operations will fail.
mysql>change master to
>master_host='59.151.15.36',master_user='replicate',master_password='123456',
> master_log_file=' mysql-bin.000016 ',master_log_pos=107;
Note: master_log_file, master_log_pos are determined from the status value detected by the master server (Master). That's what I just called attention to. master_log_file corresponds to File, and master_log_pos corresponds to Position. Versions above Mysql 5.x do not support specifying options related to the main server in the configuration file.
If the problem encountered, the following situation still occurs after following the above steps:
Then you need to reset the slave. The command is as follows
mysql>stop slave;
mysql>reset slave;
Then stop the slave thread and start over. After success, you can start the slave thread.
mysql>start slave;
2.3.4 View the status of the slave server (Slave)
Use the following command to view
mysql> show slave status\G;
Check that the following two values are Yes, which means that the slave server is set successfully.
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
If there is: Slave_IO_Running: Connecting problem, it means that the connection is unreachable. Find the problem from the following aspects:
1. The network is unreachable: check the firewall, IP
2. The password is wrong
3.pos is not correct
4. ID problem
2.4 Test Synchronization
It has been said before that there is only one table tb_mobile in the database test without data. We can first check whether the databases of the next two servers have data:
Master:59.151.15.36
Slave:218.206.70.146
Ok, now you can insert data into the Master server to see if it can be synchronized.
Master:59.151.15.36
Slave:218.206.70.146
It can be seen from the above two screenshots that the data inserted on the Master server can be found on the Slave server, which means that the dual-machine hot backup configuration is successful.
3. Mysql establishes the master-master server dual-machine hot standby configuration steps
The server still uses the current two servers
3.1 Create a sync user
At the same time, establish a connection account on the master-slave server, which must grant REPLIATION SLAVE permission. Here, because server A and server B are masters and slaves of each other, a synchronization user must be established respectively.
ServerA:
mysql> grant replication slave on *.* to 'replicate'@'218.206.70.146' identified by '123456';
mysql> flush privileges;
serverB:
mysql> grant replication slave on *.* to 'replicate'@'59.151.15.36' identified by '123456';
mysql> flush privileges;
3.2 Modify the configuration file my.cnf
Server A
[mysqld]
server-id = 1
log-bin=mysql-bin
binlog-do-db = test
binlog-ignore-db = mysql
#Main - the main form needs to add more parts
log-slave-updates
sync_binlog = 1
auto_increment_offset = 1
auto_increment_increment = 2
replicate-do-db = test
replicate-ignore-db = mysql,information_schema
serverB:
[mysqld]
server-id = 2
log-bin=mysql-bin
master-slave need
replicate-do-db = test
replicate-ignore-db = mysql,information_schema,performance_schema
#Main - the main form needs to add more parts
binlog-do-db = test
binlog-ignore-db = mysql
log-slave-updates
sync_binlog = 1
auto_increment_offset = 2
auto_increment_increment = 2
3.3 Restart the mysql service on server A and server B respectively
The way to restart the server is the same as above, so I won’t explain it here .
3.4 Check the status of server A and server B as the main server respectively
ServerA:
serverB:
3.5 Use change master to specify the synchronization location on server A and server B respectively
ServerA:
mysql>change master to
>master_host='218.206.70.146',master_user='replicate',master_password='123456',
> master_log_file=' mysql-bin.000011 ',master_log_pos=497;
serverB:
mysql>change master to
>master_host='59.151.15.36',master_user='replicate',master_password='123456',
> master_log_file=' mysql-bin.000016 ',master_log_pos=107;
3.6 Restart the slave service thread on the A and B servers respectively
mysql>start slave;
3.7 Check the slave server status on the A and B servers respectively
mysql>show slave status\G;
Check that the following two values are Yes, which means that the slave server is set successfully.
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
3.8 Test Master-Master Synchronization Example
Test server A:
Insert a statement on server A as shown in the following figure:
Then check whether it is synchronized on server B, as shown in the figure below:
Test server B:
Insert a statement on server B as shown in the following figure:
Then check whether there is synchronized data on the slave server A, as shown in the figure below:
Finally, it can be seen from the results that the dual-system hot backup in the form of master-master can be successfully realized.
4. Description of configuration parameters
Server-id
ID values uniquely identify the master and slave servers in a replication cluster, so they must be different. Master_id must be a positive integer value between 1 and 232-1, and the value of slave_id must be a positive integer value between 2 and 232-1.
Log-bin
Indicates that the binlog is turned on. Only when this option is turned on can I/O be written to the relay-log of the Slave, which is also a prerequisite for replication.
Binlog-do-db
Indicates the database that needs to record binary logs. If there are multiple data can be separated by commas, or use multiple binlog-do-dg options.
Binglog-woman-db
Indicates the database that does not need to record binary logs. If there are multiple databases, they can be separated by commas, or use multiple binglog-ignore-db options.
Replicate-do-db
Indicates the database that needs to be synchronized. If there are multiple data, you can separate them with commas, or use multiple replicate-do-db options.
Replicate-ignore-db
Indicates the databases that do not need to be synchronized. If there are multiple databases, they can be separated by commas, or use multiple replicate-ignore-db options.
Master-connect-retry
master-connect-retry=n means that the connection between the slave server and the master server is not successful, then wait for n seconds (s) before proceeding with the management method (the default setting is 60s). If the slave server has a mater.info file, it will ignore these options.
Log-slave-updates
Configure whether the update operation on the slave library is written to the binary file. If this slave library is to be the master library of other slave libraries, then this parameter needs to be set so that the slave library of the slave library can perform log synchronization.
Slave-skip-errors
During the copying process, due to various reasons, the sql in binglo fails. By default, the copying from the database will stop and the user needs to intervene. You can set slave-skip-errors to define the error number. If the error encountered during the replication process is the defined error number, you can pass it. If the slave library is used for backup, setting this parameter will cause data inconsistency, so do not use it. It can be considered if it is to share the query pressure of the main library.
Sync_binlog=1 Or N
The default value of Sync_binlog is 0. In this mode, MySQL will not synchronize to disk. In this case, Mysql relies on the operating system to refresh the binary log binary log, just like the mechanism for the operating system to refresh other files. So if the operating system or machine (not just the Mysql server) crashes, it is possible that the last statement in the binlog is lost. To prevent this, you can use the sync_binlog global variable to synchronize the binlog with the hard disk after every N binlog writes. It is safest when the sync_binlog variable is set to 1, because in the case of a crash, your binary log binary log may only lose at most one statement or one transaction. However, this is also the slowest way (unless the disk has a cache cache with battery backup, making syncing to disk very fast).
Even if sync_binlog is set to 1, in the event of a crash, there may be inconsistencies between table contents and binlog contents. If InnoDB tables are used, the Mysql server processes the COMMIT statement, it writes the entire transaction to the binlog and commits the transaction to InnoDB. If there is a crash between two operations, when restarting, the transaction is rolled back by InnoDB, but it still exists in the binlog. The -innodb-safe-binlog option can be used to increase the consistency between InnoDB table contents and binlog. (Note: -innodb-safe-binlog is not required in Mysql 5.1 version; this option is obsolete due to the introduction of XA transaction support), this option can provide a greater degree of security, so that the binlog of each transaction (sync_binlog=1 ) and (true by default) the InnoDB log is synchronized with the hard disk. The effect of this option is that when restarting after a crash, after the transaction is rolled back, the Mysql server cuts the rolled back InnoDB transaction from the binlog. This ensures that the binlog feeds back the exact data of InnoDB tables, etc., and keeps the slave server in sync with the master server (does not receive rolled back statements).
Auto_increment_offset和Auto_increment_increment
Auto_increment_increment and auto_increment_offset are used for master-to-master replication and can be used to control the operation of AUTO_INCREMENT columns. Both variables can be set as global or local variables, and each value is assumed to be an integer value between 1 and 65,535. Setting one of the variables to 0 makes that variable 1.
These two variables affect how AUTO_INCREMENT columns are: auto_increment_increment controls how much the values in the column are incremented, and auto_increment_offset determines the starting point for AUTO_INCREMENT column values.
If the value of auto_increment_offset is greater than the value of auto_increment_increment, the value of auto_increment_offset is ignored. For example: if there is already some data in the table, the existing maximum self-increment value will be used as the initial value.