Mysql master-slave architecture construction and maintenance details

introduction

  This article is an article about MySQL master-slave replication configuration. You will gain an in-depth understanding of how to build a MySQL master-slave configuration, a powerful tool that can provide greater efficiency and security for database operations. This article will cover everything from basic environment preparation, to configuration of master-slave servers, to optimization and expansion of master-slave replication.

1. Introduction to master-slave replication

1.1 What is MySQL master-slave replication

  MySQL master-slave replication is an asynchronous replication technology for replicating data from one MySQL database (master database) to one or more MySQL databases (slave databases). This mechanism makes the slave database reach the same data state as the master database. The main workflow is that the primary database writes data change events to a binary log (binlog), and then reads and executes events in those logs from the database, thereby keeping it in sync with the primary database.

1.2 Advantages and uses of master-slave configuration

The advantages and uses of MySQL master-slave replication configuration are very diverse:

1. Data backup: Through master-slave replication, data is copied to one or more slave databases, providing a disaster recovery solution to prevent data loss.
2. Read performance improvement: For read-intensive applications, the read load can be distributed by reading from the database, thereby improving system performance.
3. Data distribution: Data can be replicated to multiple geographically distributed databases to provide users with faster local access services.
4. Real-time analysis and reporting: The slave database can be used for complex queries and report generation without affecting the performance of the master database.

2. MySQL master-slave replication principle

  MySQL master-slave replication is based on the binary log (binlog) of the master database. The primary database records all data change events in the binary log. There are two threads used for replication from the database: the IO thread and the SQL thread. These threads read and execute events in the binary log so that the slave reaches the same data state as the master.

2.1 Workflow

The workflow of master-slave replication mainly includes the following steps:

1. The primary database performs data-changing operations such as INSERT, UPDATE, or DELETE.
2. The primary database records these changes in the binary log.
3. The slave database's IO thread connects to the master database and reads new events in the binary log.
4. The IO thread of the slave database writes these events to the relay log of the slave database.
5. The SQL thread of the slave database reads the relay log and executes the events in it to synchronize the data with the main database.

2.2 Data synchronization method

  MySQL master-slave replication uses an asynchronous method for data synchronization. That is to say, after the master database records an event in the binary log, it does not wait for the slave database to execute the event, but returns immediately. It is its own responsibility to fetch and execute new events from the database. This approach improves performance in most cases, but may result in data delays or data loss in case of network outages or slave database failures.
  There is also a semi-synchronous replication method, in which the master database waits for at least one slave to receive a change event before returning after performing a change. This approach can improve data reliability, but may affect the performance of the primary database.

3. Environment preparation

  Before configuring MySQL master-slave replication, we need to prepare the corresponding environment. This includes meeting hardware and software requirements, selecting the appropriate MySQL version, and installing the MySQL server correctly.

3.1 Hardware and software requirements

• Hardware requirements: First, you need at least two computers (you can create two virtual machines for demonstration during testing), one as the master server, and the other or more as slave servers. These computers need to have sufficient processor speed, memory size, and hard disk space to support the operation of the MySQL server and data storage.

• Software requirements: An operating system such as Linux, Windows, or macOS needs to be installed on each computer. The operating system should be installed and updated to the latest stable version. In addition, some basic software tools are needed, such as the client that comes with mysql or use Navicat.

3.2 MySQL version selection

  It is also important to choose the appropriate MySQL version. Different versions of MySQL may support different features and have different performance characteristics and problems. In general, we recommend using the latest stable releases, as these contain the latest features and recent security fixes. However, in some cases, you may need to use a specific version of MySQL for specific needs or compatibility. This article will use the latest version ( mysql8.0.33 ) to build.

3.3 Install MySQL service

Install two servers with mysql service, please refer to this tutorial for the installation of mysql service: https://blog.csdn.net/dougsu/article/details/130816827

4. Configure the MySQL master server

After preparing the hardware and software environment, we need to configure the MySQL master server. This includes modifying configuration files, creating replication accounts, and starting and testing the master server. (Below we will use the Navicat tool to act as a client for query and operation)

4.1 Configuration file modification

First, we need to modify the MySQL configuration file my.cnf (my.ini on Windows). Under the **[mysqld]** section, we need to add or modify the following configuration items:

server-id=1
#开启binlog
log_bin=master-bin
log_bin-index=master-bin.index
binlog-format=MIXED

• server-id : Set a unique integer ID for each MySQL server.
• log-bin : Enable binary logging and specify the name of the log file.
• log_bin-index : the name of the binary log file index file
• binlog-format : Specifies the format of the binary log, which can be STATEMENT, ROW or MIXED.

The format of the binary log is supplemented

1. STATEMENT: MySQL will record each executed SQL statement into the binary log. The binary log only contains the executed SQL statement and does not record specific row-level changes. It is suitable for most cases, but may be in some cases Consistency issues leading to master-slave synchronization
2. ROW: MySQL will record the change information of each modified row into the binary log. The binary log will contain the actual row-level change information, not just the SQL statement, which can ensure the precise consistency of the master-slave synchronization, but will generate Larger binary log files
3. MIXED: MySQL can automatically choose to record SQL statements or row-level change information according to specific circumstances. For most cases, MySQL will use the STATEMENT format to record SQL statements, but for some situations where SQL statements cannot be accurately replayed, MySQL will use ROW The format records row-level change information, so this format combines the advantages of STATEMENT and ROW, and the most appropriate recording method can be selected in different situations.

4.2 Create a copy account

In the actual production environment, it is not recommended to use the root user directly, but create a user with full permissions to be responsible for master-slave synchronization. Use the following SQL command to create the replication account reUser , the password is abc123

CREATE USER 'reUser'@'%' IDENTIFIED BY 'abc123';
GRANT REPLICATION SLAVE ON *.* TO 'reUser'@'%';
flush privileges;

After creating a user, you can use the following command to query the configured permissions, and the query should include permissions: REPLICATION SLAVE

SHOW GRANTS FOR 'reUser'@'%';

Note : You can directly use the root account in the construction test, omitting to create a user

4.3 Starting and testing the master server

1. Restart the mysql service after modifying the configuration
2. The client executes SHOW VARIABLES LIKE 'server_id';, if the configuration is normal, it will be displayed
   
3. The client executes show master status;, if the configuration is normal, it will be displayed
   

Field explanation:

• File : Indicates the name of the binary log file currently being written.
• Position : Indicates the position (byte offset) in the binary log file currently being written.
• Binlog_Do_DB : Indicates the list of databases that enable binary logging for the master server, and only record the specified database operations.
• Binlog_Ignore_DB : Indicates the list of databases to ignore when binary logging is enabled for the master server, and the specified database operations are not logged.
• Executed_Gtid_Set : Indicates the GTID (Global Transaction Identifier) ​​set that has been executed on the master server. GTID is an identifier used to track distributed transactions.

Note : The Binlog_Do_DB, Binlog_Ignore_DB, and Executed_Gtid_Set fields may be empty in some cases, depending on your configuration and usage.

5. Configure MySQL slave server

The process of configuring a MySQL slave server is similar to configuring a master server, including modifying configuration files and starting a test server.

5.1 Configuration file modification

The MySQL configuration file my.cnf (my.ini on Windows) needs to be modified. Under the **[mysqld]** section, we need to add or modify the following configuration items:

server-id=2
#打开MySQL中继日志
relay-log=slave-relay-bin
relay-log-index=slave-relay-bin.index
#打开从服务二进制日志
log-bin=mysql-bin
#使得更新的数据写进二进制日志中
log-slave-updates=1

• server-id : set a unique integer ID for each MySQL server
• relay-log : Specifies the file name prefix used to store relay logs from the server (Slave)
• relay-log-index : Specifies the file name used to store the relay log index from the server (Slave)
• log-bin : Enable binary logging and specify the name of the log file.
• binlog-format : Specifies whether the slave server writes replication events to its own binary log. When set to 1, the slave server will write the received binary log events of the master server into its own binary log, that is, in Replication events are recorded in both the secondary log and the binary log. This option is usually used to implement cascading replication and multi-level replication in a configuration based on chain replication between master and slave servers.

5.2 Starting and testing the slave server

1. Restart the mysql service after modifying the configuration
2. The client executes SHOW VARIABLES LIKE 'server_id';, if the configuration is normal, it will display 2

6. Establish a master-slave connection

6.1 Connecting from the server to the master server

  After configuring the master and slave, we need to create a connection on the slave pointing to the master and start the replication process. This needs to use the MySQL CHANGE MASTER TOcommand, and provide the address of the master server, the username and password of the replication account, and the binary log file and location, so my configuration is as follows

CHANGE MASTER TO
MASTER_HOST='192.168.3.51',
MASTER_PORT=3306,
MASTER_USER='root',
MASTER_PASSWORD='123456',
MASTER_LOG_FILE='master-bin.000001',
MASTER_LOG_POS=157,
GET_MASTER_PUBLIC_KEY=1;

• master_host: is the address of the master server
• replication_user and replication_password: are the username and password of the replication account created on the primary server.
• master_log_file and master_log_pos: It is the name and location of the binary log of the master server, which can be obtained by executing the command on the master server SHOW MASTER STATUS(refer to Section 4.3 above).

The execution results are as follows:

6.2 Start the replication process

  After completing the above steps, the replication process has not yet started. In order to start the replication process, you need to execute the following command on the slave server:

START SLAVE;

After replication is started, the slave server will start to read the binary logs of the master server and execute the events in these logs locally, thereby keeping the data in sync with the master server.
You can view the status of the replication with the SHOW SLAVE STATUS command.

1. If the replication process is running, both the Slave_IO_Running and Slave_SQL_Running status should show "YES".
2. If the synchronization has been completed or no synchronization is required, the results of Master_Log_File and Read_Master_Log_Pos are the same as those queried in Chapter 4.3.

Query replication information can also be accessed through the built-in client show slave status \G;, the result:

Note : If you are cloning a virtual machine that has installed mysql, you need to modify the value of server-uuid in the data/auto.cnf file in the root directory of mysql . The values ​​of the two services must be different, otherwise the following error will be reported

Fatal error: The replica I/O thread stops because source and replica have equal MySQL server UUIDs; these UUIDs must be different for replication to work.

7. Verify that master-slave replication works

  After configuring MySQL master-slave replication and starting the replication process, we need to verify that replication is working properly. A common way to verify master-slave replication is to insert data on the master and then check on the slave that it was received.

7.1 Insert data to the main server for verification

On the master server, we can create a new database and insert some data. For example:

CREATE DATABASE mydb;
USE mydb;
CREATE TABLE user (id INT AUTO_INCREMENT PRIMARY KEY, name VARCHAR(50));
INSERT INTO user(name) VALUES ('tom'), ('jerry'), ('dave');

1. master-slave query

SELECT * FROM `user`

2. Query from the library

SELECT * FROM `user`

It is found that the data has been synchronized, and the latest status information is queried on the slave library:
it is found that the synchronized offset Read_Master_Log_Pos: 1097 has been modified

7.2 Error checking and problem solving

  During the process of configuring and verifying MySQL master-slave replication, you may encounter some problems. The most common problems are that the slave cannot connect to the master, or that the slave cannot read or execute the master's binary log.

  When encountering a problem, the first thing you should check is the MySQL error log to see if there are any related error messages. Additionally, SHOW SLAVE STATUSthe replication status of the slave can be viewed using the command. The result of this command contains a lot of useful information, such as the address of the master server, the username of the replica account, the error number and error message of the replica, etc.

  Based on the error message, you can determine the cause of the problem and take appropriate corrective action. Common solutions include: checking the network connection, confirming that the user name and password of the account are correct, confirming that the binary log of the master server is available, etc.

  Verifying the work of master-slave replication is an important step in configuring MySQL master-slave replication, and it is also the key to keeping master-slave replication running normally. Data consistency and availability can only be ensured when replication is working properly.

8. Maintenance and optimization of master-slave replication

  During the operation of master-slave replication, some problems may arise, which require specific processing and optimization solutions.

8.1 Network problem handling

  Network issues are probably the most common failure in master-slave replication. Once the network connection is unstable, it will directly affect the synchronization of data. Dealing with network problems requires multiple perspectives, including checking network connections, network device status, network firewall rules, and the listening port of the MySQL server. In a complex network environment, the assistance of the network administrator may be required to debug and optimize the network link.

8.2 Replication delay problem

  Replication delay, that is, the lag in which the slave database synchronizes the data of the master database, is another important factor affecting the effect of master-slave replication. Insufficient processing power, or insufficient network bandwidth, can cause replication delays. Dealing with replication delay can be done by optimizing database operations, enhancing hardware performance of slave libraries, optimizing network bandwidth, etc. In some cases, you can consider introducing middleware to use its data distribution capabilities to reduce the pressure on slave libraries.

8.3 Master-slave switching

  Master-slave switching is an important part of the high-availability solution, that is, when the master library fails, the slave library is quickly promoted to the new master library to ensure service continuity. The process of master-slave switching needs to be fully planned and tested in advance, including determining the trigger conditions for switching, the specific steps of switching, and the processing of services during the switching process. When performing master-slave switching, you also need to pay attention to data consistency to avoid data loss during the switching process.

The maintenance and optimization of master-slave replication is an important means to ensure data consistency and high service availability, which requires database administrators to have solid professional knowledge and rich practical experience.

9. Advanced Themes and Extensions

  As the business develops, more complex requirements may be encountered, such as multi-source replication, master-master replication, automatic switching, and load balancing.

9.1 Multi-source replication

  In MySQL, a slave server can replicate data from multiple master servers, which is called multi-source replication. Multi-source replication can improve data availability and read performance. Configuring multi-source replication requires creating a replication channel on the slave for each master and specifying the details of the master for each channel.

9.2 Master-Master Replication

  Master-master replication, also known as two-way replication, means that two servers are each other's master and slave. Master-master replication can improve data redundancy and availability, but it also increases the risk of replication conflicts. In order to avoid conflicts, we need to carefully design the write mode of the database, for example, each server only writes specific data.

9.3 Automatic switching and load balancing

  Automatic switching refers to automatically promoting a slave server to be the new master server when the master server fails. Automatic switching can improve service availability, but it needs to rely on additional tools or services, such as MHA, MySQL Router, etc.
  Load balancing refers to distributing read requests to multiple slave servers to improve read performance. Load balancing can be implemented through hardware devices (such as load balancers), software tools (such as proxy servers), or in applications.
  Advanced topics and extensions require the operator to have more in-depth database knowledge, as well as more practical experience. When using these advanced features, you need to consider their benefits and potential risks in order to make the best decision.

10. Conclusion

  This article discusses MySQL's master-slave replication configuration and its construction in detail, hoping to help you understand and implement this important database management strategy. The article starts with the basic concepts and principles of MySQL master-slave replication, and then deeply introduces how to prepare the environment, configure the master server and slave server, and how to establish a master-slave connection. In order to ensure the smooth progress of master-slave replication, we also discussed how to verify master-slave replication and solve possible problems. At the same time, the article also explains the maintenance and optimization of master-slave replication, as well as some advanced topics. I hope that through this article, you can have a deep understanding of MySQL's master-slave replication and improve your database management capabilities.