In the actual production environment, the reading and writing of the database are all in the same database server, which cannot meet the actual needs. Whether in terms of security, high availability, or high concurrency, it is completely unable to meet actual needs. Therefore, the data is synchronized through master-slave replication, and the concurrent load capacity of the database is improved through read-write separation. It is somewhat similar to rsync, but the difference is that rsync backs up disk files, while mysql master-slave replication backs up data and statements in the database.

1. The principle of MySQL master-slave replication

1.1, MySQL replication type

statement-based replication
row-based replication
mixed-type replication

1.2. The working process of MySQL master-slave replication

1.3. Replication types supported by mysq

STATEMENT: Statement-based replication. Execute the sql statement on the server, and execute the same statement on the slave server. Mysql uses statement-based replication by default, which has high execution efficiency.
ROW: Row-based replication. Copy the changed content instead of executing the command on the slave server.
MIXED: Mixed types of replication. By default, statement-based replication is used. Once it is found that statement-based replication cannot be accurately replicated, row-based replication will be used.

1.4. Data flow direction

Master-slave replication Learn why replication????

Guaranteeing Data Integrity Who Copy Who?

Where does the slave role copy the data of the master role?

Among the binary log files, mwsql-bin.000001 ------> record the complete sqslave to copy the binary log to the local node, save it as a relay log file, and finally perform a "recovery" operation based on this relay log, which will be executed SQL is executed synchronously inside its own database, and finally achieves consistency with the master data

dump：

Used to monitor I/O thread requests

Send the updated data in the binary log to the slave I/O thread

1.5. Working process of master-slave replication

(1) The Master node records the change of data into a binary log (bin lg), and when the data on the Master changes, it writes the change into the binary log
The slave node will detect whether the binary log of the Laster has changed within a certain time interval. If there is a change, it will start a /0 thread to request the binary event of the aster.
At the same time, the Master node starts a dump thread for each I/0 thread to send binary events to it, and saves it to the local relay log (Relaylog) of the Slave node, and the Slave node will start the SOL thread to read from the relay log The binary log is replayed locally, that is, it is parsed into sgL statements and executed one by one, so that its data is consistent with that of the Master node. Finally, the I/0 thread and the SOL thread will go to sleep and wait for the next wake-up. Synchronous and asynchronous replication

Notes: ● Relay logs will usually be located in the OS cache, so the overhead of relay logs is small.

●The replication process has a very important limitation, that is, the replication is serialized on the Slave, that is to say, the parallel update operation on the Master cannot be performed in parallel on the Slave.

1.6, synchronous replication and asynchronous replication

asynchronous replication

Synchronous replication:

semi-synchronous replication

Synchronous replication, semi-synchronous replication and asynchronous replication are common data replication technologies in database systems, which are used to maintain data consistency and availability among multiple database instances.

1. Synchronous Replication (Synchronous Replication): In synchronous replication, when the primary database (source database) performs write operations, it waits for all slave databases (target databases) to confirm that these write operations have been successfully received and applied before returning to the client end. Only after all slaves have successfully received and applied the same write operation can the master proceed to the next step. This approach ensures that all slave databases maintain full data consistency with the master database under any circumstances. However, synchronous replication has relatively low performance due to the need to wait for acknowledgment from the slave database, and may be affected by network delays and failures.

2. Semi-Synchronous Replication: Semi-Synchronous Replication is a compromise between synchronous and asynchronous replication. In semi-synchronous replication, after the primary database performs a write operation, it waits for at least one slave database to confirm receipt and apply the write operation before returning to the client. This confirmation process can be completed through network transmission. Semi-synchronous replication provides higher performance compared to synchronous replication because the master does not need to wait for acknowledgment from all slaves. However, semi-synchronous replication still guarantees data consistency between at least one slave database and the master database.

3. Asynchronous replication (Asynchronous Replication): In asynchronous replication, after the primary database performs a write operation, it does not wait for the confirmation from the secondary database, but returns it to the client immediately. The slave database will replicate the write operations of the master database as soon as possible, but there may be a delay in time. Asynchronous replication has high performance and is not limited by network delays and failures. However, due to a certain time difference between the master database and the slave database, the data consistency of asynchronous replication may be reduced.

In summary, synchronous replication provides the highest data consistency, but relatively low performance; semi-synchronous replication balances performance and data consistency; asynchronous replication provides the highest performance, but data consistency may vary reduce. Choosing the right data replication technology depends on the needs of the application and the trade-offs between data consistency and performance.

2. Separation of reading and writing

2.1. What is read-write separation?

The basic principle of read-write separation is to let the main database handle transactional addition, modification, and deletion operations (INSERT, UPDATE, DELETE), while the slave database handles SELECT query operations. Database replication is used to synchronize changes caused by transactional operations to slave databases in the cluster.

2.2. Why separate reading and writing?

Because the "write" operation of the database (it may take 3 minutes to write 10,000 pieces of data) is time-consuming. But the "reading" of the database (reading 10,000 pieces of data may only take 5 seconds). Therefore, the separation of reading and writing, the solution is that the writing of the database affects the efficiency of the query.

2.3. When to separate read and write?

The database does not necessarily have to be separated from reading and writing. If the program uses the database a lot, but there are few updates and many queries, it will be considered. Using database master-slave synchronization, and then through read-write separation can share the database pressure and improve performance.

2.4, MySQL read and write separation principle

Read-write separation means only writing on the master server and reading only on the slave server. The basic idea is to have the master database handle transactional operations, while the slave database handles select queries. Database replication is used to synchronize changes caused by transactional operations on the master database to the slave databases in the cluster.

2.5. Classification of read-write separation:

1. Based on the internal implementation of the program code

In the code, routing is classified according to select and insert. This method is also the most widely used in the production environment. The advantage is that the performance is better, because it is implemented in the program code, and there is no need to add additional equipment for hardware expenses; the disadvantage is that developers need to implement it, and operation and maintenance personnel have no way to start.

However, not all applications are suitable for implementing read-write separation in the program code. Like some large-scale and complex Java applications, if the read-write separation is implemented in the program code, the code changes will be relatively large.

2. Implementation based on the intermediate proxy layer

The agent is generally located between the client and the server. After receiving the request from the client, the agent server forwards it to the back-end database after passing the judgment. The following representative procedures are shown.

MySQL-Proxy. MySQL-Proxy is an open source project of MySQL, which can judge SQL through its own lua script.
Atlas. It is a data middle layer project based on the MySQL protocol developed and maintained by the infrastructure team of the Web platform department of Qihoo 360. It is based on mysql-proxy version 0.8.2, optimized and added some new features. 360 internally uses the mysql business run by Atlas, which carries billions of read and write requests every day. Supports things as well as stored procedures.
Amoeba. Developed by Chen Siru, the author once worked for Alibaba. The program is developed in the Java language, and Alibaba uses it in the production environment. But it does not support transactions and stored procedures.

Since the use of MySQL Proxy requires writing a large number of Lua scripts, these Lua are not ready-made, but need to be written by yourself. This is very difficult for people who are not familiar with MySQL Proxy built-in variables and MySQL Protocol.

Amoeba is a very easy to use, very portable software. Therefore, it is widely used in the proxy layer of the database in the production environment.

3. Realize master-slave replication

turn off safety mechanism

3.1, the time setting is consistent

Install NTP

Configure NTP

restart service

Set from server

Same time as master server

The operation of the two servers is the same

3.2. MySQL configuration of the master server

then reboot

The File column shows the log name, and the Position column shows the offset, which must be the same

Slave server mysql configuration file

The configuration of the two slave servers is the same

The main server enters the database and creates the library

Can be queried from the server

4. Read and write separation

4.1. Install java environment

Decompression complete

modify file name

Add in the vim /etc/profile configuration file

4.2. Install Amoeba software

Successful installation

One master and two slave servers are authorized to the amoba server at the same time

Modify the amoeba configuration file

4.3. Modify the database configuration file

/usr/local/oeba/bin/oeba start& #Start Amoeba software, press ctrl+c to return

Check whether port 8066 is enabled

4.4. Test read-write separation

Create table on master server

from server 1

from server 2

The main server creates data

client view data

Because it is a round-robin algorithm

The client will read data from slave1 and slave2 respectively, and only the data added on the two slave servers will be displayed, not the data added on the master server.

Client creates data

Can only be viewed on the master server

The master server can also only view written data

Execute start slave on the two slave servers; the data added on the master server can be synchronized

MySQL master-slave replication and read-write separation and its examples

Master-slave replication and read-write separation