A switching from a main plurality correctness
In front of 24, 25 and 26 of the first article, I introduce you to the infrastructure and MySQL master and slave replication, but these are the structure of a main one.
Most Internet application scenarios are reading and writing less, so you responsible business, is likely to have problems reading performance in the development process. Read and solve performance problems at the database level, it is necessary to involve the next two articles to discuss the frame
structure: a master multi-slave.
Today this article, we will first talk about the correctness of a master multi-switch from the. Then, in the next article and then we talk about the correctness of the solution query logic from a multi-master's.
1, the basic structure of a multi-master
As shown in FIG. 1 is a basic structure from a multi-master.
FIG 1 a basic structure from a multi-master
Drawings, dashed arrows indicate the standby relationship, i.e. A and A 'mutual backup from library B, C, D points to the primary database A. Provided from a multi-master, generally used for separate read and write, the host library responsible for all write and read a portion of, the other
read request from the library by balancing.
2, standby switching
That is, from the multi-architecture, the main fault after the main library in a standby switch main problem we have to discuss today. As shown in FIG, 2 is the master database failure occurs, the result of the standby switch.
FIG 2 from the basic structure of a multi-master - standby switching
Compared to a main one handover procedure, from a master multi-structure after the handover is completed, A 'will be the new primary library, from the library B, C, D have to be changed to A'. It is because more library from B, C, D redirect this process, the standby
switch corresponding increase in complexity.
Next, we'll take a look at how a switching system will complete the switchover process from a multi-master's.
Second, based on the main site switchover
Here, we need to review a knowledge point.
When we set the node B to the node A 'from the library, when need to perform a change master command:
CHANGE MASTER TO MASTER_HOST=$host_name MASTER_PORT=$port MASTER_USER=$user_name MASTER_PASSWORD=$password MASTER_LOG_FILE=$master_log_name MASTER_LOG_POS=$master_log_pos
1, change master command arguments detailed
This command has so six parameters:
MASTER_HOST, MASTER_PORT, MASTER_USER and MASTER_PASSWORD four parameters represent the main reservoir A ', IP, port, user name and password.
The last two parameters MASTER_LOG_FILE and MASTER_LOG_POS said master_log_pos master_log_name log files from the main library in this position to continue synchronization. And this position is what we call the synchronization site, which is the main library and log file names corresponding to the offset.
Well, here there is a problem, the Node B to set A 'from the library, it is necessary to perform change master command, will inevitably have to set the parameters of these two sites, but in the end should be how these two parameters set it?
A Node B is turned from the site of the library, A, is recorded locally. However, the same log, the site of A and A 'are different sites. Therefore, when the library from B to switch, you need to go through "to find a synchronization site," this logic.
2, the site is difficult to get accurate, it can only take an approximate location. Why do you say?
I look at this and analyze your site is generally how to get, you know what the cause of the inaccuracy. Considering the switching process can not lose the data, so we find sites, always looking for a "slightly ahead" of
And then by determining whether the transaction has been performed skip over the library from B.
1. A method for synchronization site is such that:
A method for synchronization site is such that:
1. Wait for new primary database A 'to the log transfer (relay log) of all synchronization is complete;
2. A' show master status command execution, obtained on the current A 'File date and the Position;
3. A library of origin to take fault time T;
4. a parsing tool with mysqlbinlog apos File, to give time T site.
mysqlbinlog File --stop-datetime=T --start-datetime=T
FIG output section 3 mysqlbinlog
FIG behind the end_log_pos value "123" is represented by A 'in this example, write the new time T binlog position. Then, we can put 123 this value as $ master_log_pos, with the node B
Change Master's command.
2, such a process takes a value not precise. why?
It is contemplated that you have such a situation, it is assumed that at time T, A master repository has completed execution of the INSERT statement to insert a row of data R, and has passed the binlog A 'and B, and then transmitted at the moment the main End a library of
host on the lost power.
Well, this time the state of the system is as follows:
1. B from the upper reservoir, since the synchronization of the binlog, R already exists in this line;
2. the new primary database A ', R already exists in this line also, after the log 123 is written in this position;
3. We execute commands from the change master library B, point a '123 file location of the file, it will insert the binlog this line R and the data to perform synchronization from the library to the B.
At this time, the synchronization thread library B will report Duplicate entry 'id_of_R' for key'PRIMARY ' error, prompted a primary key violation, then stop synchronization.
3, under normal circumstances, when we switch tasks, the first initiative to skip these errors, there are two commonly used methods.
An approach is to take the initiative to skip a transaction. Skip command wording is:
set global sql_slave_skip_counter=1; start slave;
Because the switching process may be repeated more than execute a transaction, so we need to receive at the beginning of the new primary database A 'from the library B, continuous observation, every encounter these errors stopped, perform a skip command, until the stop is no longer present
situation down, in order to skip all the transactions that may be involved.
2, another way, by setting the parameter slave_skip_errors, provided directly to skip a specified error.
Another way is by providing the slave_skip_errors parameters provided directly to skip a specified error. When performing the switchover, there are two types of such errors is often encountered:
1062 is the only key conflict error when data is inserted;
1032 is not found error when deleting rows of data.
Therefore, we can slave_skip_errors set to "1032,1062," so when you skip across the middle of these two errors.
It should be noted that this specified error skip method, when for the switchover, since no exact synchronization site, this method can only be created from the library, and new master library master and slave relationship.
Against this background, we are very clear in the switchover process, skip 1032 and 1062 these two types of errors are non-destructive, so it can be so set slave_skip_errors parameters. Wait until synchronization relationship between master and slave is established, and
After a period of steady execution, we need to put this parameter is set to empty, so there was really after from inconsistent master data, also skipped.
Three, GTID
By skipping sql_slave_skip_counter affairs and ignored by slave_skip_errors wrong way, although both eventually establish standby relations and the new primary database from the library B A ', but two operations are very complex and prone to
error. So, MySQL 5.6 version introduces GTID, completely solved the problem.
So, GTID in the end what it meant, and how to solve synchronization sites to find the problem? Now, I am with you briefly explain.
GTID stands for Global Transaction Identifier, is the global transaction ID, a transaction is generated at the time of submission, is the unique identifier for the transaction. It consists of two parts, the format is:
GTID=server_uuid:gno
1, GTID in the end what does this mean?
among them:
server_uuid is automatically generated when a first start example, a globally unique value; GNO is an integer, the initial value of 1, each time to commit the transaction is assigned to the transaction, and add 1.
I need you here and explain, in the official MySQL documentation, GTID format is so defined:
GTID=source_id:transaction_id
source_id here is server_uuid; and behind this transaction_id, I think are misleading, so I changed it gno. Why use transaction_id likely to cause misunderstanding it?
Because, in MySQL which we say transaction_id refers to the transaction id, transaction id is assigned during the execution of a transaction, if the transaction is rolled back, the transaction id will be incremented and the gno is only allocated when the transaction is committed.
In effect, GTID tend to be continuous, so we are represented by gno easier to understand.
Start GTID mode is also very simple, we only need to start a MySQL instance, we add parameters gtid_mode = on and enforce_gtid_consistency = on it.
2, how to solve synchronization sites to find the problem?
In GTID mode, each transaction will be followed by a GTID correspondence. This GTID There are two ways to generate, and which method to use depends on the value of session variables gtid_next.
1. If gtid_next = automatic, the representative default value. At this time, MySQL will put server_uuid: gno assigned to this transaction.
. A binlog when recording the first recording line @@ SESSION.GTID_NEXT = the SET 'server_uuid: GNO';
B GTID added GTID this present example set.
2. If the value is a specified gtid_next GTID, such as by set gtid_next = 'current_gtid' designated current_gtid, then there are two possibilities:
. a GTID If current_gtid already exists in the collection instance, the next execution of the transaction will be ignored by the system directly;
. current_gtid if B does not exist in the collection GTID instance, will be assigned to this current_gtid transaction to be performed next , that is to say the system does not need to generate new transaction GTID, and therefore do not add gno 1.
Note that a transaction can only give a current_gtid use. After the transaction commits, if you want to execute the next transaction, it is necessary to perform the set command, set the gtid_next another gtid or automatic.
Thus, each MySQL instance maintains a GTID set to correspond to "the example of all executed transactions."
This does not look easy to understand, then I will use a simple example to illustrate basic usage and you GTID's.
We created a table in the instance X, t.
CREATE TABLE `t` ( `id` int(11) NOT NULL, `c` int(11) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB; insert into t values(1,1);
FIG 4 binlog initialization data
We can see, there is a command SET @@ SESSION.GTID_NEXT before BEGIN transaction. At this time, if there are instances X from the library, and then insert the CREATE TABLE statement binlog synchronization performed in the past, then, the Executive
Prior to the transaction will be executed first line of the two SET command, and it was added that the figure of two GTID collection from GTID library.
Suppose, now this is another example of Example X Y from the library, and at this time performs the following insert statements in this Example Y:
insert into t values(1,1);
Further, in this statement GTID are examples of Y "aaaaaaaa-cccc-dddd-eeee-ffffffffffff: 10".
Then, examples of X as Y from the library, this transaction will come to perform synchronization, obviously there will be a primary key conflict, leading to synchronize threads examples of X stop. Then, how should we deal with it?
Approach is that you can perform the following sequence of statements:
set gtid_next='aaaaaaaa-cccc-dddd-eeee-ffffffffffff:10'; begin; commit; set gtid_next=automatic; start slave;
Among them, the role of the first three statements, by submitting a Space Affairs, added this GTID examples of X GTID collection. 5, show master status is the result after the empty transaction been performed.
FIG 5 show master status results
You can see examples of X Executed_Gtid_set inside, I have joined this GTID.
In this way, I'll do start slave command allows simultaneous threads execution time up, although the examples of X or Y will continue to pass over the affairs example, but because of "aaaaaaaa-cccc-dddd-eeee-ffffffffffff: 10" already exist
That GTID collection instance of X, so X instance will skip this transaction, the primary key conflict error will not occur again.
In the above sequence of statements before the start slave command also has a set gtid_next = automatic. The role of this sentence is "to restore GTID default allocation behavior," that there is a new transaction again after execution if it
is in accordance with the original distribution, continue to allocate gno = 3
Fourth, the standby switch based on the main GTID
Now, we have to understand the concept GTID, and then take a look at the usage-based backup copy of the master GTID. In GTID mode, the standby database B to set new primary database A 'from a library grammar is as follows:
CHANGE MASTER TO MASTER_HOST=$host_name MASTER_PORT=$port MASTER_USER=$user_name MASTER_PASSWORD=$password master_auto_position=1
Wherein, master_auto_position = 1 would indicate that the master and slave relationship is used GTID protocol. can be seen,
In front of us a headache MASTER_LOG_FILE and MASTER_LOG_POS parameters do not need specified.
We put this moment, examples of A 'is referred to as a set GTID SET_A, GTID set referred to as Example B set_b. Next, we look at the current standby switching logic.
1, we execute the command on the example start slave B, taking binlog logic is this:
Example B 1. The main library designated A ', based on a connection standby protocol.
2. Example B set_b sent to the master database A '.
Example 3. A 'is calculated with the difference set SET_A set_b, that is present in all SET_A, but not in the set GITD set_b judges A' binlog local transaction contains all the required difference sets.
. a If not, represents A 'Example B had the need to delete the binlog directly return an error;
B confirm if all included, A.' from their inside binlog file, find the first transaction is not set_b issued B;
4. After you start from this transaction, the subsequent reading documents, issued the order to take binlog B to perform.
In fact, this logic which contains a design idea: based GTID the standby relationship, as long as the system standby build relationships, we must ensure that the log sent to the main library library equipment is complete. Thus, if the required instance already log B
was absent, A 'refusal to issue the log B.
This is different from the primary and protocol-based sites. Site-based protocol, is determined by the library apparatus, which specifies the library prepared site, the primary site database on the hair which, determined not to log integrity.
2, based on the above description, let us look at the introduction GTID, at a handover scenario from a multi-master, standby switching is how to achieve.
It is not necessary to find sites, so from the library B, C, D are performed only change master command points to an instance A 'can.
In fact, strictly, the switchover does not need to look for is not the site, but to find the job site, in the example A 'has the inside done automatically. But because this work is automatic, so the developers HA system, it is very friendly.
After this the system 'written, the host library A' of the new primary database binlog A self-generated in the format set GTID: server_uuid_of_A ': 1-M.
If the previously set format from the repository B GTID server_uuid_of_A: 1-N, then after the handover GTID format set becomes server_uuid_of_A: 1-N, server_uuid_of_A ': 1-M.
Of course, the primary database A 'before the library is prepared by A, the main library A' and B from the library GTID set is the same. We met our expectations.
Five, GTID and online DDL
Next, I give you an example to help you understand GTID.
Methods harm than good "to improve the performance of the previous" What have MySQL "In the first 22 articles? When "I and query performance issues you mentioned peak periods slow, to analyze if the index is due to lack of performance problems caused, we can pass
through the Caucasus led to solve online. However, taking into account the impact of the new index to avoid the main cause of database performance, we can prepare Ku Jiasuo lead in the first, then switch.
I said, in a dual structure M, DDL statements prepared by the library will be passed to the implementation of the main library, in order to avoid affecting the return to the main library, through set sql_log_bin = off to turn off binlog.
Comments District One student raises a question: if this operation, which is added to the database index, but binlog did not record this update, is not it will lead to inconsistent data and log?
I mention this issue very well. At that time, I was quoted in the message GTID reply to illustrate. Today, I would like you to expand and explain.
Suppose that the two mutual backup mode library examples and examples of X or Y, and X is the current primary database, and are open GTID mode. At this time the standby switching process can become so following:
- Stop slave performed on the instance X.
- Execute DDL statements on the instance Y. Note that this does not need to close binlog.
- After execution, found this DDL statements corresponding GTID, and recorded as server_uuid_of_Y: gno.
- Examples of X to execute the following sequence:
set GTID_NEXT="server_uuid_of_Y:gno"; begin; commit; set gtid_next=automatic; start slave;
The purpose of this is to either let the updated instance Y are binlog record, but also ensures that no update on the implementation of this instance X.
Next, a switchover is complete, then follow the above-described process then execute it again.
VI Summary
In today's article, I will introduce you to the main and standby switching process a master multi-slaves. In this process, from the library to find a new main library site is a sore point. Thus, we GTID mode leads to the introduction of MySQL 5.6 version introduces the
basic concepts and usage of GTID.
It can be seen in GTID mode, switching from a multi-master is very convenient.
Therefore, if you are using a version of MySQL support GTID then, I suggest you try to do more than use GTID mode is switched from a primary.
In the next article, we can see the application GTID separate read and write mode in the scene.
Finally, we went to the Questions of time.
When you set up a master-slave relationship in GTID mode, after the execution start slave command from the library, main library needed to find out binlog has been deleted, resulting in standby creation is unsuccessful. In this case, you can think how to deal with it?
You can write your methods in the comments section, I will end next article and you discuss this issue. Thank you for listening, you are welcome to send this share to more friends to read together.
Seven, the issue of time
Finally the last article, I'll keep the problem is that if the main library are single-threaded pressure mode, the main library in the process of recovery from the library, binlog-transaction-dependency-tracking what parameters should be chosen?
The answer to this question is that this parameter should be set to WRITESET.
Since the main library is single-threaded pressure mode, so commit_id each transaction is different, then set COMMIT_ORDER mode, it can only be single-threaded execution from the library.
Similarly, since the model requires WRITESET_SESSION standby database when the application logs, log with a thread must be the same sequence performed on the primary database will cause degradation threaded into threaded replication master repository pressure mode.
So, binlog-transaction-dependency-tracking should be set to WRITESET.
Comments Guest Book thumbs board:
@ Hui Xin coming asked a good question, on the same line as updated several transactions, if commit_id the same time prepare the library is not in the parallel execution will lead to inconsistent data? The Q
answer the question is to update the same row of the transaction is not possible to commit the same time to enter the state.
@ Yang comrades to this question gives a more detailed answer, you can take a look.