mysql master copy principle and solve problems
https://blog.csdn.net/darkangel1228/article/details/80003967
mysql master-slave replication principle
Primary database will change biglog write log, the master generates a log dump thread library, from the library used to binlog i / o threads pass;
Library is generated from two threads, one I / O thread, a thread SQL;
I / o binlog thread to request the primary database, and writes the obtained log binlog relay log (relay log) file;
SQL thread, reads the relay log log file, and parsed into specific operation to achieve consistent master-slave operation, and consistent final data;
Database synchronization procedure from the main library of the serial data , that is to say in the main parallel library operations, performed in serial from the library. So this is a very important point, because the implementation of SQL features from the main library as well as a copy of the log from the library serial, under high concurrency scenarios, from the library's main library data would certainly be slower, there is a delay.
Another problem is that if the primary database goes down suddenly, and not just the data synchronized to the library, some data may not be from the library, some data may be lost.
Herein generally used parallel copying, master-slave synchronization to solve the delay problem; semi-sync, to solve the data loss problem main library .
Parallel copying , referring to the open library from multiple threads, parallel read relay log log different libraries, and then replay the logs in parallel in different libraries, which is the library-level parallelism.
Semi-sync (semi-sync copy) after the primary library refers binlog after writing the log, this time will be immediately forced to synchronize data from the library, from the library to write the log own local relay log, then will returns an ack to the main library, main library receives at least one from the library would think after ack write operation is complete .
mysql master-slave synchronization, the general advice is to read much more than write and read time when the general time-critical data is not so high, with a mysql master-slave synchronization, to be followed immediately necessary for the kind of written guarantee check the scene, by way of forced read main library.
Generally, the more serious if the delay from the master
1, sub-libraries, one master library split into four main library, concurrent writes to each of the main library 500 / s, this time delay from the master negligible
2, open parallel copy mysql support, multiple parallel library copy, if concurrent writes a library is particularly high, single library to write concurrent reached the 2000 / s, parallel to copy or meaningless. 28 law, very often say, is that the few orders table, written in 2000 / s, dozens of other tables 10 / s.
3, rewrite the code, students write the code, be careful when we are actually short term is to get the students to rewrite the code, and then insert the data directly on the update, do not query
4, if indeed it exists must be inserted, immediately requested to inquire into, then we must immediately turn to do something, set directly connected to the main database query. Do not recommend this method, you do so results in meaning to read and write separation is lost.
mysql cluster
After the sub-library sub-table problems and coping strategies
Affairs
Distributed Transaction
Reference: About distributed transaction, two-phase commit, a phase commit, research 1PC mode and compensation mechanism of Best Efforts Affairs advantage based on two-phase commit, the maximum guarantee of the "atomic" cross-database operations, distributed under the system the most stringent implementation of the transaction. Simple, small workload. Since most application servers as well as some independent Distributed Transaction Coordinator done a lot of work package, so as to introduce a distributed transaction projects in difficulty and workload basically negligible. Drawback system "horizontal" retractable rivals. You need to be based on two-phase commit distributed transactions when submitting transactions across multiple nodes to coordinate and maximize time to commit the transaction after the point of pushing to extend the execution time of the transaction on the objective, which causes the transaction to access shared the probability of conflict and deadlock when resources increased with the increase in database node, this trend will become increasingly severe, making the system at the database level, horizontal scaling of "yoke", which is a distributed system does not use a lot of Sharding the main reason for the transaction. Best Efforts 1PC transaction-based model
Reference implementation of spring-data-neo4j. Given the performance benefits of Best Efforts 1PC mode, as well as relatively simple implementation, it is most sharding framework and project adoption
Transaction compensation (equivalent power)
For those who requires a high performance, but does not require a high system consistency, consistency is often not demanding real-time systems, as long as you can reach the final consistency within a permitted time period, which makes the transaction compensation mechanism to be a viable option.
Transaction compensation mechanism was first proposed in dealing with "long transaction" in, but for distributed systems to ensure consistency also has a good reference value. Generally speaking, different error occurs immediately after the rollback of the transaction in execution mode, transactions compensation is an ex post checks and remedial measures, it is only expected to get final results within a consistent period of time to allow it. Closely related issues of implementation and compensation systems business, and no standard approach. Some common implementations are: data reconciliation checks; based on the log for comparison; periodically synchronized with the standard data sources, etc.
https://baijiahao.baidu.com/s?id=1622441635115622194&wfr=spider&for=pc