1. CAP theorem and BASE theory
CAP Theorem The
CAP principle, also known as the CAP theorem, refers to the fact that in a distributed system
- Consistency (Consistency)
In the distributed system, all data backups are the same value at the same time (equivalent to all nodes accessing the same copy of the latest data) - Availability (Avaliability)
After some nodes in the cluster fail, whether the cluster as a whole can respond to client read and write requests (high availability for data updates) - Partition tolerance (Partition tolerance)
Most distributed systems are distributed in multiple sub-networks, and each self-network is called a partition. Partition fault tolerance means that inter-area communication may fail. For example, if one server is placed in China and the other is placed in the United States, these are two zones, and communication between them may not be possible.
The principle of CAP is that these three elements can only satisfy two points at most, and it is impossible to give consideration to all three.
Generally speaking, partition fault tolerance cannot be avoided, so we can think that the P of CAP always holds. The CAP theorem tells us that the remaining C and A cannot be done at the same time.
Raft algorithm for distributed consistency
http://thesecretlivesofdata.com/raft/
For most Internet application scenarios, there are many hosts, scattered deployments, and clusters are getting larger and larger, so node failures and network failures are normal, and service availability must reach 99.999%, that is, P and A are guaranteed, and C is abandoned.
2. The problems faced
For most Internet application scenarios, there are many hosts, scattered deployments, and clusters are getting larger and larger, so node failures and network failures are normal, and service availability must reach 99.999%, that is, P and A are guaranteed, and C is abandoned.
2. BASE theory
It is an extension of CAP. The idea is that strong consistency cannot be achieved (the consistency of CAP is strong consistency), but appropriate weak consistency, that is, final consistency, can be used
BASE means
Basically available (Basically Avaliable)
Basically available means that when a fault occurs in a distributed system, it is allowed to lose part of the availability (listed in response time, functional availability) and allowed to lose part of the availability. It should be noted that the basic availability is not equivalent to
the loss of response time when the system is unavailable . Under normal circumstances, the search engine needs to return the corresponding query results to the user within 0.5 seconds, but due to a failure (such as a part of the system room power failure) Network failure), the query result response time has increased to 1~2 seconds.
Functional loss, shopping sites during the double eleven shopping peak, in order to ensure the stability of the system, some consumers will be introduced to a degraded page
soft state (Soft State )
Soft state refers to allowing the system to have an intermediate state, and the intermediate state will not affect the overall availability of the system. In distributed storage, a piece of data generally has multiple copies, and the delay that allows different copies to synchronize is a manifestation of soft state. The asynchronous replication of mysglreplication is also a manifestation.
Eventual Consistency (Eventual Consistency) Final consistency
refers to the fact that all data copies in the system can finally reach a consistent state after a certain period of time. Weak consistency is the opposite of strong consistency, and final consistency is a special case of weak consistency.
4. Strong consistency, weak consistency, and final consistency
From the perspective of the client, when multiple processes access concurrently, different strategies for how to obtain updated data in different processes determine different consistency. For relational databases, it is required that the updated data can be seen by subsequent visits, which is a strong consistency. If it can tolerate that some or all of the subsequent access is not available, it is weak consistency. If it is required to be able to access the updated data after a period of time, it is eventually consistent
Several schemes of distributed transactions
1, 2PC mode
The database supported 2PC [2 phase commit 二phase submission], also called XA Transactions
MySQL has been supported since version 5.5, Sql Server 2005 has been supported, and Oracle7 has been supported
Among them, XA is a two-phase submission protocol, which is divided into two phases:
The first stage: The transaction coordinator requires each database involved in the transaction to precommit this operation, and reflect whether it can be submitted
The second stage: the transaction coordinator requires each database to submit data
Among them, if any database denies this commit, then all databases will ask to roll back their part of the information in this transaction,
2. Flexible transaction-TCC transaction
The so-called TCC mode refers to the support to incorporate custom branch transactions into the management of global transactions.
- One-stage prepare behavior: call custom prepare logic.
- Two-stage commit behavior: call custom commit logic.
- Two-stage rollback behavior: call custom rollback logic. The so-called TCC mode refers to the support to incorporate custom branch transactions into the management of global transactions.
3. Flexible business-best-effort notification-based solution
Notifications are made according to the rules, and there is no guarantee that the data will be notified successfully, but a queryable operation interface will be provided for verification. This solution is mainly used when communicating with third-party systems, such as: notification of payment results after calling WeChat or Alipay. This solution is also implemented in conjunction with MQ, for example: sending http requests through MQ and setting the maximum number of notifications. Once the number of notifications is reached, there will be no more notifications.
Cases: bank notifications, merchant notifications, etc. (merchant notifications between major transaction business platforms: multiple notifications, check proofreading, and reconciliation documents), Alipay's payment success asynchronous callback
4. Flexible transaction-reliable information + eventual consistency plan (asynchronous notification type)
Realization: The business processing service sends a message to the real-time message service request before the business transaction is submitted. The real-time message service only records the message data, not the actual transmission. After the business transaction is submitted, the business processing service confirms the delivery to the real-time message service. The real-time message service will only be sent after receiving the confirmation sending instruction.