CAP is the first letter of Consistency, Availability, and Partition Tolerance. Different materials have slightly different interpretations of these three words, such as IBM Cloud Docs/Wikipedia/Baidu Baike, etc. The definition of CAP theory has also changed. The explanation in the first edition said it was 对于一个分布式计算系统而言，不可能同时满足一致性、可用性和分区容错三个设计规约; and the explanation in the second edition became在一个分布式系统（指互相连接并共享数据的节点的集合）中，当涉及读写操作时，只能保证一致性、可用性和分区容错三者中的两个，另一个必须被牺牲
However, it is not difficult to see from the explanation that the distributed system discussed in CAP theory emphasizes interconnection and data sharing, which means that for example, Memcache cluster is not the object of CAP theory discussion, because it does not have the characteristics of interconnection and data sharing, while MySQL cluster CAP theory is the object of discussion; CAP theory also mentions that when read and write operations, it means that it focuses on read and write operations rather than the full functions of distributed systems
Consistency
The first version of the explanation is that all nodes can see the same data at the same time. The second version of the explanation is that for a specified client, the read operation is guaranteed to return the latest write operation result. The difference is very obvious:
- The first edition considers the problem from the perspective of the node, and the second edition considers the problem from the perspective of the client
- The first version considers the data owned by the node and the second version considers the client read and write perspective
- The first edition emphasizes that nodes have the same data at the same time, while the second edition does not emphasize such a statement. In fact, for system transactions, the system is actually in an inconsistent state during the execution of the transaction, and the data of different nodes is not complete. Consistent, so the second edition emphasizes that the client read operation can obtain the latest write result, because the client cannot read the uncommitted data during the execution of the transaction. The client can only read the uncommitted data after the transaction is submitted. The data written by the transaction will be rolled back if the transaction fails, and the client will not read the data written in the middle of the transaction
Availability
The first edition emphasizes that every request can be responded with success or failure, and the second edition emphasizes that non-faulty nodes return reasonable responses (not error and timeout responses) within a reasonable time. The difference is also obvious:
- The first edition emphasizes every request, and the second edition emphasizes A non-failing node, which is obviously more rigorous, because the failed node may not be able to get a response. Only the existence of the non-failed node can meet the availability requirements.
- The first edition emphasizes that response is divided into success and failure, and the second edition emphasizes reasonable response and reasonable time, and emphasizes no error or timeout
Partition Tolerance Partition Tolerance
The first edition emphasizes that the system can continue to operate despite the loss of messages or partition errors. In the second edition, it emphasizes that the system can continue to "perform its duties" after a network partition occurs. The difference is very obvious:
- The first edition emphasizes system operation, but the second edition emphasizes that the system can still perform its duties under the premise of system operation
- The first edition emphasizes message loss or partial failure, and the second edition directly uses network partitions. The message loss (packet loss) mentioned in the first edition is just a kind of network obstacle. The second edition directly talks about the phenomenon, that is, it happens. Partition phenomenon, regardless of the reason, may be packet loss, network interruption, or network congestion, as long as the network partition is caused
CAP choice
If it is said that CAP can only choose two and must give up one and consider it in a distributed environment, then P is a necessary element, because the network itself cannot be 100% reliable. In theory, it is impossible to choose a CA architecture for a distributed system. Choose CP and AP architecture
CP-Consistency/Partition Tolerance
In order to ensure consistency, when the network partition occurs, the data on node N1 has been updated to Y, but due to the network interruption between N1 and N2, data Y cannot be synchronized to N2, and the data on node N2 is still X. At this time, the customer When the client accesses N2, it needs to return an error indicating that the client system has failed. This processing method violates the availability principle
AP-Availability/Partition Tolerance
In order to ensure availability, when the partition phenomenon occurs, the data on node N1 has been updated to Y, but due to the interruption of replication between N1 and N2, data Y cannot be synchronized to N2, and the data on N2 is still X. At this time, the client access At N2, N2 returns the current data X that it owns to the client, but in fact the current latest data is already Y, which cannot satisfy the consistency Consistency.
CAP details
The advantage of the theory is that it is clear and concise and easy to understand, but the disadvantage is that it is highly abstract and omits many details. As a result, misunderstandings and deviations may occur due to various complex situations when applying the theory to practice. CAP theory is no exception. If we need to apply CAP theory in practice, if we are not aware of these key details, we may find it difficult to implement the plan
The granularity of CAP's attention is data, not the entire system
In the definition and explanation of CAP theory, system-level concepts such as system and node are used. This has caused a lot of misleading to many people, thinking that when we are designing the architecture, the entire system should either choose CP or choose AP. But in the actual design process, each system cannot handle only one type of data, but contains multiple types of data. Some data must choose CP, and some data must choose AP. If we design, from the entire If you choose CP or AP from the system perspective, you will find that one loses the other, no matter how you do it, there are problems.
- Take a simplest user management system as an example. The user management system contains user account data (user ID, password), user information data (nickname, interests, hobbies, gender, self-introduction, etc.). Normally, user account data will be Choose CP, and user information data will choose AP. If the entire system is defined as CP, it does not meet the application scenario of user information data; if the entire system is defined as AP, it does not meet the application scenario of user account data
- Therefore, when CAP theory is put into practice, we need to classify the data in the system according to different application scenarios and requirements, and choose a different strategy (CP or AP) for each type of data, instead of directly limiting all data in the entire system to the same strategy
CAP ignores network delay
This is a very implicit assumption. Brewer did not take delay into consideration when defining consistency. In other words, when the transaction is committed, the data can be instantly replicated to all nodes. But in reality, copying data from node A to node B always takes a certain amount of time. If it is the same computer room, it may take a few milliseconds; if it is a computer room that spans different locations, it may take tens of milliseconds. This means that C in CAP theory cannot be perfectly implemented in practice. In the process of data replication, the data of node A and node B are not consistent
Don't underestimate the inconsistency of these milliseconds or tens of milliseconds. Technically, it is impossible to achieve perfect consistency in a distributed scenario. The business must require consistency. Therefore, for example, the balance of a single user or the inventory of a single product requires the choice of CP in theory, but in practice CP cannot do it. You can only choose CA, that is, you can only write in a single point. Other nodes do backups, and multi-point writing cannot be achieved in a distributed situationIt should be noted that this does not mean that this type of system cannot apply distributed architecture. It just means that "single user balance and single commodity inventory" cannot be distributed, but the entire system can still apply distributed architecture.
For example, a common distributed architecture that partitions users is shown in the following figure:
we can store data with user id 0-100 in Node1, and store data with user id 101-200 in Node2, and Client decides based on user id Which Node to visit, for a single user, read and write operations can only be performed on a certain node; for all users, some users read and write operations on Node1, and some users read and write operations on Node2
An obvious problem with this design is that when a node fails, users on this node cannot perform read and write operations. But on the whole, this design can reduce the number of users affected when the node fails. And scope, after all, it is better to affect only 20% of users than to affect all users. This is also the reason why only some users of Alipay will have business abnormalities after the excavator cuts the optical cable, rather than all users.
Under normal operating conditions, there is no choice of CP and AP, and CA can be satisfied at the same time
CAP theory tells us that distributed systems can only choose CP or AP, but in fact, the premise here is that the system has "partitioned". If there is no partition phenomenon in the system, that is to say, when P does not exist (the network connection between nodes is normal), there is no need to give up C or A. Both C and A should be guaranteed. This requires both consideration when designing the architecture. Choose CP or AP when partition occurs, and also consider how to ensure CA when partition does not occur
Also take the user management system as an example. Even if CA is implemented, different data implementation methods may be different: User account data can be implemented in a "message queue" way to achieve CA, because message queues can better control real-time performance, but It is more complicated to implement; and user information data can be implemented by "database synchronization", because the database method may have a high delay in some scenarios, but it is simple to use
Giving up does not mean doing nothing, you need to prepare for partition recovery
The CAP theory tells us that only two of the three can be taken, and the other one needs to be "sacrificed". In fact, the "sacrifice" of the CAP theory only means that we cannot guarantee C or A during the partitioning process, but it does not mean everything. Don’t do it, because most of the time during the entire operating cycle of the system is normal, and the partition phenomenon does not take long
For example, a system with 99.99% availability (commonly known as four nines) will only be unavailable for 50 minutes after one year of operation, and a 99.999% availability (commonly known as five nines) system will be unavailable for only 5 minutes after one year of operation. . Abandoning C or A during the partition does not mean giving up C and A forever. We can perform some operations during the partition so that the system can reach the state of CA again after the partition failure is resolved.
The most typical is to record some logs during the partition. When the partition failure is resolved, the system restores the data based on the logs to make it reach the CA status again. Also take the user management system as an example. For user account data, assuming CP is selected, the partition After the occurrence, node 1 can continue to register new users, node 2 cannot register new users (here is the reason for not meeting A, because node 2 will return error after receiving the registration request), at this time node 1 can register new but not synchronized The user to node 2 is recorded in the log. When the partition is restored, node 1 reads the record in the log and synchronizes to node 2. When the synchronization is completed, node 1 and node 2 have reached the state of satisfying CA at the same time
For user information data, suppose we select AP. After the partition occurs, both node 1 and node 2 can modify user information, but the modification may be different on both sides. For example, node 1 and node 2 both record unsynchronized hobby data , When the partition is restored, the system merges the data according to a certain rule, or it can completely report the data conflict, and manually choose which one should be used
ACID/BASE
ACID
ACID is a theory put forward by the database management system to ensure the correctness of transactions. ACID contains four constraints. The basic explanation is as follows
- Atomicity (atomicity): All operations in a transaction are either completed or not completed at all, and will not end in an intermediate link. If an error occurs during the execution of the transaction, it will be rolled back to the state before the transaction started, as if the transaction had never been executed
- Consistency: Before the start of the transaction and after the end of the transaction, the integrity of the database has not been destroyed
- Isolation C: The ability of the database to allow multiple concurrent transactions to read, write and modify data at the same time. Isolation can prevent data inconsistency caused by cross execution when multiple transactions are executed concurrently. Transaction isolation is divided into different levels, including read uncommitted (read uncommitted), read committed (read committed), repeatable read (repeatable read) and serializable (Serializable)
- Durability: After the transaction is completed, the modification of the data is permanent and will not be lost even if the system fails.
It is not difficult to see that A (Atomicity) in ACID and A (Availability) in CAP have completely different meanings. Although the names of C in ACID and C in CAP are consistent, they have completely different meanings. C refers to the data integrity of the database, the C in CAP refers to the data consistency in the distributed nodes, combined with the application scenario of ACID is database transactions, CAP focuses on the difference of distributed system data reading and writing. In fact, the comparison between CAP and ACID is similar
BASE
BASE is the abbreviation of Basically Available, Soft State and Eventually Consistency. The core idea is that even if strong consistency cannot be achieved (CAP consistency is strong consistency) , But the application can use a suitable way to achieve eventual consistency (Eventual Consistency)
BASE refers to basically available (Basically Available), soft state (Soft State), eventual consistency (Eventual Consistency)
Basically Available
When a distributed system fails, it is allowed to lose part of the availability, that is, to ensure that the core is available. The keywords here are "part" and "core". Which ones are specifically selected as businesses that can be lost and which are businesses that must be guaranteed are one item Challenging job
Soft State
Allow the system to have an intermediate state, and the intermediate state will not affect the overall availability of the system. The intermediate state here is the data inconsistency in the CAP theory.
Eventual Consistency
After a certain period of time, all data copies in the system can finally reach a consistent state. The keywords here are "a certain time" and "final". "A certain time" is strongly related to the characteristics of the data, and different data can be tolerated. The inconsistency time is different; "Final" means that no matter how long it takes, it will eventually reach a consistent state
The BASE theory is essentially an extension and supplement to CAP, more specifically, it is a supplement to the AP scheme in CAP. When analyzing the CAP theory, I mentioned two points related to BASE:
- The CAP theory ignores the delay, and the delay in practical applications is unavoidable. This means that there is no perfect CP scene, even if it is a data replication delay of a few milliseconds, within this few milliseconds interval, The system does not meet the CP requirements, so the CP scheme in the CAP actually achieves final consistency, but the "certain time" refers to a few milliseconds.
- The sacrifice of consistency in the AP solution only refers to the partitioning period, rather than giving up consistency forever. This is actually the extension of the BASE theory. Consistency is sacrificed during the partitioning, but after the partition failure recovery, the system should reach final consistency.
Based on the above analysis, ACID is the theory of database transaction integrity, CAP is the theory of distributed system design, and BASE is the extension of AP scheme in CAP theory.