database transaction
Before talking about distributed transactions, let's start with database transactions. Database transactions may be familiar to everyone and are often used in the development process. But even so, many people may still be unclear about some details. For example, many people know several characteristics of database transactions: Atomicity, Consistency, Isolation, and Durability, or ACID for short. But further down, for example, when you ask what isolation refers to, you may not know, or you know what isolation is, but then ask what levels of isolation are implemented in the database, or what are the differences between them at each level? may not know when.
This article does not intend to introduce these database transactions, if you are interested, you can search for relevant information. However, there is a knowledge point we need to understand, that is, if the database suddenly loses power when committing a transaction, how does it recover? Why mention this knowledge point? Because the core of the distributed system is to deal with various abnormal situations, which is also the complex part of the distributed system, because the distributed network environment is very complex, and this kind of "power failure" failure is much more than that of a single machine, so we are doing distributed system. This is the first thing to consider when setting up the system. These anomalies may be machine downtime, network anomalies, message loss, out-of-order messages, data errors, unreliable TCP, loss of stored data, other anomalies, etc...
Let's go on to talk about the situation where the local transaction database is powered off. How does it ensure data consistency? We use SQL Server as an example, we know that we are using SQL Server database is composed of two files, a database file and a log file, usually, the log file is much larger than the database file. When the database performs any write operation, the log must be written first. For the same reason, when we execute a transaction, the database will first record the redo operation log of the transaction, and then start to actually operate the database. Write the log file to the disk, then when there is a sudden power failure, even if the operation is not completed, when the database is restarted, the database will perform undo rollback or redo rollback according to the current data situation, which ensures the strong data. consistency.
Next, let's talk about distributed transactions.
distributed theory
When the performance of our single database has a bottleneck, we may partition the database. The partition mentioned here refers to the physical partition. After partitioning, different libraries may be on different servers. At this time, the ACID can no longer adapt to this situation, and in this ACID cluster environment, it is almost difficult to ensure the ACID of the cluster, or even if it can be achieved, the efficiency and performance will be greatly reduced. The most important thing is that it is difficult to achieve. The new partition is expanded. At this time, if we pursue the ACID of the cluster, our system will become very poor. At this time, we need to introduce a new theoretical principle to adapt to the situation of this cluster, which is the CAP principle or the CAP theorem. , then what does the CAP theorem mean?
CAP theorem
The CAP theorem was proposed by Professor Eric Brewer of the University of California, Berkeley, who pointed out that WEB services cannot satisfy the following three properties at the same time:
Consistency: The client knows that a series of operations will happen at the same time (effective)
Availability: Every operation must end with a predictable response
Partition tolerance: Even if a single component is unavailable, the operation can still be completed
Specifically, in a distributed system, in any database design, a Web application can only support at most two of the above properties at the same time. Obviously, any scale-out strategy relies on data partitioning. Therefore, designers must choose between consistency and usability.
This theorem holds true in distributed systems so far! Why do you say that?
At this time, some students may move the 2PC (two-phase submission) of the database to speak. OK, let's take a look at the two-phase commit of the database.
Students who have an understanding of database distributed transactions must know the 2PC supported by the database, also known as XA Transactions.
MySQL has been supported since version 5.5, SQL Server 2005 has been supported, and Oracle 7 has been supported.
Among them, XA is a two-phase commit protocol, which is divided into the following two phases:
Phase 1: The transaction coordinator asks each database involved in the transaction to precommit this operation and reflects whether it can be committed.
Phase 2: The transaction coordinator asks each database to commit data.
Among them, if any one database vetoes the commit, then all databases will be asked to roll back their part of the information in this transaction. What's the downside of this? At first glance we can get consistency across database partitions.
If the CAP theorem is true, then it must affect availability.
If the availability of a system represents the sum of the availability of all components involved in performing an operation. Then in the two-phase commit process, the availability represents the sum of the availability in each database involved. We assume that each database has 99.9% availability during the two-phase commit process, then if the two-phase commit involves two databases, the result is 99.8%. According to the system availability calculation formula, assuming 43,200 minutes per month, 99.9% availability is 43,157 minutes, and 99.8% availability is 43,114 minutes, which is equivalent to an increase of 43 minutes of downtime per month.
Above, it can be verified that the CAP theorem is theoretically correct. We will see CAP here first, and we will talk about it later.
BASE theory
In distributed systems, we often pursue availability, and its important program is higher than consistency, so how to achieve high availability? The predecessors have proposed another theory for us, that is, the BASE theory, which is used to further expand the CAP theorem. BASE theory refers to:
Basically Available
Soft state
Eventually consistent
The BASE theory is the result of a trade-off between the consistency and availability in CAP. The core idea of the theory is: we cannot achieve strong consistency, but each application can adopt an appropriate method according to its own business characteristics to make the system achieve Eventual consistency .
With the above theory, let's look at the problem of distributed transactions.
Distributed transaction
In a distributed system, to implement distributed transactions, there are no more than those solutions.
1. Two-phase commit (2PC)
Like the database XA transaction mentioned in the previous section, the two-phase commit is based on the principle of using the XA protocol. We can easily see some details in the middle such as commit and abort from the process of the following figure.
Two-phase commit is a solution that sacrifices some availability for consistency. In terms of implementation, in .NET, two-phase submission in distributed systems can be programmed with the help of the API provided by TransactionScop. For example, this part of the function is implemented in WCF. However, between multiple servers, it is necessary to rely on DTC to complete transaction consistency. Microsoft has MSDTC service under Windows, and it is more tragic under Linux.
In addition, TransactionScop cannot be used for transaction consistency between asynchronous methods by default, because the transaction context is stored in the current thread, so if it is an asynchronous method, you need to explicitly pass the transaction context.
Advantages: It ensures strong data consistency as much as possible, and is suitable for key areas that require high data consistency. (Actually, there is no 100% guarantee of strong consistency)
Disadvantages: complex implementation, sacrificing usability, great impact on performance, not suitable for high-concurrency and high-performance scenarios, if the distributed system is called across interfaces, there is currently no implementation solution in the .NET world.
2. Compensation Transaction (TCC)
TCC is actually the compensation mechanism adopted. Its core idea is: for each operation, a corresponding confirmation and compensation (revocation) operation must be registered. It is divided into three stages:
The Try phase is mainly to test the business system and reserve resources
The Confirm phase is mainly to confirm and submit the business system. When the Try phase is successfully executed and the Confirm phase is started, the default Confirm phase will not go wrong. That is: as long as the Try is successful, the Confirm must be successful.
The Cancel stage is mainly to cancel the execution of the business in the state of business execution error and need to be rolled back, and release the reserved resources.
举个例子，假入 Bob 要向 Smith 转账，思路大概是：
我们有一个本地方法，里面依次调用
1、首先在 Try 阶段，要先调用远程接口把 Smith 和 Bob 的钱给冻结起来。
2、在 Confirm 阶段，执行远程调用的转账的操作，转账成功进行解冻。
3、如果第2步执行成功，那么转账成功，如果第二步执行失败，则调用远程冻结接口对应的解冻方法 (Cancel)。
优点： 跟2PC比起来，实现以及流程相对简单了一些，但数据的一致性比2PC也要差一些
缺点： 缺点还是比较明显的，在2,3步中都有可能失败。TCC属于应用层的一种补偿方式，所以需要程序员在实现的时候多写很多补偿的代码，在一些场景中，一些业务流程可能用TCC不太好定义及处理。
三、本地消息表（异步确保）
本地消息表这种实现方式应该是业界使用最多的，其核心思想是将分布式事务拆分成本地事务进行处理，这种思路是来源于ebay。我们可以从下面的流程图中看出其中的一些细节：
基本思路就是：
消息生产方，需要额外建一个消息表，并记录消息发送状态。消息表和业务数据要在一个事务里提交，也就是说他们要在一个数据库里面。然后消息会经过MQ发送到消息的消费方。如果消息发送失败，会进行重试发送。
消息消费方，需要处理这个消息，并完成自己的业务逻辑。此时如果本地事务处理成功，表明已经处理成功了，如果处理失败，那么就会重试执行。如果是业务上面的失败，可以给生产方发送一个业务补偿消息，通知生产方进行回滚等操作。
生产方和消费方定时扫描本地消息表，把还没处理完成的消息或者失败的消息再发送一遍。如果有靠谱的自动对账补账逻辑，这种方案还是非常实用的。
这种方案遵循BASE理论，采用的是最终一致性，笔者认为是这几种方案里面比较适合实际业务场景的，即不会出现像2PC那样复杂的实现(当调用链很长的时候，2PC的可用性是非常低的)，也不会像TCC那样可能出现确认或者回滚不了的情况。
优点： 一种非常经典的实现，避免了分布式事务，实现了最终一致性。在 .NET中 有现成的解决方案。
缺点： 消息表会耦合到业务系统中，如果没有封装好的解决方案，会有很多杂活需要处理。
四、MQ 事务消息
有一些第三方的MQ是支持事务消息的，比如RocketMQ，他们支持事务消息的方式也是类似于采用的二阶段提交，但是市面上一些主流的MQ都是不支持事务消息的，比如 RabbitMQ 和 Kafka 都不支持。
以阿里的 RocketMQ 中间件为例，其思路大致为：
第一阶段Prepared消息，会拿到消息的地址。
第二阶段执行本地事务，第三阶段通过第一阶段拿到的地址去访问消息，并修改状态。
也就是说在业务方法内要想消息队列提交两次请求，一次发送消息和一次确认消息。如果确认消息发送失败了RocketMQ会定期扫描消息集群中的事务消息，这时候发现了Prepared消息，它会向消息发送者确认，所以生产方需要实现一个check接口，RocketMQ会根据发送端设置的策略来决定是回滚还是继续发送确认消息。这样就保证了消息发送与本地事务同时成功或同时失败。
遗憾的是，RocketMQ并没有 .NET 客户端。有关 RocketMQ的更多消息，大家可以查看这篇博客
优点： 实现了最终一致性，不需要依赖本地数据库事务。
缺点： 实现难度大，主流MQ不支持，没有.NET客户端，RocketMQ事务消息部分代码也未开源。
五、Sagas 事务模型
Saga事务模型又叫做长时间运行的事务（Long-running-transaction）, 它是由普林斯顿大学的H.Garcia-Molina等人提出，它描述的是另外一种在没有两阶段提交的的情况下解决分布式系统中复杂的业务事务问题。你可以在这里看到 Sagas 相关论文。
我们这里说的是一种基于 Sagas 机制的工作流事务模型，这个模型的相关理论目前来说还是比较新的，以至于百度上几乎没有什么相关资料。
该模型其核心思想就是拆分分布式系统中的长事务为多个短事务，或者叫多个本地事务，然后由 Sagas 工作流引擎负责协调，如果整个流程正常结束，那么就算是业务成功完成，如果在这过程中实现失败，那么Sagas工作流引擎就会以相反的顺序调用补偿操作，重新进行业务回滚。
比如我们一次关于购买旅游套餐业务操作涉及到三个操作，他们分别是预定车辆，预定宾馆，预定机票，他们分别属于三个不同的远程接口。可能从我们程序的角度来说他们不属于一个事务，但是从业务角度来说是属于同一个事务的。
他们的执行顺序如上图所示，所以当发生失败时，会依次进行取消的补偿操作。
因为长事务被拆分了很多个业务流，所以 Sagas 事务模型最重要的一个部件就是工作流或者你也可以叫流程管理器（Process Manager），工作流引擎和Process Manager虽然不是同一个东西，但是在这里，他们的职责是相同的。在选择工作流引擎之后，最终的代码也许看起来是这样的
SagaBuilder saga = SagaBuilder.newSaga("trip")
.activity("Reserve car", ReserveCarAdapter.class)
.compensationActivity("Cancel car", CancelCarAdapter.class)
.activity("Book hotel", BookHotelAdapter.class)
.compensationActivity("Cancel hotel", CancelHotelAdapter.class)
.activity("Book flight", BookFlightAdapter.class)
.compensationActivity("Cancel flight", CancelFlightAdapter.class)
.end()
.triggerCompensationOnAnyError();
camunda.getRepositoryService().createDeployment()
.addModelInstance(saga.getModel())
.deploy();这里有一个 C# 相关示例，有兴趣的同学可以看一下。
优缺点这里我们就不说了，因为这个理论比较新，目前市面上还没有什么解决方案，即使是 Java 领域，我也没有搜索的太多有用的信息。
分布式事务解决方案：CAP
上面介绍的那些分布式事务的处理方案你在其他地方或许也可以看到，但是并没有相关的实际代码或者是开源代码，所以算不上什么干货，下面就放干货了。
在 .NET 领域，似乎没有什么现成的关于分布式事务的解决方案，或者说是有但未开源。具笔者了解，有一些公司内部其实是有这种解决方案的，但是也是作为公司的一个核心产品之一，并未开源...
鉴于以上原因，所以博主就打算自己写一个并且开源出来，所以从17年初就开始做这个事情，然后花了大半年的时间在一直不断完善，就是下面这个 CAP。
Github CAP：这里的 CAP 就不是 CAP 理论了，而是一个 .NET 分布式事务解决方案的名字。
详细介绍：
http://www.cnblogs.com/savorboard/p/cap.html
相关文档：
http://www.cnblogs.com/savorboard/p/cap-document.html
夸张的是，这个解决方案是具有可视化界面（Dashboard）的，你可以很方面的看到哪些消息执行成功，哪些消息执行失败，到底是发送失败还是处理失败，一眼便知。
最夸张的是，这个解决方案的可视化界面还提供了实时动态图表，这样不但可以看到实时的消息发送及处理情况，连当前的系统处理消息的速度都可以看到，还可以看到过去24小时内的历史消息吞吐量。
最最夸张的是，这个解决方案的还帮你集成了 Consul 做分布式节点发现和注册还有心跳检查，你随时可以看到其他的节点的状况。
最最最夸张的是，你以为你看其他节点的数据要登录到其他节点的Dashboard控制台看？错了，你随便打开其中任意一个节点的Dashboard，点一下就可以切换到你想看的节点的控制台界面了，就像你看本地的数据一样，他们是完全去中心化的。
你以为这些就够了？不，远远不止：
CAP 同时支持 RabbitMQ，Kafka 等消息队列
CAP 同时支持 SQL Server, MySql, PostgreSql 等数据库
CAP Dashboard 同时支持中文和英文界面双语言，妈妈再也不用担心我看不懂了
CAP 提供了丰富的接口可以供扩展，什么序列化了，自定义处理了，自定义发送了统统不在话下
CAP 基于MIT开源，你可以尽管拿去做二次开发。（记得保留MIT的License）
这下你以为我说完了？ 不！
你完全可以把 CAP 当做一个 EventBus 来使用，CAP具有优秀的消息处理能力，不要担心瓶颈会在CAP，那是永远不可能， 因为你随时可以在配置中指定CAP处理的消息使用的进程数， 只要你的数据库配置足够高...
说了这么多，口干舌燥的，你不 Star 一下给个精神上的支持说不过去吧？ ^_^
2号传送门： https://github.com/dotnetcore/CAP
不 Star 也没关系，我选择原谅你~
总结
通过本文我们了解到两个分布式系统的理论，他们分别是CAP和BASE 理论，同时我们也总结并对比了几种分布式分解方案的优缺点，分布式事务本身是一个技术难题，是没有一种完美的方案应对所有场景的，具体还是要根据业务场景去抉择吧。 然后我们介绍了一种基于本地消息的的分布式事务解决方案CAP。