Distributed systems, distributed database challenges encountered

　　Distributed system, when i access relational databases / o occupy too high, insufficient memory, visited slow down the case, the policy stream popular sub-library sub-table can be considered, but so will the arrival of many new technical challenges .

1. Distributed Transaction

　　When the application or monomer, the monomer database, all without considering the consistency of the transaction, because mysql has helped us deal with affairs issues (ACID), but this is only for the next single case, if a plurality of databases, master-slave backup, separate read and write, it will cause inconsistencies affairs case, then what distributed transactions, distributed

　　Transaction how to solve it?

　　1. The concept of matters concerning

　　　　Local affairs: the advantages of a local transaction is supported by strict ACID properties, efficient, reliable, state can only be maintained in Explorer, and simple application programming model.

　　　　Global Affairs: When the transaction to be managed globally by the global transaction manager global transaction, the transaction manager responsible for managing the global state of affairs and resources involved, consistent collaborative resources to submit rollback.

　　　　Two-phase transactions: two-phase transaction commit uses the X / OPEN organization as defined in DTP model , through abstracted AP, TM, RMthe concept of strong consistency guarantees affairs. Wherein TMand RMused between XAprotocol for bidirectional communication. Compared with the traditional local affairs,

　　　　　　　　　　XA transaction increases the prepare phase, the database in addition to passively accept the submission instructions, it can also reverse inform the caller whether the transaction can be committed. So TMwe can prepare to collect the results of all branches of the transaction, and finally submitted atoms, to ensure strong transactional consistency.

　　　　BASE theory: BA refers to the availability of basic services, support partition failed, S represents a flexible state, that is, allow a short time is not synchronized, E represents the final consistency, the final data is the same, but real time is inconsistent. Atomicity and durability must be guaranteed from the fundamental need for availability, performance and service degradation, and only reduce the requirement of consistency and isolation.

　　　 CAP theorem: For shared data systems, CAP can only have two of them, any two scenes that have adapted really business systems is usually a mixture of ACID with the CAP. Distributed system is the most important thing is to meet business needs, rather than the pursuit of highly abstract, absolute system characteristics. C indicates that consistency, that is, all users see the data is the same.

　　　　　　　　　A indicates the availability of means can always find a copy of the data available. P represents the partition fault tolerance, can tolerate network outages and other failures.

　　2. Solution

　　　　1. The two-phase (2PC) Solutions

　　　　　　　　Two-phase commit is actually relatively simple, here are two resources to prepare and submit two interfaces.

　　　　　　　　Due to the isolation of mutually exclusive requirements, during the execution of the transaction, all of the resources are locked, this is only suitable for short transaction execution time determined. However, in order to ensure the consistency of distributed transactions, mostly used serialized isolation level to ensure transactional consistency, this will reduce the throughput of the system.

　　　　　　　　But because the cost of 2PC protocol is relatively high, there are global lock problems, performance will be relatively poor. Now we basically do not use this strong uniform solution.

　　　　2.TCC Services Compensation Scheme

　　　　　　　　　　　　　　TCC is the origin of the name which contains the try, confirm, cancel three operations.

　　　　　　　　　　　　　　Compared with the two-phase commit, TCC is located in the business service layer, no separate preparation stage, Try flexibility to choose the operating business resource lock granularity. TCC is to solve the problem through the final system performance consistency this design, a great inspiration to our design choices. Some technical problems when the system can be solved by business modeling approach.

　　　　3.Saga Affairs

　　　　　　Saga This concept comes from three decades ago a database paper Sagas , a Saga transaction is a transaction when the transaction consisting of multiple short-long. In a distributed transaction scenario, we have seen a distributed transaction Saga transaction is a local transaction consisting of a plurality, each local transaction has a corresponding transaction compensation.

　　　　　　在Saga事务的执行过程中，如果某一步执行出现异常，Saga事务会被终止，同时会调用对应的补偿事务完成相关的恢复操作，这样保证Saga相关的本地事务要么都是执行成功，要么通过补偿恢复成为事务执行之前的状态。

　　　　　　Saga定义了一个事务中的每个子事务都有一个与之对应的反向补偿操作。由Saga事务管理器根据程序执行结果生成一张有向无环图，并在需要执行回滚操作时，根据该图依次按照相反的顺序调用反向补偿操作。Saga事务管理器只用于控制何时重试，何时补偿，

　　　　　　并不负责补偿的内容，补偿的具体操作需要由开发者自行提供。

　　　　　　SAGA可以看做一个异步的、利用队列实现的补偿事务。

　　　　　　其适用于无需马上返回业务发起方最终状态的场景，例如：你的请求已提交，请稍后查询或留意通知之类。

　　　　　　将上述补偿事务的场景用SAGA改写，其流程如下：

- - 　　　　订单服务创建最终状态未知的订单记录，并提交事务
  - 　　　　现金服务扣除所需的金额，并提交事务
  - 　　　　订单服务更新订单状态为成功，并提交事务

　　　　以上为成功的流程，若现金服务扣除金额失败，那么，最后一步订单服务将会更新订单状态为失败。

　　　　其业务编码工作量比补偿事务多一点，包括以下内容：

- - 　　　　订单服务创建初始订单的逻辑
  - 　　　　订单服务确认订单成功的逻辑
  - 　　　　订单服务确认订单失败的逻辑
  - 　　　　现金服务扣除现金的逻辑
  - 　　　　现金服务补偿返回现金的逻辑

　　　　但其相对于补偿事务形态有性能上的优势，所有的本地子事务执行过程中，都无需等待其调用的子事务执行，减少了加锁的时间，这在事务流程较多较长的业务中性能优势更为明显。同时，其利用队列进行进行通讯，具有削峰填谷的作用。

　　　　因此该形式适用于不需要同步返回发起方执行最终结果、可以进行补偿、对性能要求较高、不介意额外编码的业务场景。

　　　　但当然SAGA也可以进行稍微改造，变成与TCC类似、可以进行资源预留的形态。

　　　　4.基于消息事务　　　　

　　　　　　基于消息实现的事务适用于分布式事务的提交或回滚只取决于事务发起方的业务需求，其他数据源的数据变更跟随发起方进行的业务场景。

　　　　　　举个例子，假设存在业务规则：某笔订单成功后，为用户加一定的积分。

　　　　　　在这条规则里，管理订单数据源的服务为事务发起方，管理积分数据源的服务为事务跟随者。

　　　　　　从这个过程可以看到，基于消息队列实现的事务存在以下操作：

- - 　　　　订单服务创建订单，提交本地事务
  - 　　订单服务发布一条消息
  - 　　　　积分服务收到消息后加积分

　　　　我们可以看到它的整体流程是比较简单的，同时业务开发工作量也不大：

- - 　　　　编写订单服务里订单创建的逻辑
  - 　　　　编写积分服务里增加积分的逻辑

　　　　　　可以看到该事务形态过程简单，性能消耗小，发起方与跟随方之间的流量峰谷可以使用队列填平，同时业务开发工作量也基本与单机事务没有差别，都不需要编写反向的业务逻辑过程。因此基于消息队列实现的事务是我们除了单机事务外最优先考虑使用的形态。