Domain Driven Design (DDD) Practice Road (2): Event Driven and CQRS

This article first appeared in vivo micro-channel public number of Internet technology 
links: https://mp.weixin.qq.com/s/Z3uJhxJGDif3qN5OlE_woA
Author: wenbo zhang

[Road of Domain Driven Design Practice] A series of wonderful articles in the past:

" Domain Driven Design (DDD) Practice Road (1) " mainly describes the DDD principles at the strategic level.

This is the second article in the "Road of Domain Driven Design Practice" series, which analyzes how to use events to separate software core complexity. Explore why CQRS is widely used in DDD projects and how to implement the CQRS framework on the ground. Of course, we must also be vigilant about the lessons of failure, and then choose the correct response after pros and cons.

1. Foreword: Start with logistics details

Everyone is familiar with logistics tracking. It records in detail what happened at what time, and the data is immutable as an important document. I understand that there are several aspects behind the value: the business party can control each sub-process and know where it is currently; on the other hand, when it is necessary to trace back, the entire historical process can be played back only by recording each step.

In my previous article, I proposed that "software projects are also the category of human social production relations, but the labor achievements we have created are invisible." Therefore, we can use the idea of ​​logistics tracking to develop software projects and disassemble complex processes into individual steps, sub-processes, and states. This is consistent with our event division. This is a typical case of event-driven.

2. Domain Events

Domain events (Domain Events) is a concept in Domain Driven Design (DDD) that is used to capture what has happened in the domain we are modeling.

Domain events themselves are also part of the universal language (Ubiquitous Language) and become the language of communication for all project members, including domain experts.

For example, in the aforementioned cross-border logistics example, after the goods reach the bonded warehouse, it is necessary to assign staff to sort and subcontract, then "the goods have arrived in the bonded warehouse" is an area event.

First, in terms of business logic, this event is related to the success or failure of the entire process; at the same time, it will trigger subsequent sub-processes; and for the business side, this event is also a landmark milestone, which means that the goods on its own will be delivered to In your own hands.

So generally speaking, a domain event has the following characteristics: higher business value, which helps to form a complete business closed loop, which will lead to further business operations. It should also be emphasized here that domain events have clear boundaries.

For example: if you are modeling a restaurant checkout system, then "customer has arrived" is not the focus of your concern, because you cannot immediately ask the other party for money when the customer arrives, and "customer has placed an order "Is an event useful for the checkout system.

1. Events in the field of modeling

When modeling events in the field, we should name the events and attributes according to the general language in the context of bounds. If the event is generated by a command operation on the aggregate, then we usually name the domain event according to the name of the operation method.

For the above example "Goods have arrived in the bonded warehouse", we will publish corresponding domain events

GoodsArrivedBondedWarehouseEvent (Of course, you can also remove the name of the aggregation in the context of clear boundaries and directly model it as ArrivedBondedWarehouseEvent, which is a naming convention).

The name of the event indicates what happened after the execution of the command method on the aggregation. In other words, the pending item and the uncertain state cannot be regarded as domain events.

An effective method is to draw a state flow diagram of the current business, including pre-operations and the resulting state changes. The expression here is the state that has been changed so we do n’t have to express past tense, such as deleting or canceling, which represents It has been deleted or canceled.

Then model the events in the nodes. The following picture shows the file cloud storage business. We model the "past tense" events, PreUploadedEvent, ConfirmUploadedEvent, and RemovedEvent for pre-upload, upload completion confirmation, and deletion.

2. Interpretation of domain event codes

package domain.event;

import java.util.Date;
import java.util.UUID;

/**
 * @Description:
 * @Author: zhangwenbo
 * @Since: 2019/3/6
 */
public class DomainEvent {

    /**
     * 领域事件还包含了唯一ID，
     * 但是该ID并不是实体（Entity）层面的ID概念，
     * 而是主要用于事件追溯和日志。
     * 如果是数据库存储，该字段通常为唯一索引。
     */
    private final String id;

    /**
     * 创建时间用于追溯，另一方面不管使用了
     * 哪种事件存储都有可能遇到事件延迟，
     * 我们通过创建时间能够确保其发生顺序。
     */
    private final Date occurredOn;

    public DomainEvent() {
        this.id = String.valueOf(UUID.randomUUID());
        this.occurredOn = new Date();
    }
}

When creating domain events, there are two points to note:

• Domain events themselves should be immutable (Immutable);

• Domain events should carry contextual data related to the event, but not the state data of the entire aggregate root. For example, when you create an order, you can carry the basic information of the order, and for the user to update the order receiving address event AddressUpdatedEvent, you only need to include the order, user, and new address.

public class AddressUpdatedEvent extends DomainEvent {
    //通过userId+orderId来校验订单的合法性；
    private String userId; 
    private String orderId;
    //新的地址
    private Address address;
    //略去具体业务逻辑
}

3. Storage of domain events

The immutability and traceability of the event determine the principle that it must be persisted. Let's take a look at several common solutions.

3.1 Separate EventStore

In some business scenarios, a separate event storage center will be created, which may be Mysql, Redis, Mongo, or even file storage. Take Mysql as an example here. Business_code and event_code are used to distinguish different events of different businesses. The specific naming rules can be based on actual needs.

Here you need to pay attention to the scenario where the data source is inconsistent with the business data source. We want to ensure that when the business data is updated, the event can be recorded accurately. In practice, try to avoid using distributed transactions, or try to avoid its cross-database scenario. You have to think about how to compensate. It must be avoided that the user updated the delivery address, but the AddressUpdatedEvent failed to save.

The general principle is Say No for distributed transactions. In any case, I believe that there are more methods than problems. In practice, we can always think of solutions. The difference is whether the solution is concise and decoupling.

# 考虑是否需要分表，事件存储建议逻辑简单
CREATE TABLE `event_store` (
  `event_id` int(11) NOT NULL auto increment,
  `event_code` varchar(32) NOT NULL,
  `event_name` varchar(64) NOT NULL,
  `event_body` varchar(4096) NOT NULL,
  `occurred_on` datetime NOT NULL，
  `business_code` varchar(128) NOT NULL,
  UNIQUE KEY (`event id`)
) ENGINE=InnoDB COMMENT '事件存储表';

3.2 Store with business data

In a distributed architecture, each module is relatively small, which is precisely "autonomy". If the current business data volume is small, you can store the event together with the business data, and use the relevant identifier to distinguish whether it is real business data or event records; or establish the business's own event storage in the current business database, but consider the event storage The magnitude is necessarily greater than the actual business data, consider whether you need to divide the table.

The advantages of this scheme: data autonomy; avoid distributed transactions; no additional event storage center is required. Of course, its disadvantage is that it cannot be reused.

4. How to release domain events

4.1 Domain events are sent by domain aggregation

/*
* 一个关于比赛的充血模型例子
* 贫血模型会构造一个MatchService，我们这里通过模型来触发相应的事件
* 本例中略去了具体的业务细节
*/
public class Match {
    public void start() {
        //构造Event....
        MatchEvent matchStartedEvent = new MatchStartedEvent();
        //略去具体业务逻辑
        DefaultDomainEventBus.publish(matchStartedEvent);
    }

    public void finish() {
        //构造Event....
        MatchEvent matchFinishedEvent = new MatchFinishedEvent();
        //略去具体业务逻辑
        DefaultDomainEventBus.publish(matchFinishedEvent);
    }

    //略去Match对象基本属性
}

4.2 Event Bus VS Message Middleware

Domain events within microservices can achieve business collaboration between different aggregations through the event bus or using application services. That is, when a domain event occurs in a microservice, since most of the integration of events occurs in the same thread, it is not necessary to introduce message middleware. However, if an event updates multiple aggregated data at the same time, according to the principle of DDD "one transaction only updates one aggregate root", you can consider introducing message middleware to adopt different transactions for different aggregate roots in microservices through asynchronous methods

Three, Saga distributed transactions

1, Saga overview

Let's see how to use Saga mode to maintain data consistency?

Saga is a mechanism for maintaining data consistency in a microservices architecture, which can avoid the problems caused by distributed transactions.

A Saga represents one of multiple services that need to be updated, that is, Saga consists of a series of local transactions. Each local transaction is responsible for updating the private database of its service. These operations still rely on the ACID transaction framework and function library that we are familiar with.

Mode: Saga

Through the use of asynchronous messages to coordinate a series of local transactions, thereby maintaining data consistency between multiple services.

See (strongly recommended):https://microservices.io/patterns/data/saga.html

Saga is one step less than TCC's Try operation. TCC needs to interact with the transaction participants twice no matter whether the final transaction succeeds or fails. Saga only needs to interact with the transaction participants once when the transaction is successful. If the transaction fails, additional compensation rollback is required.

• Each Saga consists of a series of sub-transaction Ti;

• Each Ti has a corresponding compensation action Ci, which is used to cancel the result caused by Ti;

As you can see, compared with TCC, Saga has no "reserved" action, and its Ti is directly submitted to the library.

There are two execution orders for Saga:

• success：T1, T2, T3, ..., Tn ；

• failure：T1, T2, ..., Tj, Cj,..., C2, C1，其中0 < j < n；

So we can see that the revocation of Saga is very important. It can be said that the difficulty of using Saga lies in how to design your rollback strategy.

2. Saga implementation

Through the above example, we have a preliminary feeling for Saga, now let's discuss how to achieve it in depth. When starting Saga through system commands, the coordination logic must select and notify the first Saga participant to perform local transactions. Once the transaction is completed, Saga coordinates the selection and calls the next Saga participant.

This process continues until Saga has completed all steps. If any local transaction fails, Saga must perform compensation transactions in the reverse order. The following different methods can be used to construct Saga's coordination logic.

2.1 Collaborative (choreography)

Saga's decision-making and execution sequence logic is distributed among each participant in Saga, and they communicate by exchanging events.

(Quoted from the relevant chapter of "Microservice Architecture Design Pattern")

1. The Order service creates an Order and publishes the OrderCreated event.

2. The Consumer service consumes the OrderCreated event, verifies whether the consumer can place an order, and publishes the ConsumerVerified event.

3. The Kitchen service consumes the OrderCreated event, verifies the order, creates a trouble ticket in the CREATE_PENDING state, and publishes the TicketCreated event.

4. The Accounting service consumes the OrderCreated event and creates a Credit Card Authorization in the PENDING state.

5. The Accounting service consumes TicketCreated and ConsumerVerified events, charges consumers' credit cards, and publishes credit card authorization failure events.

6. The Kitchen service uses a credit card authorization failure event and changes the status of the trouble ticket to REJECTED.

7. Order service consumption credit card authorization failure event, and change the order status to rejected.

2.2 Orchestration

Concentrate Saga's decision and execution sequence logic in a Saga orchestrator class. The Saga Orchestrator sends imperative messages to each Saga participant, instructing those participant services to complete specific operations (local transactions). Similar to a state machine, when the participant service completes the operation, it sends a state command to the orchestrator to decide what to do next.

(Quoted from the relevant chapter of "Microservice Architecture Design Pattern")

Let's analyze the execution process

1. Order Service first creates an Order and an order creation controller. After that, the flow of the path is as follows:

2. Saga orchestrator sends a Verify Consumer command to Consumer Service.

3. Consumer Service responds to the Consumer Verified message.

4. Saga orchestrator sends the Create Ticket command to Kitchen Service.

5. Kitchen Service responded to the Ticket Created message.

6. The Saga coordinator sends an authorization card message to the Accounting Service.

7. The Accounting Service Department replies with the card authorization message.

8. Saga orchestrator sends the Approve Ticket command to Kitchen Service.

9. Saga orchestrator sends an order approval command to the order service.

2.3 Compensation strategy

In the previous description, we said that Saga's most important thing is how to handle exceptions. The state machine also defines many abnormal states. If the above 6 will fail, trigger AuthorizeCardFailure, at this time we will end the order and roll back the previously submitted transaction. There is a need to distinguish between verification transactions and transactions that require compensation.

 A Saga is composed of three different types of transactions: compensable transactions (can be rolled back, so there is a compensation transaction); critical transactions (this is the key point of Saga's success or failure, such as 4 account withholding); and repeatability Transaction, it does not need to be rolled back and guaranteed to be completed (such as 6 update status).

In Create Order Saga, the createOrder () and createTicket () steps are compensable transactions and have a compensation transaction that cancels their update.

The verifyConsumerDetails () transaction is read-only, so no compensation transaction is required. The authorizeCreditCard () transaction is the key transaction of this Saga. If the consumer's credit card can be authorized, then this Saga is guaranteed to be completed. The steps approveTicket () and approveRestaurantOrder () are repeatable transactions after critical transactions.

It is particularly important to carefully dismantle each step and then evaluate its compensation strategy. As you can see, each type of transaction plays a different role in the countermeasure.

4. CQRS

The concept of events was described earlier, and how Saga solved complex transactions was analyzed. Now let's see why CQRS is widely adopted in DDD. In addition to the characteristics of separate read and write, we use an event-driven way to practice Command logic can effectively reduce the complexity of the business.

When you understand how to model events, how to avoid complex transactions, when to use message middleware and when to use the event bus, you can understand why it is CQRS and how to apply it correctly.

(The picture comes from the network)

The following is the design in our project. Why the Read / Write Service appears here is to encapsulate the call. The service internally sends events based on aggregation. Because I found that in actual projects, many people will ask me for XXXService instead of XXX model for the first time, so in projects where DDD is not fully popular, it is recommended that you adopt this centering strategy. This is also in line with our decoupling. The other party relies on my abstraction ability. However, whether I am based on DDD or traditional process code is irrelevant and transparent to it.

Let's first look at the timing relationship between events and processors.

Here is still taking the file cloud storage service as an example. The following is the core code of some processors. The comment line is an interpretation of the function, usage and extension of the code, please read it carefully.

package domain;

import domain.event.DomainEvent;
import domain.handler.event.DomainEventHandler;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

/**
 * @Description: 事件注册逻辑
 * @Author: zhangwenbo
 * @Since: 2019/3/6
 */

public class DomainRegistry {

    private Map<String, List<DomainEventHandler>> handlerMap =
        new HashMap<String, List<DomainEventHandler>>();

    private static DomainRegistry instance;

    private DomainRegistry() {
    }

    public static DomainRegistry getInstance() {
        if (instance == null) {
            instance = new DomainRegistry();
        }
        return instance;
    }

    public Map<String, List<DomainEventHandler>> getHandlerMap() {
        return handlerMap;
    }

    public List<DomainEventHandler> find(String name) {
        if (name == null) {
            return null;
        }
        return handlerMap.get(name);
    }

    //事件注册与维护，register分多少个场景根据业务拆分，
    //这里是业务流的核心。如果多个事件需要维护前后依赖关系，
    //可以维护一个priority逻辑
    public void register(Class<? extends DomainEvent> domainEvent,
                         DomainEventHandler handler) {
        if (domainEvent == null) {
            return;
        }
        if (handlerMap.get(domainEvent.getName()) == null) {
            handlerMap.put(domainEvent.getName(), new ArrayList<DomainEventHandler>());
        }
        handlerMap.get(domainEvent.getName()).add(handler);
        //按照优先级进行事件处理器排序
        。。。
    }
}

Example of file upload complete event.

package domain.handler.event;

import domain.DomainRegistry;
import domain.StateDispatcher;
import domain.entity.meta.MetaActionEnums;
import domain.event.DomainEvent;
import domain.event.MetaEvent;
import domain.repository.meta.MetaRepository;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;
import javax.annotation.Resource;

/**
 * @Description:一个事件操作的处理器
 * 我们混合使用了Saga的两种模式，外层事件交互；
 * 对于单个复杂的事件内部采取状态流转实现。
 * @Author: zhangwenbo
 * @Since: 2019/3/6
 */

@Component
public class MetaConfirmUploadedHandler implements DomainEventHandler {

    @Resource
    private MetaRepository metaRepository;

    public void handle(DomainEvent event) {
        //1.我们在当前的上下文中定义个ThreadLocal变量
        //用于存放事件影响的聚合根信息(线程共享)

        //2.当然如果有需要额外的信息，可以基于event所
        //携带的信息构造Specification从repository获取
        // 代码示例
        // metaRepository.queryBySpecification(SpecificationFactory.build(event));

        DomainEvent domainEvent = metaRepository.load();

        //此处是我们的逻辑
        。。。。

        //对于单个操作比较复杂的，可以使用状态流转进一步拆分
        domainEvent.setStatus(nextState);
        //在事件触发之后，仍需要一个状态跟踪器来解决大事务问题
        //Saga编排式
        StateDispatcher.dispatch();
    }

    @PostConstruct
    public void autoRegister() {
        //此处可以更加细分，注册在哪一类场景中，这也是事件驱动的强大、灵活之处。
        //避免了if...else判断。我们可以有这样的意识，一旦你的逻辑里面充斥了大量
        //switch、if的时候来看看自己注册的场景是否可以继续细分
        DomainRegistry.getInstance().register(MetaEvent.class, this);
    }

    public String getAction() {
        return MetaActionEnums.CONFIRM_UPLOADED.name();
    }

    //适用于前后依赖的事件，通过优先级指定执行顺序
    public Integer getPriority() {
        return PriorityEnums.FIRST.getValue();
    }
}

Event bus logic

package domain;

import domain.event.DomainEvent;
import domain.handler.event.DomainEventHandler;
import java.util.List;

/**
 * @Description:
 * @Author: zhangwenbo
 * @Since: 2019/3/6
 */

public class DefaultDomainEventBus {

    public static void publish(DomainEvent event, String action,
                               EventCallback callback) {

        List<DomainEventHandler> handlers = DomainRegistry.getInstance().
            find(event.getClass().getName());
        handlers.stream().forEach(handler -> {
            if (action != null && action.equals(handler.getAction())) {
                Exception e = null;
                boolean result = true;
                try {
                    handler.handle(event);
                } catch (Exception ex) {
                    e = ex;
                    result = false;
                    //自定义异常处理
                    。。。
                } finally {
                    //write into event store
                    saveEvent(event);
                }

                //根据实际业务处理回调场景，DefaultEventCallback可以返回
                if (callback != null) {
                    callback.callback(event, action, result, e);       
                }
            }
        });
    }
}

5. Autonomous services and systems

DDD emphasizes the autonomy of the bounded context. In fact, from a smaller granularity, the object still needs to have the four characteristics of autonomy, namely: minimum completeness, self-fulfillment, stable space, and independent evolution. Among them, self-fulfillment is the key, because it is not dependent on the outside, so it is stable, and it is possible to evolve independently because of stability. This is why the hexagonal architecture is more common in DDD.

(The picture comes from the network)

6. Conclusion

The events, Saga, and CQRS solutions described in this article can be used individually, and can be applied to one of your methods or your entire package. We don't have to practice a whole set of CQRS in the project, as long as some of the ideas in it solve a certain problem in our project.

Maybe you have sharpened your knife now and are ready to practice these skills in the project. However, we must understand that "every coin has two sides." Not only do we see the advantages of high expansion, decoupling, and easy layout, we still need to understand the problems it brings. After pros and cons analysis, it is the correct way to decide how to realize it.

• This type of programming mode has a certain learning curve;

• The complexity of applications based on messaging;

• It is difficult to deal with the evolution of events;

• There are certain difficulties in deleting data;

• Querying the event repository is very challenging.

However, we still have to realize that in its suitable scenario, the hexagonal architecture and DDD tactics will accelerate our domain modeling process, and also force us to explain a domain from a strict general language perspective, not a requirement. Any way that puts more emphasis on the core domain rather than technology implementation can increase business value and give us a greater competitive advantage.

Attachment: References

1. Pattern: Saga

2. Distributed transactions: Saga model

3. Book: "Microservice Architecture Design Pattern"