06 | Domain Events: The Key to Decoupling Microservices

This article mainly talks about "Domain Events".

During event storming (Event Storming), we found that in addition to business behaviors such as commands and operations, there is also a very important event that usually leads to further business operations. In DDD, this event is called field events.

This is just the simplest definition and does not allow us to really understand it. So what exactly are domain events? What is the technical implementation mechanism of domain events? In this article, we will focus on solving these two major problems.

domain event

Domain events are a very important part of the domain model, used to represent events that occur in the domain. A domain event will lead to further business operations, while realizing business decoupling, it also helps to form a complete business closed loop.

For example, a domain event could be a step in a business process,

• For example, after the payment of the insurance business is completed, the action of transferring the insurance application to the insurance policy is triggered;
• It may also be an event that occurs during the scheduled batch processing, such as batch processing to generate a quarterly premium payment notice, triggering the operation of sending a payment email notification;
• Or a follow-up action triggered after an event occurs, such as entering the wrong password three times in a row, triggering the action of locking the account.

So how to identify domain events?

It's very simple, and it is strongly related to the definition just mentioned. When doing user journey or scenario analysis, we need to capture the key words in the mouth of business, demand personnel or domain experts: "If ... happens, then ..." "When finished ..., please notify ..." "Occurs ... When ..., then..." and so on. In these scenarios, if a certain event occurs and further actions are triggered, then this event is likely to be a domain event.

Why do domain events use eventual consistency instead of the direct invocation of traditional SOA?

Let's review one of the design principles of aggregation mentioned in the [previous article] : use eventual consistency outside the boundary . A transaction can only change the state of at most one aggregate. If a business operation involves multiple aggregate state changes, the eventual consistency of domain events should be adopted.

The domain event-driven design can cut off the strong dependencies between domain models. After the event publishing is completed, the publisher does not need to care whether the subsequent event processing of the subscriber is successful. This can realize the decoupling of the domain model and maintain the independence of the domain model and data integrity. consistency. When the domain model is mapped to the microservice system architecture, domain events can decouple microservices, and the data between microservices does not require strong consistency, but event-based eventual consistency.

Going back to specific business scenarios, we found that some domain events occur between aggregations within microservices, some between microservices, and both. Generally speaking, domains across microservices Event handling mostly. When designing microservices, events in different domains are handled differently.

1. Domain events in microservices

When a domain event occurs between aggregates in a microservice, the event entity construction and event data persistence are completed after the domain event occurs, the publisher aggregate publishes the event to the event bus, and the subscriber receives the event data to complete subsequent business operations.

The integration of most events in microservices takes place in the same process, and the process itself can control transactions well, so it is not necessarily necessary to introduce message middleware. But if an event updates multiple aggregates at the same time, according to the DDD principle of "only one aggregate is updated in one transaction", you need to consider whether to introduce an event bus. However, the event bus in microservices may increase the complexity of development, so you need to combine application complexity and benefits for comprehensive consideration.

Application services within microservices can be cross-aggregated through cross-aggregated service orchestration and composition, and accessed through service calls. This method is usually used in scenarios that require high real-time performance and data consistency. This process will use distributed transactions to ensure that the data of the publisher and the subscriber are updated successfully at the same time.

2. Domain events between microservices

Cross-microservice domain events will achieve business collaboration between different bounded contexts or domain models. Its main purpose is to achieve microservice decoupling and reduce the pressure of real-time service access between microservices.

There are many scenarios where domain events occur between microservices, and the event processing mechanism is more complicated. Events across microservices can drive business processes or data to flow directly between different subdomains or microservices.

The event mechanism across microservices should generally consider event construction, publishing and subscription, event data persistence, message middleware, and even the introduction of distributed transaction mechanisms when event data persistence may also need to be considered.

Access between microservices can also be directly invoked by application services to achieve real-time access to data and services. The disadvantage is that simultaneous data changes across microservices require the introduction of distributed transactions to ensure data consistency. The distributed transaction mechanism will affect system performance and increase the coupling between microservices, so we should try to avoid using distributed transactions.

Cases related to domain events

Let me introduce you to a case of related domain events in the insurance underwriting business process.

The generation of an insurance policy has gone through many subdomains, business state changes, and cross-microservice business data transfer. This process will generate a lot of domain events, which facilitate the flow and role conversion of insurance business data and objects between different microservices and subdomains.

In the picture below, I have listed several key processes to illustrate how to use domain event-driven design to drive underwriting business processes.

The starting point of the event: the customer purchases insurance - the business personnel completes the policy entry - generates the insurance application form - starts the payment action.

1. The insurance microservice generates a payment notice and publishes the first event: the payment notice has been generated, and publishes the payment notice data to the message middleware. The collection microservice subscribes to the payment notice event and completes the payment operation. The payment notice has been generated, and the field event is over.
2. After the collection microservice completes the payment, the second domain event is released: the payment is completed, and the payment data is published to the message middleware. The original subscriber receiving payment microservice becomes the publisher at this time. The original event publisher insured microservice is converted into a subscriber. After receiving the payment information and confirming that the payment is complete, the insurance application micro-service completes the operation of converting the insurance application form into an insurance policy. The payment has been completed and the field event is over.
3. After the insurance application form is transferred to the insurance form, the insurance microservice publishes the third field event: the policy has been generated, and the policy data is published to the message middleware. After the policy microservice receives the policy data, it completes the policy data saving operation. The policy has been generated and the field event is over.
4. After the insurance policy microservice finishes saving the policy data, a series of domain events will occur later, and the policy data will be sent to microservices such as commission, payment and reinsurance in a concurrent manner through the message middleware, all the way to finance, and after the policy is completed All subsequent business processes. I won't go into details here.

In short, the asynchronous mechanism driven by domain events can promote the flow of business processes and data between different microservices, realize the decoupling of microservices, reduce the pressure of service calls between microservices, and improve user experience.

The overall structure of domain events

The execution of domain events requires a series of components and technologies to support them. Let's take a look at the overall technical architecture diagram of this domain event. Domain event processing includes: event construction and publishing, event data persistence, event bus, message middleware, event receiving and processing, etc. Let's talk about it one by one. 

1. Event construction and publishing

The basic attributes of an event include at least: unique event identifier, time of occurrence, event type, and event source. The unique event identifier should be globally unique so that the event can be transmitted in multiple bounded contexts without ambiguity. The basic attributes of the event mainly record the data of the event itself and the background of the event.

In addition, there is another more important thing in the event, that is, the business attribute, which is used to record the business data at the moment when the event occurs, and these data will be transmitted to the subscriber along with the event to carry out the next business operation.

Event basic attributes and business attributes together constitute an event entity, and the event entity depends on the aggregate root. After a domain event occurs, the business data in the event will not be modified, so the business data can be saved in the form of serialized value objects. This storage format is also relatively easy to parse and obtain in the message middleware.

In order to ensure the uniformity of the event structure, we will also create an event base class DomainEvent (refer to the figure below), and subclasses can extend properties and methods. Since the event does not have too much business behavior, the implementation method is generally relatively simple.

The event entity needs to be constructed and persisted before the event is published. There are many ways to publish events. You can publish them to the event bus or message middleware through application services or domain services, or you can use the timing program or database log capture technology to obtain incremental event data from the event table and publish them to the message middleware. .

2. Event data persistence

Event data persistence can be used for data reconciliation between systems, or to implement auditing of publisher and subscriber event data. When encountering message middleware, subscriber system downtime or network interruption, the follow-up business flow can still be continued after the problem is solved to ensure data consistency.

There are two schemes for event data persistence, and you can choose according to your business scenarios during implementation.

• Persist in the event table of the local business database, and use local transactions to ensure the consistency of business and event data.
• Persist to the shared event database. It should be noted here that the business database and the event database are not in the same database, and their data persistence operations will cross databases. Therefore, a distributed transaction mechanism is needed to ensure the strong consistency of business and event data. The result is that it will affect the system performance. certain influence.

3. Event Bus (EventBus)

The event bus is an important component to implement domain events between aggregations in microservices, and it provides services such as event distribution and reception. The event bus is an in-process model, which traverses the list of subscribers between aggregations in microservices, and transfers data synchronously or asynchronously. The event distribution process is roughly as follows:

• If it is a subscriber (other aggregation) in the microservice, it will be directly distributed to the specified subscriber;
• If it is a subscriber outside the microservice, save the event data to the event library (table) and send it to the message middleware asynchronously;
• If there are both internal and external subscribers of the microservice, it will be distributed to the internal subscribers first, the event message will be saved to the event library (table), and then sent to the message middleware asynchronously.

4. Message middleware

Most of the cross-microservice domain events use message middleware to implement cross-microservice event publishing and subscription. The products of message middleware are very mature, and there are many optional technologies on the market, such as Kafka, RabbitMQ, etc.

5. Event reception and processing

The microservice subscriber adopts the monitoring mechanism at the application layer to receive the event data in the message queue, and after completing the persistence of the event data, it can start further business processing. Domain event handling can be implemented in domain services.

Cases related to domain event operation mechanism

Here I use the payment notice event of the underwriting business process to explain to you the operating mechanism of domain events. This domain event happens between the insurance application and collection microservices. The domain event that occurs is: the payment notice has been generated. The next business operation is: payment.

Starting point of the event: The order issuer generates an insurance application form, and after the underwriting is passed, initiates the operation of generating a payment notice.

1. The insurance microservice application service calls the domain services createPaymentNotice and createPaymentNoticeEvent in the aggregation to create payment notices and payment notice events respectively. The payment notice event class PaymentNoticeEvent inherits the base class DomainEvent.
2. Use warehousing services to persist business and event data related to payment notices. In order to avoid distributed transactions, these business and event data are persisted to the local insured microservice database.
3. Through the database log capture technology or timing program, the event incremental data is obtained from the database event table and published to the message middleware. Note here: Event publishing can also be done through application services or domain services.
4. The collection microservice subscribes to the payment notice event message topic from the message middleware at the application layer, and after monitoring and obtaining the event data, the application service calls the domain service of the domain layer to persist the event data to the local database.
5. The collection microservice calls the domain service PayPremium of the domain layer to complete the payment.
6. The event is over.

Tip: After the payment is completed, the microservices in the follow-up process will also generate many new field events, such as the payment has been completed, the policy has been saved, and so on. These subsequent event processing is basically similar to the processing mechanism of 1-6.

Summarize

This article mainly talks about domain events and the processing mechanism of domain events. Domain event-driven is a very mature technology that has been widely used in many distributed architectures. Domain events are an important concept of DDD. When designing, we should focus on domain events, use domain events to drive business flow, try to adopt event-based eventual consistency, reduce the pressure of direct access between microservices, and realize microservices. The decoupling between domain models maintains the independence and data consistency of the domain model.

 In addition, the domain event-driven mechanism can realize the mode of one publisher and N subscribers, which is basically impossible in the traditional direct service invocation design.