MASA Framework - EventBus Design

Overview

The publish-subscribe model is used to decouple different architectural layers, and it can also be used to solve the interaction between isolated businesses

advantage:

• loose coupling
• Crosscutting concerns
• Testability
• event driven

<!-- more -->

publish-subscribe model

Publishers send messages to subscribers through the dispatch center. The dispatch center resolves the relationship between publishers and subscribers to ensure that messages can be delivered to subscribers.

Publishers and subscribers do not know each other. Publishers only publish messages to the dispatch center, while subscribers only care about the types of messages they subscribe to.

Order-preserving execution for multiple subscribers

In the common publish-subscribe model, it is rare to see similar statements. But in actual business, we have similar requirements. For a message, the dispatch center coordinates multiple subscribers to execute the message in sequence, and at the same time, the message processed by the previous subscriber can be delivered to the next subscriber. This not only retains the characteristics of the publish-subscribe model, but also has the characteristics of sequential execution logic.

A small thought: If the configuration of EventBus supports dynamic adjustment, can the execution order of the business be dynamically arranged and combined?

In other words, it may provide a possibility for in-process workflow

Event Sourcing

An event-driven architectural pattern that can be used for auditing and traceability

• based on event-driven architecture
• event as fact
• The view of business data generated by event calculation can be persisted or not

CQRS (Separation of Responsibility for Command Queries)

CQRS is an architectural pattern that decouples the implementation of the change model from the query model

Event Sourcing & CQRS

Event sourcing works well with CQRS

• While persisting the event to the Event Store in the Command Handler, a final view is calculated in real time to the View DB for query display
• In Query, you can get the latest status through View DB, or you can replay events through Event Store to verify View or use it for more rigorous business.

Saga

A Saga is a long-lived transaction decomposed into a collection of sub-transactions that can be interleaved. where each subtransaction is a real transaction that maintains database consistency

• Each Saga consists of a series of sub-transaction Ti
• Each Ti has a corresponding compensation action Ci, which is used to undo the results caused by Ti

Two execution orders

• T1, T2, T3...[Tx retry]...,Tn
• T1, T2, ..., Tj, Cj,..., C2, C1

Two recovery strategies

• backward recovery, backward recovery, compensating all completed transactions, if any sub-transaction fails. That is, the second execution order mentioned above, where j is the sub-transaction with an error, the effect of this approach is to cancel all previous successful sub-transations, so that the execution result of the entire Saga is canceled
• forward recovery, retrying the failed transaction, assuming that each subtransaction will eventually succeed. For scenarios where success is necessary, the execution order is similar to this: T1, T2, ..., Tj (failure), Tj (retry), ..., Tn, where j is the sub-transaction where the error occurred . Ci is not required in this case

Class View for BuildingBlocks

As an interface standard, BuildingBlocks does not have too many interference implementation methods. It only retains the most basic functional process restrictions to achieve the minimum set of EventBus functions. As for whether to implement the subscription relationship based on the interface or the feature, it is left to Contrib to decide.

event

Publish/Subscribe for local events

• IEvent: event interface, which IEvent<TResult>is the basic event interface with return value
• IEventHanldler<TEvent>: Event handler interface, ISagaEventHandler<TEvent>which provides basic interface requirements for the implementation of Saga
• IMiddleware<TEvent>: middleware interface that allows preprocessing actions to be mounted before event execution and finishing actions after time execution
• IEventBus: Event bus interface, used to send events, and provide subscription relationship maintenance and additional function execution

integration event

Publish/Subscribe for cross-process events

• IntegrationEventLog: Integrated event log, used to implement the message model of the local message table
• IIntegrationEventLogService: Integrated event log service interface
• ITopic: Topic to publish/subscribe to
• IIntegrationEvent: Integrated event interface
• IIntegrationEventBus: Integrated event bus, an event bus for cross-process calls

CQRS

Used to decouple the implementation of the change model from the query model

• IQuery<TResult>: Query interface
• IQueryHandler<TCommand,TResult>: query handler interface
• ICommand: An interface that can be used for commands such as additions, deletions, and modifications
• ICommandHandler<TCommand>: Instruction processor interface

Event Bus

To complete the above functions, we need to use EventBus, which needs to have the following basic functions

• receive events
• Maintain subscription relationship
• Forward events

Receiving and forwarding events

These two functions can actually be combined into one interface, the publisher calls Publish, and then the Event Bus forwards it according to the subscription relationship.

Maintain subscription relationship

In .Net projects, our common method for scanning automatic registration is 接口and特性

MediatR supports interface way to scan event subscription relationship, for example:IRequestHandler<,>

public class PingHandler : IRequestHandler<Ping, string>
{
    public Task<string> Handle(Ping request, CancellationToken cancellationToken)
    {
        return Task.FromResult("Pong");
    }
}

If your code cleanliness isn't outrageously high, maybe you hope so

public class NetHandler : IRequestHandler<Ping, string>, IRequestHandler<Telnet, string>
{
    public Task<string> Handle(Ping request, CancellationToken cancellationToken)
    {
        return Task.FromResult("Pong");
    }

    public Task<string> Handle(Telnet request, CancellationToken cancellationToken)
    {
        return Task.FromResult("Success");
    }
}

It looks alright? What if there are many?

So is there a way to fix this?

characteristic! Let's look at an example

public class NetHandler
{
    [EventHandler]
    public Task PingAsync(PingEvent @event)
    {
        //TODO
    }

    [EventHandler]
    public Task TelnetAsync(TelnetEvent @event)
    {
        //TODO
    }
}

Seems like we've found a way out

Order-preserving execution for multiple subscribers

Progressing through events layer by layer can indeed satisfy the sequential execution scenario, but if you are surrounded by a large number of events with infinite nesting dolls, you may need another way out, see the following example:

public class NetHandler
{
    [EventHandler(0)]
    public Task PingAsync(PingEvent @event)
    {
        //TODO
    }

    [EventHandler(1)]
    public Task LogAsync(PingEvent @event)
    {
        //TODO
    }
}

As long as the parameter is the same Event, it will be executed in the order of EventHandler's Order.

Saga

What should I do if the execution fails? If the two methods cannot be combined with the local transaction because one of them needs to call a remote service, can you help me roll back?

Come on, SAGA will let you do another cancellation action for you. At the same time, it also supports the retry mechanism and whether to ignore the cancellation action of the current step.

Let's pre-set the scene first:

1. Call CheckBalanceAsync to check the balance
2. Call WithdrawAsync, throw exception
3. Retry WithdrawAsync 3 times
4. Call CancelWithdrawAsync

code show as below:

public class TransferHandler
{
    [EventHandler(1)]
    public Task CheckBalanceAsync(TransferEvent @event)
    {
        //TODO
    }

    [EventHandler(2, FailureLevels.ThrowAndCancel, enableRetry: true, retryTimes: 3)]
    public Task WithdrawAsync(TransferEvent @event)
    {
        //TODO
        throw new Exception();
    }

    [EventHandler(2, FailureLevels.Ignore, enableRetry: false, isCancel: true)]
    public Task CancelWithdrawAsync(TransferEvent @event)
    {
        //TODO
    }
}

AOP

For a business scenario, add a parameter validation to all Commands before execution

We provide Middleware, which allows doing cross-cutting concerns like a Russian nesting doll (.Net Middleware)

public class LoggingMiddleware<TEvent>
    : IMiddleware<TEvent> where TEvent : notnull, IEvent
{
    private readonly ILogger<LoggingMiddleware<TEvent>> _logger;

    public LoggingMiddleware(ILogger<LoggingMiddleware<TEvent>> logger) => _logger = logger;

    public async Task HandleAsync(TEvent @event, EventHandlerDelegate next)
    {
        _logger.LogInformation("----- Handling command {EventName} ({@Event})", typeof(TEvent).FullName, @event);
         await next();
    }
}

Register DI

builder.Services.AddTransient(typeof(IMiddleware<>), typeof(LoggingMiddleware<>))

MASA EventBus full feature list

• receive events
• Maintain subscription relationship - interface
• Maintaining Subscription Relationships - Features
• Multiple subscribers execute sequentially
• Forward events
• Saga
• AOP
• UoW
• Automatically open and close transactions

Integration Event Bus

Event Bus for cross-service, support for eventual consistency, local message table

Pub/Sub

Provides the Pub Sub interface and provides a default implementation based on Dapr Pub/Sub

local message table

Provides local message saving and UoW linkage interface, and provides default implementation based on EF Core

Instructions

Enable Dapr Event Bus

builder.Services
    .AddDaprEventBus<IntegrationEventLogService>(options=>
    {
        options.UseUoW<CatalogDbContext>(dbOptions => dbOptions.UseSqlServer("server=localhost;uid=sa;pwd=Password;database=test"))
               .UseEventLog<CatalogDbContext>();
        )
    });

Define Integration Events

public class DemoIntegrationEvent : IntegrationEvent
{
    public override string Topic { get; set; } = nameof(DemoIntegrationEvent);//dapr topic name

    //todo other properties
}

Define DbContext (not required, defining DbContext can link the local message table with business transactions)

public class CustomDbContext : IntegrationEventLogContext
{
    public DbSet<User> Users { get; set; } = null!;

    public CustomDbContext(MasaDbContextOptions<CustomDbContext> options) : base(options)
    {

    }
}

Send Event

IIntegrationEventBus eventBus; // from DI
await eventBus.PublishAsync(new DemoIntegrationEvent());

Subscribe to Events (Dapr Pub/Sub based version)

[Topic("pubsub", nameof(DomeIntegrationEvent))]
public async Task DomeIntegrationEventHandleAsync(DomeIntegrationEvent @event)
{
    //todo
}

Domain Event Bus

The ability to provide both Event Bus and Integration Event Bus in the domain, allowing events to be sent in real time or triggered once at Save

Domain Event Bus is the most complete capability, so the use of Domain Event Bus is equivalent to having enabled Event Bus and Integration Event Bus. Within Domain Event Bus, event classification will be automatically coordinated and distributed to Event Bus and Integration Event Bus.

Enable Domain Event Bus

builder.Services
.AddDomainEventBus(options =>
{
    options.UseEventBus()//Use in-process events
        .UseUoW<CustomDbContext>(dbOptions => dbOptions.UseSqlServer("server=localhost;uid=sa;pwd=P@ssw0rd;database=idientity"))
        .UseDaprEventBus<IntegrationEventLogService>()///Use cross-process events
        .UseEventLog<LocalMessageDbContext>()
        .UseRepository<CustomDbContext>();
})

Add DomainCommand

Domain Event is an in-process event, IntegrationDomainEvent is a cross-process event

public class RegisterUserSucceededIntegrationEvent : IntegrationDomainEvent
{
    public override string Topic { get; set; } = nameof(RegisterUserSucceededIntegrationEvent);

    public string Account { get; set; } = default!;
}

public class RegisterUserSucceededEvent : DomainEvent
{
    public string Account { get; set; } = default!;
}

In-process event subscription

[EventHandler]
public Task RegisterUserHandlerAsync(RegisterUserDomainCommand command)
{
    //TODO
}

Cross-process event subscription

[Topic("pubsub", nameof(RegisterUserSucceededIntegrationEvent))]
public async Task RegisterUserSucceededHandlerAsync(RegisterUserSucceededIntegrationEvent @event)
{
    //todo
}

Send DomainCommand

IDomainEventBus eventBus;//from DI
await eventBus.PublishAsync(new RegisterUserDomainCommand());

scenes to be used

• Take into account the connection of legacy systems
• Wandering in the cloud and the non-cloud
• stream computing
• Microservice decoupling and cross-cluster communication (need to change Dapr Pub/Sub to Dapr Binding, not difficult)
• Some AOP class scenarios

Summarize

Event-driven can solve the problems of some specific scenarios. Everything has two sides. Using such a complex pattern in an already simple business scenario will bring a lot of burden.

Apply what you have learned, there is no end to learning.

open source address

MASA.BuildingBlocks ：https://github.com/masastack/MASA.BuildingBlocks

MASA.Contrib ：https://github.com/masastack/MASA.Contrib

MASA.Utils ：https://github.com/masastack/MASA.Utils

MASA.EShop ：https://github.com/masalabs/MASA.EShop

If you are interested in our MASA Framework, whether it is code contribution, use, issue, please contact us

{{o.name}}

{{m.name}}