MASA Framework Event Bus - Cross-Process Event Bus

overview

The cross-process event bus allows publishing and subscribing to messages transmitted across services, and the publishing and subscribing of services are not in the same process

In Masa Framework, cross-process bus events provide a program that can be used out of the box

• IntegrationEvents : provides outbox mode
  • IntegrationEvents.Dapr : The release of messages is realized with the help of Dapr
  • EventLogs.EFCore : Provider of integrated event logs based on EFCore, providing message recording and status update, failure log retry, deletion of expired log records, etc.

getting Started

Cross-process events Daprare not strongly bound to each other. Masa Framework uses the Daprprovided pub/sub capability. If you don’t want to use it, you can also replace it with other implementationsDapr , but currently only the implementation provided by Masa Framwork

• Install .NET 6.0
• Install Dapr

1. Create a new ASP.NET Core empty project Assignment.IntegrationEventBusand install Masa.Contrib.Dispatcher.IntegrationEvents.Dapr, , Masa.Contrib.Dispatcher.IntegrationEvents.EventLogs.EFCore, Masa.Contrib.Data.EFCore.Sqlite, Masa.Contrib.Data.UoW.EFCore,Masa.Contrib.Development.DaprStarter.AspNetCoreMicrosoft.EntityFrameworkCore.Design

dotnet new web -o Assignment.IntegrationEventBus
cd Assignment.IntegrationEventBus

dotnet add package Masa.Contrib.Dispatcher.IntegrationEvents.Dapr --version 0.7.0-preview.8 // 使用dapr提供的pubsub能力
dotnet add package Masa.Contrib.Dispatcher.IntegrationEvents.EventLogs.EFCore --version 0.7.0-preview.8 //本地消息表
dotnet add package Masa.Contrib.Data.EFCore.Sqlite --version 0.7.0-preview.8 //使用EfCore.Sqlite
dotnet add package Masa.Contrib.Data.UoW.EFCore --version 0.7.0-preview.8 //使用工作单元
dotnet add package Masa.Contrib.Development.DaprStarter.AspNetCore --version 0.7.0-preview.8 //开发环境使用DaprStarter协助管理Dapr Sidecar
dotnet add package Microsoft.EntityFrameworkCore.Design --version 6.0.6 //方便后续通过CodeFirst迁移数据库

1. Create a new user context class UserDbContextand inherit from itMasaDbContext

public class UserDbContext : MasaDbContext
{
    public UserDbContext(MasaDbContextOptions<UserDbContext> options) : base(options)
    {
    }
}

1. registration DaprStarter, assistance management Dapr Sidecar, modificationProgram.cs

if (builder.Environment.IsDevelopment())
{
    builder.Services.AddDaprStarter();
}

It needs to be run by Daprpublishing integration events Dapr, the online environment can be run through , and the development environment can be run Kuberneteswith Dapr StarterDapr , so it only needs to be used in the development environment

1. Register cross-process event bus, modify classProgram

builder.Services.AddIntegrationEventBus(option =>
{
    option.UseDapr()
        .UseEventLog<UserDbContext>()
        .UseUoW<UserDbContext>(optionBuilder => optionBuilder.UseSqlite($"Data Source=./Db/{Guid.NewGuid():N}.db;"));
});
var app = builder.Build();

#region dapr 订阅集成事件使用
app.UseRouting();

app.UseCloudEvents();
app.UseEndpoints(endpoints =>
{
    endpoints.MapSubscribeHandler();
});
#endregion

1. Added integration events for user registration eventsRegisterUserEvent

public record RegisterUserEvent : IntegrationEvent
{
    public override string Topic { get; set; } = nameof(RegisterUserEvent);

    public string Account { get; set; }

    public string Mobile { get; set; }
}

1. Open Assignment.IntegrationEventBusthe folder where you are, open cmd or Powershell to execute

dotnet ef migrations add init //创建迁移
dotnet ef database update //更新数据库

1. Send cross-process events, modifyProgram

app.MapPost("/register", async (IIntegrationEventBus eventBus) =>
{
    //todo: 模拟注册用户并发布注册用户事件
    await eventBus.PublishAsync(new RegisterUserEvent()
    {
        Account = "Tom",
        Mobile = "19999999999"
    });
});

1. Subscribe to events, modifyProgram

app.MapPost("/IntegrationEvent/RegisterUser", [Topic("pubsub", nameof(RegisterUserEvent))](RegisterUserEvent @event) =>
{
    Console.WriteLine($"注册用户成功: {@event.Account}");
});

DaprSubscription events are not abstracted for the time being. Currently , the native subscription method is used . We will support Bind in the future. At that time, the subscription method will not be changed due to the replacement of the implementation of pubsub.

Although cross-process events are currently only supported Dapr, this does not mean that you are out of touch with RabbitMq, Kafkaetc. Publish/Subscribe is Dapran abstract ability. There are many components that implement publish and subscribe. , RabbitMqis Kafkaone of the implementations, if you want to learn more about them The relationship between, you can refer to:

1. Teach you how to learn Dapr
2. PubSub Proxy

Source code interpretation

First of all, we need to know the basic knowledge points:

• IIntegrationEvent: Integrated event interface, inheriting IEvent (local event interface), ITopic (subscription interface, publish and subscribe topic), ITransaction (transaction interface)
• IIntegrationEventBus: Integrated event bus interface, used to provide the function of sending integration events
• IIntegrationEventLogService: Interface for integrated event log service (provides the functions of saving local logs, modifying the status to in progress, success, failure, deleting expired logs, and obtaining a list of logs waiting to be retried)
• IntegrationEventLog: Integration event log, providing a model of the local message table
• IHasConcurrencyStamp: Concurrency mark interface (classes that implement this interface will be automatically RowVersionassigned)

Masa.Contrib.Dispatcher.IntegrationEvents

Provides the implementation class of the integrated event interface, and supports the outbox mode, where:

• IPublisher: sender of integration events
• IProcessingServer: background service interface
• IProcessor: processing program interface (all programs will be obtained in the background processing program)
  • DeleteLocalQueueExpiresProcessor: Delete the expired program (delete from the local queue)
  • DeletePublishedExpireEventProcessor: Delete the local message program that has expired and successfully published (deleted from Db)
  • RetryByLocalQueueProcessor: Retry local message records (obtained from the local queue, condition: the sending status is failed or in progress and the number of retries is less than the maximum number of retries and the retry interval is greater than the minimum retry interval)
  • RetryByDataProcessor: Retry local message records (obtained from Db, condition: the sending status is failed or in progress and the number of retries is less than the maximum number of retries and the retry interval is greater than the minimum retry interval, and it is not in the local retry queue)
• IntegrationEventBus: Implementation of IItegrationEvent

Masa.Contrib.Dispatcher.IntegrationEventsOnly the function of the outbox is provided in , but the publishing of integrated events IPublisheris provided by the implementation class of , and the function of obtaining the local message table by Db is IIntegrationEventLogServiceprovided by the implementation class of , which respectively belong to the functions of Masa.Contrib.Dispatcher.IntegrationEvents.Daprand Masa.Contrib.Dispatcher.IntegrationEvents.EventLogs.EFCore, which is why Packages need to be referenced to use integration events

• Masa.Contrib.Dispatcher.IntegrationEvents
• Masa.Contrib.Dispatcher.IntegrationEvents.Dapr
• Masa.Contrib.Dispatcher.IntegrationEvents.EventLogs.EFCore

How to quickly access other implementations

Then some friends will ask, I don’t use it now Dapr, and I don’t want to access it for a while in the future Dapr. I want to access it by myself to realize the release of integration events.

Of course it is possible, if you want to implement integration events by yourself, then you will encounter two situations at this time

Access side supports outbox mode

Take the library CAP , which is widely used by the community, as an example. Since it has completed the outbox mode, we don’t need to deal with the local message table, and we don’t need to consider the management of local message records. Then we can do this

1. Create a new class library Masa.Contrib.Dispatcher.IntegrationEvents.Cap, add Masa.BuildingBlocks.Dispatcher.IntegrationEventsreferences, and installDotNetCore.CAP

dotnet add package DotNetCore.CAP

1. Add new class IntegrationEventBusand implementIIntegrationEventBus

public class IntegrationEventBus : IIntegrationEventBus
{
    private readonly ICapPublisher _publisher;
    private readonly ICapTransaction _capTransaction;
    private readonly IUnitOfWork? _unitOfWork;
    public IntegrationEventBus(ICapPublisher publisher, ICapTransaction capTransaction, IUnitOfWork? unitOfWork = null)
    {
        _publisher = publisher;
        _capTransaction = capTransaction;
        _unitOfWork = unitOfWork;
    }
    
    public Task PublishAsync<TEvent>(TEvent @event) where TEvent : IEvent
    {
        // 如果使用事务
        // _publisher.Transaction.Value.DbTransaction = unitOfWork.Transaction;
        // _publisher.Publish(@event.Topic, @event);
        throw new NotImplementedException();
    }

    public IEnumerable<Type> GetAllEventTypes()
    {
        throw new NotImplementedException();
    }

    public Task CommitAsync(CancellationToken cancellationToken = default)
    {
        throw new NotImplementedException();
    }
}

CAP already supports local transactions, using the currently IUnitOfWorkprovided transactions to ensure the atomicity of data

1. Create a new class ServiceCollectionExtensionsand register the customization Publisherto the service collection

public static class ServiceCollectionExtensions
{
    public static DispatcherOptions UseRabbitMq(this IServiceCollection services)
    {
         //todo: 注册RabbitMq信息
         services.TryAddScoped<IIntegrationEventBus, IntegrationEventBus>();
         return dispatcherOptions;
    }
}

Those who have implemented the outbox mode can be used directly without reference

• Masa.Contrib.Dispatcher.IntegrationEvents
• Masa.Contrib.Dispatcher.IntegrationEvents.Dapr
• Masa.Contrib.Dispatcher.IntegrationEvents.EventLogs.EFCore

The above has not been actually verified. If you are interested, you can try it. Welcome to mention it at any time.pr

The receiving party does not support outbox mode

I want direct access RabbitMq, but I don't have outbox mode myself, so how can I do it?

Since Masa.Contrib.Dispatcher.IntegrationEventsthe outbox mode has been provided, if we only want to replace an implementer who publishes events, then we only need to IPublisherimplement

1. Create a new class library Masa.Contrib.Dispatcher.IntegrationEvents.RabbitMq, add Masa.Contrib.Dispatcher.IntegrationEventsproject references, and installRabbitMQ.Client

dotnet add package RabbitMQ.Client //使用RabbitMq

1. Add new class Publisherand implementIPublisher

public class Publisher : IPublisher
{
    public async Task PublishAsync<T>(string topicName, T @event, CancellationToken stoppingToken = default) where T : IIntegrationEvent
    {
        //todo: 通过 RabbitMQ.Client 发送消息到RabbitMq
        throw new NotImplementedException();
    }
}

1. Create a new class DispatcherOptionsExtensionsand register the customization Publisherto the service collection

public static class DispatcherOptionsExtensions
{
    public static DispatcherOptions UseRabbitMq(this Masa.Contrib.Dispatcher.IntegrationEvents.Options.DispatcherOptions options)
    {
         //todo: 注册RabbitMq信息
         dispatcherOptions.Services.TryAddSingleton<IPublisher, Publisher>();
         return dispatcherOptions;
    }
}

1. How to use a custom implementationRabbitMq

builder.Services.AddIntegrationEventBus(option =>
{
    option.UseRabbitMq();//修改为使用RabbitMq
    option.UseUoW<UserDbContext>(optionBuilder => optionBuilder.UseSqlite($"Data Source=./Db/{Guid.NewGuid():N}.db;"));
    option.UseEventLog<UserDbContext>();
});

Source code of this chapter

Assignment12

https://github.com/zhenlei520/MasaFramework.Practice

open source address

MASA.Framework：https://github.com/masastack/MASA.Framework

---

If you are interested in our MASA Framework, whether it is code contribution, use, or issue, please contact us

• WeChat：MasaStackTechOps
• QQ：7424099

{{o.name}}

{{m.name}}