Abstract: A difficult problem that combined applications need to face is how to solve the problem of integration standards between various applications. For example, applications may only support one of HTTP, TCP and other protocols, and the lack of a unified communication standard will bring business to the architecture. Difficulty comes. The following describes how the event grid (EventGrid) solves this problem.
In the new era of SaaS, business adaptability requirements lead enterprises to turn to technical architectures that support rapid, secure and efficient application changes. As a key technology to accelerate digitalization, composable application is one of the important strategic technologies proposed by Gartner in 2022. It is built from business-centric modular components, enabling technology and business teams to be more agile and more effective for reuse code. One of the challenges that a combined application needs to face is how to solve the integration standard problem between various applications. For example, an application may only support one of the protocols such as HTTP and TCP, and the lack of a unified communication standard brings business to the architecture. difficulty. The following describes how the event grid (EventGrid) solves this problem.
Event Grid is a new-generation serverless event bus launched by HUAWEI CLOUD middleware in the cloud-native era. It carries and manages various events from traditional applications, cloud service applications, and SaaS partner applications, helping application developers to easily build high-availability, Loosely coupled, distributed event-driven architecture. As a key part of the event-driven architecture under the serverless architecture, it provides elastic, asynchronous, and decentralized event governance services, and provides core functions such as aggregation, model verification, filtering, routing, transformation, and push for events, as well as fault-tolerance. Enhanced features such as testing, dead letter storage, event query tracking, event workflow, etc.
As can be seen from the overall architecture diagram of EventGrid above, EventGrid connects a series of cloud services as event sources. These cloud services include distributed messaging services (such as RocketMQ, Kafka, and Pulsar), object storage services (OBS), and distributed cache services (Redis). The event source will generate management classes, data classes and custom events, and then push them to the event channel of the EventGrid's event bus (Bus Runtime) through the EventGrid SDK. The event bus configures event channels for EventGrid users on a tenant-by-tenant basis. A tenant allows multiple event channels to carry events from different event sources. For example, the default event channel stores events generated by HUAWEI CLOUD, while the custom event channel stores events of applications and microservices. EventGrid users consume events by subscription. By defining event sources, event targets, event filtering and conversion rules in the subscription configuration items, EventGrid can extract relevant events from the event bus and push them to the defined event targets in real time. . The push method can be synchronous or asynchronous: synchronous push is generally based on the HTTP protocol, which is suitable for applications and microservices; asynchronous push is generally pushed to the message queue of SaaS partner applications.
Therefore, HUAWEI CLOUD EventGrid links surrounding cloud services, cloud-native applications, and SaaS partner applications in an event-driven manner, realizes decoupling between services and applications, focuses on the advantages of its own services, and connects through various event models for HUAWEI CLOUD Create more application scenarios and enrich the developer ecosystem of HUAWEI CLOUD.
Using EventMesh as the engine
HUAWEI CLOUD EventGrid has introduced the open source star project Apache EventMesh as its runtime engine. As a cloud-native event-driven architecture middleware, Apache EventMesh is used to separate applications and back-end event stores (Event Store), supporting a wide range of application scenarios, including hybrid clouds and traditional data centers, as well as distributed architectures with different technology stacks.
Apache EventMesh is similar in concept to EventGrid. It also connects to event sources, aggregates events, and pushes them to clients. Its highlight is that its kernel can support plug-in. According to different event application scenarios, different plug-ins can be connected to match. For example, in the event connector (Event Connector), you can connect RocketMQ Connector or Kafka Connector. In terms of HTTP authorization (Http-Auth), BasicAuth or TokenAuth can be referenced. Such a flexible architecture helps create an ecosystem that connects different products and services.
EventMesh gRPC features
The latest v1.4.0 release of Apache EventMesh adds several important features and architectural optimizations compared to previous versions. One of the highlights is support for gRPC. gRPC is a modern high-performance RPC (Remote Procedure Call) framework based on HTTP/2. Compared with HTTP protocol, gRPC supports two-way asynchronous communication between Client and Server, defines API interface data model through Protobuf, and supports multi-language SDK. By implementing the gRPC protocol, Apache EventMesh can integrate the original TCP and HTTP protocols, making the current runtime lightweight. At the same time, its SDK can be extended to support multiple languages, such as Java, Go, JavaScript and so on.
Protobuf Design
The introduction of EventMesh into gRPC starts with designing the Protobuf definition (eventmesh-client.proto). We define the method and event model of the EventMesh event sending and subscription service in the eventmesh-client.proto file. Due to space reasons, the following only introduces the key parts of the definition. For the complete document, please refer to:
1. The event sending service provides the following interfaces.
service PublisherService {
rpc publish(SimpleMessage) returns (Response);
rpc requestReply(SimpleMessage) returns (SimpleMessage);
rpc batchPublish(BatchMessage) returns (Response);
}Events are presented in the SimpleMessage data model. Event sending supports three modes: synchronous sending, asynchronous sending and batch sending. Among them, synchronous sending means that the event producer sends events to EventMesh, and waits for the event to be successfully pushed to the event consumer, and receives the return of the event consumer before completing the entire end-to-end event sending process; asynchronous sending means that the event producer You can send events to EventMesh without waiting for the event to be successfully pushed to the event consumer; batch sending refers to asynchronously sending a batch of events to EventMesh.
2. The event model is as follows:
message RequestHeader {
string env = 1;
string region = 2;
string idc = 3;
string ip = 4;
string pid = 5;
string sys = 6;
string username = 7;
string password = 8;
string language = 9;
string protocolType = 10;
string protocolVersion = 11;
string protocolDesc = 12;
}
message SimpleMessage {
RequestHeader header = 1;
string producerGroup = 2;
string topic = 3;
string content = 4;
string ttl = 5;
string uniqueId = 6;
string seqNum = 7;
string tag = 8;
map<string, string> properties = 9;
}The event model can support a variety of protocols, including CNCF CloudEvents, OpenMessenging and EventMesh native event protocols, which are reflected in the protocol-related fields of SimpleMessage. In addition, the model has producerGroup and topic for event routing; ttl, uniqueId and seqNum are used for event management. The request header carries the basic information of the event producer: env, region, idc, ip and sys.
3. The event subscription service provides the following interfaces:
service ConsumerService {
rpc subscribe(Subscription) returns (Response);
rpc subscribeStream(stream Subscription) returns (stream SimpleMessage);
rpc unsubscribe(Subscription) returns (Response);
}Event subscription supports two methods: cluster and broadcast. In cluster mode, only one instance in the event consumer cluster can consume events; in broadcast mode, every instance in the cluster consumes events. These subscription modes are defined in the subscription data model. In addition, the subscription service provides two subscription interfaces: subscribe API and subscribeStream API. Subscribe API pushes events to consumers through url, where url is also called webhook. This scenario is suitable for cloud-native microservices and custom applications and functions. The subscribeStream API pushes events to consumers through gRPC Bidirectional Streaming, and allows event consumers to return acknowledgment information (Ack) to event producers. This satisfies the needs of the producer RequestReply to synchronize event delivery.
Server-side multi-threaded concurrency
In order to improve the performance of event production and consumption, the EventMesh server (EventMesh Runtime) defines a thread pool (ThreadPool) in the gRPC service, and configures independent parameters for the different performance requirements of event production and consumption. These parameters can be found in the EventMesh configuration file (eventmesh.properties). For example, the following are the number of threads for event production, subscription and push, respectively.
eventMesh.server.sendmsg.threads.num=50
eventMesh.server.clientmanage.threads.num=30
eventMesh.server.pushmsg.threads.num=50When the gRPC service is started, it will listen to the client's request. Once a new request comes in, it will be distributed to the thread pool of the corresponding service, and then processed by the corresponding processor, so that the processing of the next request will not be blocked. , thereby increasing the amount of concurrency.
public void publish(SimpleMessage request, StreamObserver<Response> responseObserver){
cmdLogger.info("cmd={}|{}|client2eventMesh|from={}|to={}", "AsyncPublish",
EventMeshConstants.PROTOCOL_GRPC, request.getHeader().getIp(),
eventMeshGrpcServer.getEventMeshGrpcConfiguration().eventMeshIp);
EventEmitter<Response> emitter = new EventEmitter<>(responseObserver);
threadPoolExecutor.submit(() -> {
SendAsyncMessageProcessor sendAsyncMessageProcessor = new SendAsyncMessageProcessor(eventMeshGrpcServer);
try {
sendAsyncMessageProcessor.process(request, emitter);
} catch (Exception e) {
logger.error("Error code {}, error message {}", StatusCode.EVENTMESH_SEND_ASYNC_MSG_ERR.getRetCode(),
StatusCode.EVENTMESH_SEND_ASYNC_MSG_ERR.getErrMsg(), e);
ServiceUtils.sendRespAndDone(StatusCode.EVENTMESH_SEND_ASYNC_MSG_ERR, e.getMessage(), emitter);
}
});
}For example, the above code is the implementation of the event sending service (publish service). It uses threadPoolExecutor to send events to the thread pool for downstream SendAsyncMessageProcessor to process.
Load balancing and retries for event consumers
On the event consumer side, EventMesh supports load balancing. Users can specify multiple event consumers in the event subscription to consume events. Each consumer will be a URL (Webhook) or gRPC client (stream), and these consumers can be deployed in a cluster or in an active-standby mode. Then EventMesh will select one of the consumers to push events through algorithm calculation. If the current consumer cannot consume events, EventMesh will select the next consumer. By default, EventMesh selects consumer push events in a polling manner to achieve load balancing for consumers. Developers can select consumers by providing other algorithms through plugins.
The event push failed, possibly due to network reasons, the current consumer is busy or offline. EventMesh tries to retry first, with a few seconds interval between each retry, and a maximum of 3 retries. If 3 retries fail, EventMesh will choose the next consumer. The retry interval can be defined in the configuration file.
gRPC non-Blocking 和 Blocking Stub
To improve client performance, gRPC provides non-Blocking Stubs. After using the non-Blocking Stub client to send a request, you do not need to wait for the server to reply, and proceed to the next step. This is suitable for scenarios where a large number of events are produced. However, in the event RequestReply scenario, the client needs to wait for the event to be successfully pushed to the event consumer synchronously, and we need a blocking stub of gRPC. By providing the client with two options, it can more flexibly meet different event production and consumption scenarios. At present, EventMesh SDK uses Blocking Stub, and the next step is to use non-Bocking Stub to improve the high concurrency performance of the client.
Support multi-language SDK
Before Apache EventMesh v1.3.0 only the Java SDK was available. This has limitations for accessing other cloud-native application scenarios. Because many cloud services are developed in Go, Python and JavaScript (NodeJS framework). Some cloud-native open source projects, such as KNative and Dapr, also use Go as the development language. Therefore, EventMesh has an urgent need to support multi-language SDKs to continue to enrich the ecosystem of application developers.
The development tools of gRPC come with multi-language code generation tools, including Java, Go, NodeJS, PHP, Python, C#, and C++, etc. After EventMesh implements the gRPC protocol, it can use this tool to quickly provide a multi-language SDK. Subsequent changes to the communication API only need to synchronously modify the Protobuf model definition and regenerate the code. Iteration is fast and maintenance is simple.
Integrate TCP and HTTP protocols
gRPC supports 4 client and server communication RPC mechanisms: Unary RPC, Server Streaming RPC, Client Streaming RPC and Bidirectional Streaming RPC. EventMesh v1.4.0 integrates the v1.3.0 SDK API of TCP and HTTP, using Unary RPC and Bidirectional Streaming RPC provides five SDK APIs, Event Publish, Broadcast, Request-Reply, Webhook Subscription, and Event Stream Subscription.
By integrating TCP and HTTP protocols, EventMesh Runtime and SDK architecture are more lightweight and code maintenance is reduced. At the same time, users do not need to consider which protocol to choose when using the SDK. Users do not need to learn and perceive the communication protocol at the SDK level, gRPC can satisfy all their event production and consumption scenarios.
Out-of-the-box gRPC samples
In order to allow EventMesh developers to better experience the new features of gRPC, the community provides complete documentation and multiple code samples for Event Publisher and Event Subscriber. Here are the relevant documents:
The first document describes how to deploy and start the EventMesh server. The first part of the document describes how to remotely deploy EventMesh in a virtual machine, which is suitable for deployment in test and production environments; the second part describes how to deploy EventMesh in a local development environment, which is suitable for common local development and debugging scenarios. The following command line shows how to deploy and start EventMesh Runtime in a Linux environment
> tar -zxvf Eventmesh_1.3.0-release.tar.gz
> cd bin
> sh start.sh
> tail -f ./logs/eventmesh.outThe second document introduces how to use EventMesh SDK to develop applications to produce and consume events, including TCP, HTTP and gRPC. The EventMesh project has a client-side code sample (eventmesh-example), including the code of gRPC Event Publisher and Event Subscriber. For example, we can import the EventMesh project into a Java IDE (such as Intellij IDEA). Follow the steps below to start gRPC Event Publisher and Subscriber.
1. Modify eventmesh-examples/src/main/resources/application.properties to point to the deployed EventMesh Runtime.
2. Start Event Publisher, run
Eventmesh-example/src/main/java/
org.apache.eventmesh.grpc.pub.eventmeshmessage.AsyncPublishInstance
3. Start Event Subscriber, run
Eventmesh-example/src/main/java/
org.apache.eventmesh.grpc.sub.app.SpringBootDemoApplication
As a new tool for composable applications, the introduction of gRPC is undoubtedly a great boost to integration standardization. Of course, in addition to communication standards, there are also integration standards such as schema and governance that need to be unified. Event Grid will continue to make its own explorations in these areas.
Click Follow to learn about HUAWEI CLOUD's new technologies for the first time~