Event-driven system architecture has long been commonplace in daily platform development, sending events through message queues, and then constructing corresponding producers and consumers respectively. However, in the traditional business development model, different events have different formats, and the event formats generated by different producers are also different. The formats that consumers can consume are also very different. Essentially, events, producers, and consumers are still coupled. of. How to solve this problem? That is the CloudEvent we are going to talk about today.
Introduction to CloudEvent
From the description of CloudEvents on the official website, we can see that CloudEvent is essentially a specification for describing event data. Therefore, for the study of CloudEvents, sometimes we should understand its design specifications more than the data structure form it presents. Just like everyone learns the tcp protocol, you are not going to learn what the field is, but to understand why there is this field and what is the problem it solves.
How to decouple
For the learning of CloudEvents, the author adopts a top-down approach to learning, that is, first understand how CloudEvents decouples events, consumers, and producers on the platform, and then think about the details of the underlying related fields 
an event The life cycle of cloudevent usually includes three links of production, transmission, and consumption. Below we will introduce the differences between cloudevent and traditional event development models.
Event production
In the traditional development model, the events produced by different businesses are also different, and the event itself will have relatively less data, more of a signal transmission role, that is, notify the back-end service that a certain type of event has occurred, and then The context data of the event is constructed by the corresponding system, and the business logic is processed. In Cloudevents, more attention is paid to the consistency and completeness of events. 
In order to ensure that events can be uniformly distributed, parsed and processed, Cloudevents adopts a similar layered event encapsulation mechanism, namely "event protocol" and "event data" two layers. The event protocol means that Cloudevent defines the format of the underlying event, that is, everyone encapsulates the event according to a set of standard specifications, so that the event can be processed and distributed uniformly. The event-specific data is stored in the corresponding data field
Integrity is my personal understanding, that is, the events we build in the Cloud environment need to include their current complete context data, so that the subsequent system has enough information for business logic processing and decision-making. This can avoid the need for the back-end system to assemble the context corresponding to the current event after receiving the event, which is mainly to solve the problem that the related data may no longer be in the state when the event occurred due to the delay in transmission, and the state inconsistent
Event transmission
After the event is generated, it is usually sent to the corresponding message broker service for temporary storage. In traditional business, a specific message protocol is usually selected for transmission, which usually involves two parts: serialization and transmission protocol. 
In the transport protocol, Cloudevents supports common industry standard protocols such as HTTP, AMQP, MQTT, SMT, etc., and supports common vendors and platforms such as kafka, AWS Kinesis, Azure Event Grid, Alibaba Cloud EventBridge, and users can follow their own The supplier corresponding to the scene selection distributes the corresponding event
In terms of serialization, cloudevents supports common standard protocols such as HTTP, AMQP, Kafka, etc., without requiring users to manually serialize related protocols
Event consumption
The consumer of the event is usually interested in the type of event it is concerned about, and because the format of the message is uniform, we can easily route the message according to the content in the message body through the corresponding platform, and distribute it to the corresponding event consumer , The consumer of the event only needs to be responsible for receiving the corresponding event without paying attention to other information
More about the Cloudevents event, I will continue to share it later, and then I will introduce how we designed the system based on cloudevent.
Getting started scene
In the previous article, we introduced our service catalog system. Different basic services should be connected to the service catalog. The formats of the basic services are different, and the system of billing and efficiency statistics should be connected. It may be connected later. The company’s event stream platform, how to uniformly distribute and process the heterogeneous data in these heterogeneous modules, our architecture is as follows:
Message processing flow
First, at the message sending end, we build the corresponding message based on cloudevent, and encapsulate the context data of the current event into data, and then send it to the company's message queue system. The company’s message queue completes the corresponding event distribution and routing. The corresponding event receiver only needs to define the events that it cares about, and does not need to monitor the specific MQ. It only needs to define an HTTP interface that accepts messages, corresponding to the message The routing and distribution functions are implemented by the company's MQ
service consumer end to parse the event information passed by the message queue, parse out the corresponding data structure, and then perform business processing. If you add modules or add new event consumption requirements in the future, you only need to implement the corresponding logic.
data structure
Combined with the specifications of Cloudevents, we define the data structure of our own internal system. The main structures used are as follows:
Here we mainly introduce some of our additional fields and the definition according to our own scenario:
type
the surface Source and type describe the current system events, except that the type is a structured data, in accordance with the structure of our corresponding charging efficiency statistics module, you can get this data to do Some branch logic is dealt with.
resources: The changed resource list
is to identify which related resource changes are triggered by the current event, such as adding a hard disk to a virtual machine, which actually contains two kinds of resources, the virtual machine and the corresponding disk resource
Integrated service catalog
As mentioned earlier, we implemented a service proxy module using the service and the provided API specifications. The main usage scenarios of Cloudevent in the service proxy module are as follows: 
1. After the service catalog receives the service change request and saves the database, the corresponding cloudevent event occurs To the message queue
2. Set the corresponding routing and forwarding rules in the message queue, and send the corresponding event to the service agent module
3. The service agent module parses according to the type field, obtains the corresponding back-end service address, and parses it from the message Output the corresponding data and send the data to the real back-end service
4. After the real back-end service receives the structured data, it processes its own business logic, and sends the corresponding event after the processing is completed.
5. The service proxy module parses out according to the event Related resources, call the corresponding platform to obtain the data of the current resource, and generate the event
6. The service catalog module receives the corresponding service instance data and stores it in its own database
If there are subsequent changes, only the corresponding event needs to be generated to the message queue, and stages 5-6 will be repeated
Although the link is a bit long, the system design of the entire link is actually very simple. There is no need to pay attention to the communication, reliability, fault tolerance, and coupling between the systems (message queuing service guarantees). If you want to expand in the future, then you will A module will do. To consume new events, write a new interface and edit the routing rules to realize the access of new modules.
Follow-up
Recently, I was in poor health. I just finished going to the hospital today, so I can’t stay up all night. I hope that I can catch up with the project schedule in the future, and there will be more scenarios later. If the schedule catches up a year ago, I will take the time to write the code practice of the application management module that I want to do, and then give it to Let’s share the application management based on the above-mentioned practices, but if the project is cut off, this is it. When the concept of the upper level has been introduced recently, I will share some of the source code design with you. If you are friends with me in similar directions, you are also welcome to add WeChat to communicate.
Cloud native study notes address: https://www.yuque.com/baxiaoshi/tyado3
WeChat ID: baxiaoshi2020 Public number: Graphical source code