Hello, I am watching the mountains.
This article is derived from the translation invitation of Concurrent Programming Network. The translation is the event-driven content of Jakob Jenkov's "Software Architecture" . Although it is a 2014 article, it is not out of date from the software architecture level.
The following is the text.
Event-driven architecture is an architectural style in which systems or components interact with each other by sending and responding to events. When an event occurs, component A does not directly call component B, but only sends out an event. Component A does not know which components listen for and handle these events. Event-driven architecture can be used within and between processes. For example, event-driven will be used extensively in GUI frameworks. [Translator's Note: Many systems currently use microservice architecture, and event-driven applications are more widely used. In addition, as I mentioned in the concurrency model tutorial , the assembly line concurrency model (AKA reactive, non-blocking concurrency model) also uses an event-driven architecture.
This article mainly introduces the event-driven architecture between processes. When the term is mentioned later, it also refers to the way of process interaction.
Event-driven architecture between processes
Event-driven architecture is an architectural style. First, request events are stored in one or more event queues, and then events are forwarded from these event queues to back-end services to process these events.
Because events can be regarded as message flows, event-driven architecture is also called message-driven architecture or stream processing architecture. Stream processing architecture can also be called lambda architecture. In order to ensure uniformity, the term event-driven will continue to be used in the following text.
Event queue
In an event-driven architecture, you will have one or more centralized event queues, and all events will be stored in the centralized event queue before being processed. A simple example is given below:
When events are inserted into the queue, they are ordered so that they can be processed sequentially.
Event log
When writing to the event queue, messages may be written to the event log (usually disk storage). If a system crash occurs, the system only needs to replay the event log to recover to the state before the crash. The following is an example of an event-driven architecture, which includes an event log for persistent events:
We can also back up the system state by backing up the event log. Before deploying the new version of the system in a production environment, you can use this backup data to test its performance. Or, replay some errors by replaying the backup of the event log.
Event collector
Requests are all transmitted over the network, such as HTTP or other protocols. In order to be consistent, events from different sources can be received through the event collector. The following is an example of an event-driven architecture with an event collector added:
Response queue
Sometimes, we also need to return a response to the request (that is, an event). Therefore, many event-driven architectures also have a response queue in addition to an event queue. The following is an example of an event-driven architecture that includes an event queue (enqueue queue) and response queue (dequeue queue):
As you can see, the response queue must be routed to the correct event collector. For example, if an HTTP collector (essentially a web server) sends a request received via HTTP to the event queue, the response generated by the event may also need to be sent back to the client via the HTTP collector.
Normally, the response queue will not be persistent, which means that it will not be written to the event log, and only input events will be persisted to the event log.
Read event vs. write event
If all incoming requests are considered events, these events need to be pushed to the event queue. If the event queue is persisted (persistent to the event log), it means that all events need to be persistent. Usually persistence is slow. If we can filter out some events that do not need persistence, we can improve the performance of the queue.
The reason why we persist the event to the event log is that we can replay the event log and reconstruct the system state changes caused by the event. In order to support this feature, it is actually only necessary to persist events that change the state of the system. In other words, we only need to divide events into read events and write events. The read event only reads the system data and does not change it, and the write event changes the system data.
By dividing events according to reads and writes, we only need to persist the messages of write events. This will improve the performance of the event queue, and the increase ratio depends on the ratio between read and write events.
In order to divide the event into read and write events, it is necessary to make a distinction before the event arrives in the event queue, that is, in the event collector. Otherwise, the event queue cannot know whether the arriving event needs to be persisted.
You can also split the event queue into two, one event queue for storing read events, and one event queue for storing write events. In this way, reading events will not be slower than writing events, and the event queue does not need to check whether each event needs to be persisted. The read event queue does not need to be persisted, and the write event queue always persists events.
The following is an example of an event-driven architecture, where the event queue is divided into read and write event queues:
The arrows in the example above are messy, but it is actually much easier to create 3 drop columns and distribute messages between them.
Challenges of event log replay
A big advantage of the event-driven architecture is that in the event of a system crash or system restart, only the event log needs to be replayed to rebuild the system state. This is a great advantage when the log can be replayed independently of time and surrounding systems.
However, replaying event logs completely independent of time is sometimes difficult to achieve. Next, we will introduce some challenges of event log replay.
Processing dynamic data
As mentioned earlier, system data may be modified when writing event processing. In some cases, this kind of data modification is affected by dynamic data during event processing. For example, the date and time the event was processed or the currency exchange rate for a specific date and time.
These dynamic data will cause difficulty in event replay. If the event log is replayed at different times, the service handling the event may parse different dynamic values, such as other dates and times or other exchange rates. Therefore, if the event log is replayed on a different date, there may be inconsistencies between the reconstructed system data and the data generated by the original processing event.
To solve the problem of dynamic data, you can let the write event queue mark the required dynamic data in the event. However, to implement this scheme, the event queue needs to know what dynamic data each event message requires. This will complicate the design of the event queue. Every time new dynamic data is needed, the event queue needs to know how to find these dynamic data.
Another solution is that the write event queue only marks the date and time of the event on the write event. Using the original date and time of the event, the service handling the event can find the dynamic data corresponding to the given date and time. For example, you can query the current exchange rate through the original date and time. This requires services that handle events to query dynamic data based on date and time, but this is just an ideal state.
Interaction with external systems
Another challenge of event log replay is coordination with external systems. For example, the event log contains an order from an e-commerce platform. When this event is processed for the first time, the order needs to be sent to an external payment gateway to charge the customer's credit card.
If you replay the event log, you don't want to charge the customer again for the same order. Therefore, it is not desirable to send the order to an external payment gateway when the event is replayed.
Event log replay solution
Solving the replay event log problem is not easy. Some systems have no problems and can directly replay the event log; some systems may need to know the date and time of the original event; some systems may need to know more similar to the original data obtained from the external system during the original processing of the event.
Replay mode
In any case, any service that listens to events in the event queue must know whether the incoming event is the original event or the replay event. In this way, the processing service can determine how to process dynamic data or how to interact with external systems.
Multi-step event queue
Another solution is to use a multi-step event queue. The first step is to collect all write events; the second step is to analyze dynamic data; the third step is to interact with external systems. If you need to replay the event log, just skip the first and second steps and replay the third step. How to implement it depends on the specific system design.
Hello, I am Kanshan, public account: Kanshan’s Lodge, a 10-year-old backend, open source contributors to Apache Storm, WxJava, and Cynomys. Main job: programmer, part-time job: architect. Swim in the code world, enjoy life in drama.
Original link: Event-driven Architecture
Translation: https://www.howardliu.cn
Translation link: Software Architecture-Event Driven Architecture
CSDN Homepage: http://blog.csdn.net/liuxinghao
CSDN Blog Post: Software Architecture-Event Driven Architecture
