Micro-service architecture, how to build a data reporting services?

Scene Description

In the micro-service architecture, each service is responsible for its own database of micro, micro A service is not allowed to directly connect micro-services database of B operate.

There are two micro-services, a service order is, is a service user.

There is a demand for data reports: Generate a containing user information in the order report .

This requires two data acquisition service, the connection summary.

How to build the data reporting services?

Scheme 1 directly connected to the database

Direct connection to order service, customer service database to obtain the required data, after processing can get.

Very simple, but there are obvious problems.

The first is the destruction of the above mentioned principle that micro-services, even directly to someone else's database, too rude.

There is a more serious problem, change the database table structure if the order is supposed to service and customer service?

Reporting Services must change with it together, the sensitivity is too high.

Scenario 2 data aggregation

No direct data, and call these two services REST API interface to obtain the desired data.

Solve the problem of the previous program, but the biggest problem with this method is poor performance .

Reporting Services requires the latest data, will often visit these two services, with the increase in data size, three service performance will be getting lower and lower.

Scheme 3 Batch pull data

Establish a database to report their own service, using a timing routine, batch pull data from two database service, into its own database.

Solve the problem of the previous program, the performance improved significantly, but like going back to the first issue of the program, undermining the principle of micro-services, but also extremely sensitive to changes in the data table structure.

Benefit is that because of their own database, more convenient, better the performance.

Scenario 4 event push model

Order service, customer service, the data tables further back, generates an event, a message posted to the system (for example kafka), Reporting Services subscriptions related topics, the received data is written to its own database.

benefit:

Loosely coupled, business services and reporting services does not call the relationship, whether it is business interface layer or database layer.
Data consistency, quasi real-time, business services data tables more event messages immediately after sending the report service can quickly consume.
性能好，数据吞吐量增加后，报告服务可以增加处理事件的 worker，提供处理能力。
扩展性好，方便以后添加更多的数据处理需求，例如实时分析，而且，以后可能不止是做订单报告，可能会对更多的业务系统数据进行分析，到时，新服务只需把自己的数据变更事件发送到消息系统中即可。

翻译整理自：

https://medium.com/@muneeb.ahmed20/building-a-reporting-service-in-microservice-architecture-8d5bf3b90fb70

推荐阅读