Database design of micro-services

Separate databases:

One of the key micro-service design database design, the basic principle is that each service has its own separate database, and only micro services themselves can access the database. It is based on the following three reasons.

Optimization Services Interface : The interface between the micro-service small as possible, only the best service invocation interface (RPC or message), no other interfaces. If the service is not exclusive micro own database, the database has become a part of the interface, which greatly expanded the range of interfaces.
Troubleshooting : errors in the production environment and most of them are related to the database, the data is either out of the question, either use the database out of the question. When you can not completely control access to the database, there will be a variety of errors. It may be connected directly to another program data directly with the client to access the database of your database or other departments, and these are finding out in the program, increasing the difficulty of error investigation. If the problem is in the program, as long as the changes to your code, then this error will not have. The errors mentioned above, you'll no prediction when they will happen again.
Performance Tuning : Performance tuning is the same, you need to have full control over the database in order to ensure its performance. If other departments must access the database, and only the query, then you can additionally create a read-only database, so that they are in another database queries, so it will not affect your library.

Ideal design your database only you can access the service, you only call their own data in the database, all access to the other services have come to realize micro (See service call by calling among "the best micro-services design " ). Of course, in practical applications, a simple service call may not meet the performance or other requirements, different micro services are much need to share some data.

Share data:

Data sharing between the micro-services can have under four ways.

Static tables:

There are some static database tables, such as national, may be used in many applications, and internal procedures need to do to connect (join) the table of national data to generate end-user display, such a service call on a micro efficiency is not high, affecting performance. One way is configured in each micro-services in such a table, it is read-only, so you can do a database connection. Of course, you need to ensure data synchronization. In most cases this program is acceptable because of the following two points:

Static database table structure is essentially the same: because once the table structure has changed, you not only want to change the database table all micro services, but also to modify the program for all micro-services.
Data changes in the database table does not frequent: this data is not synchronized with the workload. In addition, when you synchronize the database will always be delayed, if the data does not change frequently then you have a lot to choose from a synchronized manner.

Read-only access business data:

If you need to read other databases in the dynamic business data, the ideal way is to service calls. If you just call the service to do some other micro-computing, performance under normal circumstances it is acceptable. If you need to make the connection data, you can use program code to do, instead of SQL statements. If after the test performance does not meet the requirements, then you can consider yourself to build a database table read-only data. Data synchronization are basically two ways. If it is event-driven, message will be synchronized by the way, if RPC is a synchronous manner mode, use the database itself to provide or third-party synchronization software.

Under normal circumstances, you may only need a few other database tables, each table only a few fields. At this time, other database is the ultimate source of the data, control all write operations and the corresponding business logic validation, we call it the main table. Your library can be called from a read-only table. When data is written to a master table, will send a broadcast message, read-only data in the table all have a listen for messages from the micro-service table and updates. But then you have to be especially careful, because the risk is much greater than the static table. It's the first change table structure will be more frequent, and it changes completely beyond your control. Unlike static second service data table which is frequently updated, so that data synchronization requirements relatively high. To decide how much of the delay according to the specific business requirements is acceptable.

In addition it has two problems:

Volume of data : the amount of data in the database is a major factor affecting performance. Because this data is outside, is not conducive to grasp its traffic laws, it is difficult for capacity planning, performance tuning can not be better.
** ** leaked interfaces: the interface between the micro-services had only service call interface, then you can make any changes to internal procedures and databases, without affecting other services. Now the database table structure has become a part of the interface. Once the interface after the release, basically it can not be changed, which greatly limits your flexibility. Fortunately, because the other built a table, with a buffer, when the main table modification, may not need to synchronize updates from the table.

Unless you can service calls (no local database read-only) way to complete all the features, or whether you are a micro service with RPC mode or event-driven integration, the above mentioned problems are inevitable. But you can change the database through rational planning to reduce the impact of problems caused by the above, the following will explain in detail.

Read and write access to business data:

This is the most complex case. In general, you have a table is the primary table and other tables from the table. Master table contains the main information, and the main message is copied to the table, but there will be micro-services need to write additional fields from the table. Such micro-local service to both read from the table also has write operation. And the main table and the table has a relationship of priorities. It derived from the primary key table master table, a main table and therefore must first, and then from there table.

file

Image Source

The figure is an example. Suppose we have two film-related micro-services, a movie forum, users can comment on the movie. The other is a movie store. "Movie" is a shared table, one on the left is the film forum library, it's "movie" table is the main table. On the right is a movie store library, it's "movie" table from the table. They share the "id" field (primary key). Main table is the main source of data, but the table "quantity" and "price" there is no field in the primary table. After the master table data is inserted, a message, a message received from the table, insert the data into a local "movie" table. And modify the table from the table will be "quantity" and "price" field. In this case, give each field is assigned a unique source (micro-services), only the source of the initiative have the right to change the field, other micro-services can only passively change (change again after changing the source of the message sent by the reception). In this example, the source of "quantity" and "price" field is the right of the table, the source of all other fields of the left table. The present example "quantity" and "price" in the presence of only from the table, the data thus written is unidirectional, from the direction of the main table to table. If the primary table also need these fields, they have to be written back, it becomes a two-way data is written.

Direct access other databases:

This is the way to be absolutely prohibited. Many bugs and performance problems in production environments are generated by this approach. Since the above three methods is another new local read-only database tables, generating the physical isolation of the database, the database so that a performance problem will not affect another. In addition, when the main library table structure changes, you can temporarily remain unchanged from the library table, so that the program can run. If direct access to other people's libraries, the main library a modification, other micro-service program will give an error immediately. See ApplicationDatabase .

Backward-compatible database updates:

As can be seen from the above discussion, modify the database table structure is affecting a wide range of things. In the micro-service architecture, the shared table in the other services there will be a read-only copy. Now when you want to change the table structure, but also need to consider the impact on other micro-services. When the monomer (Monolithic) architecture, in order to ensure the deployment can be rolled back, the database update is backward compatible. Another reason for compatibility is to support the blue-green release (Blue-Green Deployment). In this deployment mode, you have both the old and new versions of the code, determined by each load balancing requests to that version. They can share a database (which requires the database is backward-compatible), you can use different data. Simple database update in terms of the following types:

Increasing the table or field : if the field null Preferably, this operation is backward compatible. If a non-null value must insert a default value.
Delete table or field : can be removed temporarily reserved tables or fields, after several versions and then deleted.
Changes to field names : add a new field, the data field is copied from the old to the new field, the old field and the new field synchronization (used for the transition period) with a database trigger (or program). Then after a few versions to delete them (see the original field Update your Database Schema Without Downtime ).
Modify the table name : If your database supports updatable views, the easiest way is to change the name of the table, and then create an updatable view point to the original table (See Evolutionary Database Design ). If the database does not support updatable view, the methods used to modify the field name is similar, you need to create a new table and make data synchronization.
Review field types : almost the same changes to field names, except when the copy of the data, data type conversion needs to be done.

The benefits of backward compatibility database update is deployed when there is a problem. For rollback. As long as the program is rolled back on the line without having to roll back the database. Generally only roll back a version rollback. Those who need to remove tables or fields at the time of this deployment do not make changes, or wait until a few versions, make sure there is no problem and then deleted. Another advantage is that it will not immediately have a direct impact on other micro-sharing service in the table. When this micro-services upgrade, other micro-updating service can assess the impact of these databases and then decide whether to do the appropriate program or database modifications.

Cross-service things:

A micro-service difficulty is how to achieve cross-service support things. Two-phase commit (Two-Phase Commit) has been unable to meet demand proof of performance, and now basically no one use. It was unanimously approved method called Saga. Its principle is to write a compensation operation (Compensating Transaction) for the things of each operation, one by one and then perform each operation in compensation rollback phase. Example below, a total of three operations T1, T2, T3 in one things. Each operation to define a compensation operations, C1, C2, C3. When things forward order execution is executed first T1, when the rollback is executed in the reverse order of the first C3. Things of each operation (forward operation and compensation operation) are being packaged as a command (Command), Saga execution coordinator (Saga Execution Coordinator (SEC)) responsible for executing all commands. Before execution, all commands are sequentially stored in the log are then removed Saga coordinator execute the command from the log are executed in sequence. When an execution error occurs, this error can also be written to the log, and all commands being executed is stopped, it began to roll back the operation.

file

Image Source

Saga relax the requirements for consistency (Consistency), it can ensure that the final consistency (Eventual Consistency), and therefore the data in the execution things are inconsistent, and this inconsistency can be seen by other processes. In life, in most cases, we ask for consistency and not so high, short-term inconsistency is acceptable. Such as bank transfer operations, which in the implementation process is not executed in a database of things, but rather is divided into two movements by accounting way to perform, but also to ensure that the eventual consistency.

Saga principle seems very simple, but to correct implementation still has some difficulty. Its core problem is that the handling of errors, we should say that it is fully aware of the need to write in another article, I talk about the main points. Network environment is not reliable, the command being executed may not return results for a long time, then, first, you have to set a time-out. Second, because you do not know the reason does not return a value that has been completed but the network command a problem, or did not complete sacrifice, so I do not know whether to perform the compensation operation. The correct approach is time to retry the original command, until completion confirmation, and then perform the compensation operation. But there is a requirement for command, that this operation must be idempotent (Idempotent), which means it can perform many times, but the end result is still the same.

In addition, the compensation operation Some operations are relatively easy to generate, such as payment operations, as long as you return the money on it. But some operations, such as e-mail after completion there is no way back to the state before, and then you can then send a message to correct previous information. Thus compensation operation does not necessarily return to the original state, the original but offset effect generated by the operation. If you want to know more, see here .

Split micro services:

Our original program mostly single program, but now want to split it into micro-services, what should be done to reduce the impact on existing applications it?

file

Image Source

We do use the above example in Fig. It consists of two programs, one is the "Styling app", another is "Warehouse app", they share the following map database, library has three tables, "core client", "core sku", "core item".

file

Image Source

Suppose we want to split up a micro service called "client-service", it requires access to "core client" list. The first step, we first split from the original program code out into a service database does not move, this service is still pointing to the original database. Other programs no longer directly access the service management table, but to get the data service call or build another shared table.

file

Image Source

The second step, a database table and then split out of service, then the service will have its own micro database, and no longer need the original shared database. Then it becomes a micro-service real sense of.

A split above the micro-talk service, if there is more than split, a need to follow a sequence Mentioned above method.

In addition, Martin Fowler in his article "Break Monolith into Microservices" There was a good suggestion. That is, when you split the service from monomers program, do not just want to split up the code. Because now it needs may have been somewhat different from the original, the original design may not quite apply. Moreover, the technology has been updated, the code should be changed accordingly transformation. A better approach is to rewrite the original function (instead of rewriting the original code), to focus on resolving business functions, rather than on the code split, with a new design and technology to achieve this business function.

in conclusion:

Database design is a key point of micro-service design, the basic principle is that each service has its own micro separate database, and only micro services themselves can access the database. Data sharing between the micro service through service calls, or main, from a table of the embodiment. When sharing data, to find the right way synchronization. In the micro-service architecture, extensive modifications affect the database, you need to ensure that this change is backward compatible. Cross-service standard way things are Saga. When micro program into the split service monomers may stepwise, to minimize impact on existing applications.

index:

This article from the blog article multiple platforms OpenWrite release!