Ants gold dress data access layer has three core components: data access framework ZDAL, data access agents DBP and OceanBase proxy server OBProxy. Data in this chapter involving ZDAL and OBProxy two components. ZDAL the station as a data access standard components, provides not only a sub-library sub-table, separate read and write, and other standards distributed Sequence application ability, also provides a link trace shadow pressure measurement, a unit, switching and other disaster recovery technology risks. OBProxy as an access entrance OceanBase, providing OceanBase routing address, read and write separation database capabilities, taking into account the implementation of resource efficiency and cost point of view, from the day we are born OBProxy uses a separate process of application deployment mode near the current production environment on hundreds of thousands remain in several levels of the process.
This article describes the current problems by ants gold dress data access layer encountered, solutions ideas, the evolution of the directions of the three aspects, expect to be able to think of some elaborate DB Mesh under development by and allow more students to recognize the DB Mesh. We expect to DB Mesh way data access layer to sink on a unified infrastructure for new business quickly using internal ants gold dress years of technical risks, and can continue to enjoy more performance, dividend and other technical resources .
Background
With the rapid development of business, ZDAL as a client-side component model, there has been a business couple, advance version of iteration, multi-language support and other issues. OBProxy is OceanBase database specifically tailored proxy server business is no natural coupling, supports multiple languages. OceanBase user data stored in the form of multiple copies on each OBServer, OBProxy responsible for receiving the SQL request sent by the user, parses the SQL request and calculates partition information, the request is forwarded to the user the optimal target Observer, and the results to the user. There are sub-library sub-table assembly ZDAL clothes inside the ant gold, used in a multi OceanBase clusters and unitized scene. Both with the use of sub-library sub-table assembly ZDAL doing business layer flow allocation, OceanBase partitioning support level expansion capabilities of the database.
We further look at the core logic ZDAL and OBProxy these two components.
ZDAL core logic:
SQL parser: Get table name and the sub-library sub-table fields;
Rules Engine: computing sub-library sub-table results;
executive level: rewrite the SQL sent to the database, and make the result set and users;
OBProxy core logic:
Protocol parser: Resolution Protocol in SQL, and after Parse partition table name field;
router: Observer where the partition table is calculated;
the IO layer: forwarding requests to OBServer, returning a result set to the client;
As can be seen, the execution path OBProxy and ZDAL these two components have a certain degree of repetition, such as the two components are made SQL parsing and analysis table names and field. This leads to some loss of performance, and this is repeated to bring more iterative development effort.
Based on the foregoing considerations and OBProxy ZDAL both incorporated into one component is a natural solution to the sink by ZDAL OBProxy logic, so that OBProxy sub-library sub-table is provided, separate read and write, sequence distributed middleware capacity, this component we named ODP (Open Database Proxy).
ODP as a process close to the business end of the deployment, although logically independent, upgrade but still need to change the container business, the iterative process of upgrading to promote the work is more laborious and ODP but the entire product tip of the iceberg, requires a lot of effort to build infrastructure under the iceberg, for example, how to solve the problem of ODP operation and maintenance, transparent to the user to switch the program, pushing the issue of complex configuration, the financial level database key management issues. We found the problem with ants inside this part of the gold suit large-scale practice of ServiceMesh encountered a relatively large degree of overlap, so together with ServiceMesh build the underlying infrastructure for this complete product program named DBMesh. Below we will briefly explain the evolution path and direction.
Solutions
Sidecar mode to decouple deployment
From a performance point of view and resource costs, OBProxy ants gold dress has been in the business by way of near-end deployment, the number of daily services to maintain in the hundreds of thousands of levels. Near the business end of the deployment, while providing high efficiency, but it also brings the complexity of the operation and maintenance, at the same time when the need to upgrade version, also need to do together and notification service releases and upgrades. Make a single application upgrades, which are mostly in months, if it comes to a full stop level upgrade, to basically not very good estimate.
But with the cloud and the development of native Kubernetes, so Sidecar mode is more mature, that run in a container next to the container business, this is very fitting that we end business processes nearly OBProxy deployment, simply OBProxy process transformed into OBProxy Sidecar can independently upgrade, release, operation and maintenance. Meanwhile ants gold dress has a very deep practice in the field of cloud native, has the world's largest Kubernetes clusters and cluster ServiceMesh, will OBProxy Sidecar also become more reasonable and facilitate the implementation of the.
Double Ten this year, about 10% of all the tens of thousands of PODs to ODP Sidecar mode, the Sidecar way that allows business to quickly enjoy the benefits of infrastructure software iteration, this two-eleven ODP repair an unexpected lead log printing a room appear slow SQL problems in the traditional local way process, the need to promote all business conduct an iterative, publishing, packaging time period basically not controllable. This year let all the gray applications involving independent, upgrade takes only two days.
Merge duplicate logic technology to get bonus
Solve the problem of deployment model through rapid iteration and independent way to upgrade in order to make it possible functions of the sinking floor. We will most functional sub-library sub-table components are sinking into the ODP, such as: sub-library sub-table function, separate read and write functions, distributed Sequence function, the whole link tracking. As shown below:
After the entire sub-library sub-table component functionality sink, two-eleven this year in our core system on line, get some very impressive technology dividend:
Performance: dispensed additional SQL parsing and routing function by merging the calculations, the average response time SQL system on-line two-eleven hundred microseconds can be lowered;
start speed: ODP and applications only need to establish a connection, without a plurality of sub-library initialization data source, the initialization time from several tens of seconds down to tens of milliseconds;
memory decline: the same speed and start, and the application of the reduced number of connections ODP, also saves memory end use application, since production You can drop hundreds of MB;
Mesh technology to build infrastructure to improve risk
R & D students will be more focus on ODP data link, how to improve the performance of the data plane, how to add more SQL features, and ignore the demands of the non-SQL to perform the link. DBMesh as an infrastructure application side, more to consider how to better manage Sidecar, high security data access prevention and control, to meet the requirements of the three axes configuration changes, the greatest degree of saving resource costs.
We gave birth Mesh technology risk and the ability to build a team of ants gold dress ServiceMesh entire cloud original, Sidecar priority to improve operation and maintenance capabilities and the ability to change three tricks, follow-up will be deployed to the ODP Sidecar resources on the host to the greatest degree of optimization ODP occupation, will gradually sink more pressure measured as a shadow, gray room, traffic mirroring and technical risks.
Summary
DBMesh allow data access from client mode to switch to the agency model, to the application brought great start to optimize speed. Sidecar deployment model is the optimal mode SQL average response time and resource utilization. The more technical risks precipitation in, so that rapid application of new technology to enjoy the level of financial risks, the risk of technical indicators of the stock of the application better. We look forward to a unified data access infrastructure, allowing businesses to use standard technology components, reduce maintenance costs and learning, focusing only on business development and innovation.
about the author
Yi Hongwei (nickname: corrections), research and development work of ants gold dress senior technical experts, is responsible for the data link layer of middleware and OceanBase direction. The main concern in distributed database technology, high-performance servers, database availability, distributed transactions, unit architecture, etc., and micro-services, cloud native, Mesh technology has a certain understanding.
Related Reading
Ants gold dress Service Mesh series of large-scale landing - Gateway chapter
of ants gold dress Service Mesh series of large-scale landing - RPC articles
ants gold dress Service Mesh series of large-scale landing - operation and maintenance articles
ants gold dress Service Mesh series of large-scale landing - news articles
ants Kim served Service Mesh series of large-scale ground - the core articles
Service Mesh landing head pro said: ants gold dress two-eleven four questions