Thanks to Alipay's core system practice for more than ten years, OceanBase has been gradually applied to core systems in many key fields such as finance, operators, government, and energy since it was officially commercialized in 2017.
In the process of going deep into the core applications of enterprises, we gradually discovered that in order to become a qualified enterprise-level product, it is not enough to rely solely on a powerful database kernel. Only through a complete product tool system can we truly help enterprises to be more secure and efficient solve data management problems.
Therefore, we put the construction of the OceanBase tool system in an equally important position as the kernel. We hope to provide customers with automated, refined and systematic data management solutions through a complete set of tool systems covering the entire lifecycle management of data.
With the assistance of the tool system, ordinary users no longer need to master in-depth architecture knowledge, and can also perform management work efficiently, which greatly reduces the difficulty for users to get started.
The OceanBase tool system has provided one-stop graphical capabilities including migration assessment, data synchronization, data integration, development and debugging, monitoring and operation and maintenance. At the same time, continue to strengthen enterprise-level management capabilities, keep the interface open, and effectively help enterprises use the OceanBase database efficiently, stably, safely and conveniently.
Shortly after the release of OceanBase Server 3.2, as the most important ecological tool product connecting database users and database kernels, the OceanBase 3.X tool family was released on January 6, 2022. In terms of intelligence, ease of use and security All have been comprehensively improved and improved.
1. Operation and maintenance management platform OCP
Under the OceanBase distributed architecture, the number of nodes continues to grow with business development, and the deployment environment gradually changes from physical hosts to various virtualization facilities, private clouds, public clouds, and even containers, which brings database operation and maintenance, management, and monitoring. The complexity of the enterprise customers are facing new challenges.
OceanBase Operation and Maintenance Management Platform (OceanBase Cloud Platform, OCP for short) is an enterprise-level product tailored for OceanBase database clusters. Provides graphical monitoring management and diagnostic capabilities for OceanBase clusters, including full lifecycle management, fault recovery, performance diagnosis, monitoring alarms, and intelligent optimization of database components and related resources. It is designed to help customers manage OceanBase clusters more efficiently, reduce the IT operation and maintenance costs of enterprises and the learning costs of users.
OCP closely adheres to the system features and version functions of OceanBase distributed database, and supports the safe production of OceanBase database with three tricks: monitorable, rollbackable, and grayscale capable, effectively solving many pain points encountered in current database monitoring, diagnosis and operation and maintenance management , becoming the best choice for enterprise customers to operate and maintain OceanBase.
OCP performance monitoring example
Sample OCP Database Performance Report
In the new version, OCP has added a new active session history report, which provides database historical active session information, assists users, and performs tenant performance diagnosis; backup recovery supports secondary backup, and also optimizes the visual content of OCP users with different permissions, enabling users What you see is what you get. OCP has added the ability to take over OceanBase Proxy, realizing the integrity of the takeover link. In addition, the SQL diagnosis adds an export function, which is convenient for customers to analyze problems and so on. (* See the official website for complete product update records)
01 "Monitoring-alarm-diagnosis" full-link integration, one-click processing of intelligent diagnosis
Alarm monitoring and fault diagnosis are basic capabilities provided by OCP.
In the new version released this time, through intelligent correlation analysis, OCP innovatively organically combines several continuous links of "monitoring-alarm-diagnosis". Starting from monitoring or alarming, it adopts a humanized guidance method to help customers quickly find the root cause of the fault, and provides corresponding suggestions and one-click processing capabilities.
For example, alarm information in performance monitoring can be intelligently associated with diagnostic information of clusters, tenants, and nodes. When the QPS RT of a certain cluster suddenly increases, the observer process with abnormal time-consuming can be located through intelligent drilling on the monitoring page, and further expand the observer process information, you can see the abnormal IO time-consuming of the host where the process is located, and finally locate A physical disk fails, and the customer is advised to replace it.
This update adds richer troubleshooting methods. For example, in daily database operation and maintenance scenarios, SQL performance degradation is the most common problem. There are various possible reasons, including: execution plan changes, SQL execution blocking, hot row locks, and so on. For different reasons, OCP provides targeted repair capabilities for customers to choose: by binding historical plans, performance degradation caused by changes in execution plans can be repaired; through SQL diagnostic capabilities, keyword information that leads to hot rows can be found, and further Limit the flow of keywords, and control the number of concurrent requests for this hotspot value at the database level. Through the SQL concurrency rate limiting and keyword rate limiting capabilities provided at the database level, customers no longer need to rely on traditional application rate limiting, which simplifies the design of the application architecture.
02 Unique remote disaster recovery deployment, continuous online management platform
With the continuous enhancement of OCP functions, it has become an integral part of the entire data management system. At the same time, more and more enterprise customers require OCP to provide the same disaster recovery capability as OceanBase. In the face of failures and disasters, customers not only need the database service to be online, but also need the management platform OCP to maintain the service.
OCP provides remote disaster recovery capabilities in this update. When OceanBase is deployed in multiple locations, the OCP cluster can also be deployed across regions. When the OCP cluster in one of the cities fails, the other OCP cluster will undertake the operation and maintenance, monitoring and alarm capabilities of the OceanBase cluster to ensure that the management platform remains online.
2. Data Migration Synchronization Service OMS
Under the trend of continuous and accelerated digitization of the entire industry, heterogeneous data terminal transmission and big data governance have become challenges for enterprises in the digitalization process. Data migration, data synchronization, and integration and management of heterogeneous data have become inevitable tasks in the digital transformation of enterprises.
OceanBase Migration Service (OMS for short) is a one-stop data transmission and synchronization product of OceanBase. It supports multiple relational databases, data replication between message queues and OceanBase, and is a data transmission service that integrates data migration, real-time data synchronization and incremental data subscription.
Data Migration Service OMS Function Description
3.2.2 is a milestone version of OMS. This version consolidates the data replication capabilities between Oracle, DB2, MySQL and other databases and OceanBase. The product functions are more complete and the performance is comprehensively improved. In scenarios such as large data volumes and large transactions, in-depth optimization has been carried out on multiple levels such as product stability and resource consumption. 3.2.2 Provides the industry-leading automatic synchronization function of incremental DDL between heterogeneous databases; improves the continuity of database replication through intelligent means; overcomes the problem of automatic operation and maintenance synchronization tasks. At the same time, new support for cross-regional deployment is added, so as to more conveniently and efficiently support multiple business scenarios such as database migration and version upgrade, database cross-regional disaster recovery, business active-active, and real-time data analysis. (* See the official website for complete product update records)
OMS interface example
01 Strategizing, OMA predicts potential migration risks in advance
-
Data Migration Assessment OMA (OceanBase Migration Assessment) is an important part of OMS. It can evaluate the compatibility between SQL syntax and database objects, display the results through visual reports, and provide suggestions for modification.
-
In addition to evaluating static objects, OMA can also replay the workload of the source database on the target database to estimate the performance after migration, discover performance bottlenecks in advance, optimize SQL with performance problems, and provide support for the success of the actual migration. Assure.
With this update, OMA will be involved in the entire lifecycle of database system migrations.
-
Compatibility assessment: complete the database portrait of the source database through OMA, which is convenient for users to understand the current database topology, application topology and overall database load, so as to formulate corresponding migration strategies.
-
Performance evaluation: Capture the source database SQL through OMA or analyze the load file, replay the corresponding SQL to the target OceanBase database, verify the correctness and performance of the SQL for a long time, and provide automatic transformation and automatic optimization suggestions. To help OceanBase find and solve compatibility and performance problems. In addition, OMA also supports the load playback function to perform stress testing on the simulated real traffic or multiple traffic of the OceanBase database, evaluate the stress testing performance of the OceanBase database, and provide reports to facilitate decision-making for users and reduce migration risks.
-
When the OceanBase database is upgraded or changed: use OMA's data playback function to test the performance of the new version of the OceanBase database and provide reports to reduce the risk of upgrades and changes.
Sample OMA Migration Assessment Report
02 Off-site multi-active data replication technology, both high availability and high performance
Data synchronization is one of the important functions of OMS, and remote multi-active is the most common scenario faced by data synchronization tools. In order to better support data synchronization for business in remote multi-active and cross-city disaster recovery scenarios, OMS has newly added support for multi-regional deployment. Users can flexibly configure resource nodes in various regions, with the help of OMS's powerful data replication capabilities , The application can realize reading and writing nearby, which greatly reduces the consumption of network bandwidth transmission.
An important technical problem faced in remote multi-active scenarios is to prevent data from being cyclically written in complex topologies. OMS adopts the original transaction traffic identification technology, which can distinguish in the data flow whether the traffic comes from business writing or synchronous platform writing, so as to avoid repeated synchronization of data among multiple nodes. While ensuring that business writes in multiple regions will be synchronized to other regions, it avoids cyclical replication of data.
03 High-performance distributed data verification technology to ensure safe and accurate data replication
Data correctness is a basic requirement for data replication scenarios, especially for the core business areas of the financial industry. The data synchronization platform needs to have the ability to repeatedly verify the correctness of the data to ensure that the data can neither be lost nor wrong during the synchronization process.
In this update, OMS provides high-performance verification capabilities for streaming and batch data, which can be divided into stand-alone and distributed modes, which can meet the diverse usage scenarios of users. In the hybrid replication scenario of multiple data sources, in addition to data consistency verification, one-click correction of data with inconsistent verification can also be performed. Through the distributed concurrent verification technology, OMS can verify the replication links of thousands of instances at the same time, so that the entire verification process maintains ultra-high performance.
04 Continuously improve data replication capabilities to ensure easy migration of business systems
OMS has significantly enhanced data replication capabilities.
In the incremental synchronization scenario, it provides the ability to automatically synchronize DDL to intelligently solve the problem of object definition changes during synchronization of homogeneous or heterogeneous databases. With the help of the intelligent DDL conversion framework, OMS can closely match the core functions and compatibility capabilities of OceanBase, provide professional and accurate heterogeneous conversion rules, and use best practice solutions to complete the automatic conversion of heterogeneous database DDL.
Added support for lightweight ETL capabilities, users can configure rules to filter data according to DML types, and can also customize where conditions to filter data. The data verification function of OMS will automatically process according to the filter conditions to ensure that the whole link adopts a unified filtering strategy for data replication and data verification.
05. A large number of usability optimizations, user experience continues to upgrade
The new version adds dozens of user experience optimizations, such as:
-
Optimize the visual operation process, easily configure the migration synchronization link, provide multi-dimensional system monitoring, and reduce the cost of getting started and maintenance;
-
Provide a more efficient and controllable alarm mechanism to ensure that users keep abreast of link status and possible risks;
-
Supports the selection of migration objects by configuring wildcard rules, which is convenient for users to quickly and batch-set migration objects;
-
The new project label function is convenient for users to use labels to filter, group, manage, migrate and synchronize projects.
3. Developer Center ODC
OceanBase Developer Center (OceanBase Developer Center, referred to as ODC) is an enterprise-level database development platform for OceanBase. ODC supports connecting to MySQL and Oracle tenant databases in OceanBase, and provides database developers with functions such as daily program development, code debugging, SQL tuning and diagnosis, session management, and data import and export. ODC adopts a mature B/S architecture, is cross-platform, lightweight, and easy to deploy. It also provides a desktop stand-alone version, which meets the needs of individual developers to get started quickly and improves the collaboration efficiency between developers and DBAs.
After a new round of optimization and improvement, the new version 3.2 is ushered in. The core goal of ODC 3.2 is to establish a permission model and take the first step towards security control. At the same time, the stability and ease of use have been improved to a higher level, and a lot of optimization work has been done in SQL execution, object interaction, result set viewing and editing, aiming to provide users with a better experience. (* See the official website for complete product update records)
ODC console example
01 Added public resource management to realize enterprise-level security access control
In daily work, database managers often encounter this need: when developers are expected to have permission to access or operate certain libraries, such users are not expected to obtain database account passwords, and their permissions can be revoked in real time. Relying on the database account system cannot meet these requirements. In this case, we can only rely on the external permission system.
As an enterprise database development platform, ODC 3.2 provides the capability of a public resource management console. Only certain users with the administrator role can log in to the public resource management console. After entering the management console, the administrator can create new users and grant rights to users through roles. At the same time, for existing users, the administrator can also modify their basic information, adjust their permissions, and delete them.
The introduction of public resource management and control services facilitates centralized management and control of user connections by administrators. By connecting with the enterprise user authority system, the integrated database connection authority management is realized, which greatly improves the security of the system and the convenience of management.
02Flexible result set interaction capabilities, the experience is closer to classic desktop tools
Traditional desktop development tools usually adapt the result set to external programs, and can realize copy and paste operations with programs such as Excel, which greatly improves the convenience of the development process.
Although ODC runs on a browser, it also optimizes the result set interaction. The new version supports copying and pasting row and column data within the result set and between Excel. When viewed in column mode, it can interact with table data; copy part of the result set content and output it to the clipboard. When the content of the queried result set exceeds the screen display, Ctrl+mouse scrolling is supported to view the content beyond the screen. When the result set is edited, the text field supports line break through Shift+Enter in the cell and supports setting the default value for the field cell.
This iteration of ODC also optimizes the creation and management interaction of functions, stored procedures, sequences, and package objects, and adjusts the editing interaction of session attributes, and adjusts the prompt style when the connection is disconnected, which will no longer affect the existing View the contents of the window.
write at the end
The OceanBase database grew up in the Internet business environment, and the rapid growth of its tool system is inseparable from the production practice of enterprise customers in the core system. Here, we would like to express our heartfelt thanks to all the customers who accompanied the OceanBase tool system along the way.
In the future, we will continue to focus on customer needs, listen to the needs of more corporate customers in real scenarios, and constantly hone the quality and experience of products in practice. The ease of use, stability, security, and functional scope of the product will also continue to improve, and it will play a greater role in the operation and management of the core system of the enterprise.