OceanBase Developer Center (hereinafter referred to as ODC) is an open source database development and database management collaboration tool. It has been developed for more than three years since the first version was launched. ODC has gradually evolved from a tool specially built for OceanBase. Developer tools have evolved into enterprise-level database collaborative development tools that support multiple data sources. Currently, product capabilities cover multiple directions such as SQL development, change risk management and control, data security compliance, and data lifecycle management.
On August 18, ODC was officially open sourced. The open source protocol adopts Apache License 2.0. Everyone is welcome to join the ODC open source community.
In enterprise-level database collaborative development, security and efficiency often become team collaboration problems. The goal of ODC is to resolve this contradiction between safety and efficiency. In the past year, we have invested a lot of energy in building collaborative management and control capabilities, introduced the concept of integrating R&D and operation and maintenance, and collaborative management and control methods based on project dimensions, and also provided a series of data desensitization, data life cycle management, etc. safety compliance and efficiency improvement capabilities. Next, we will briefly introduce several important directions around the just-released version 4.2.0.
▋Cross -terminal collaboration: efficient SQL collaborative development anytime, anywhere
In enterprise-level application scenarios, SQL collaborative development and data management often involve developers and database administrators using different operating systems (Linux, Windows, Mac). However, such diversity often also creates collaboration challenges, as differences between different operating systems can hinder efficient teamwork. Faced with this pain point, users need a seamless cross-platform solution to ensure that users of various operating systems can participate in collaborative development and data management without any barriers.
It is against this background that ODC provides users with a cross-terminal SQL development experience through modern Web architecture, making it easy to participate in collaborative development regardless of whether you are using Linux, Windows or Mac operating systems. At the same time, it provides powerful support for more than 25 development functions, from SQL execution, object management, data viewing and editing, and even enterprise-level features such as PL debugging, database operation and maintenance, and data generation, making collaboration more efficient and seamless. SQL development capability is a capability that ODC has been building. Currently, this capability covers seven directions: SQL execution, PL life cycle, object management, execution analysis, data import and export, data generation, and database operation and maintenance.
In addition, the latest ODC 4.2.0 version comprehensively upgrades the product form of the SQL development module and provides the following new capabilities, including:
-
Supports switching databases in the SQL window, allowing users to easily reuse the contents of the SQL window;
-
Optimized the parameter input for PL debugging and supports debugging of various PL input parameters;
-
Enhanced database export capabilities, supporting the import of export content compatible with pl/sql developer;
-
It enriches execution analysis capabilities and supports OceanBase version 4.0 and above to view full-link diagnosis at the SQL level.
▋Enterprise -level risk control: Guard every change in the SQL development process
When an enterprise's development team is less than 10 people, it may be possible to control database change risks through trust and real-time communication. But as the team grows in size, the job responsibilities of each team member will be subdivided. For example, the business may have multiple branches, and each branch has corresponding R&D, R&D director, test, test director, DBA, etc. It will be difficult to control change risks by relying on original communication methods.
The database is the core asset of the company. The DBA needs to be responsible for the stability of the database. At the same time, all participants in the project team should be responsible for the data in the database. However, due to the differences in the professional skills and business understanding of each role, it is very easy to do bad things with good intentions, and you may accidentally change the wrong fields or delete the wrong data.
Faced with this situation, the DBA can choose to control the change risk by performing all changes involving the database by himself, but this will inevitably lead to an excessive workload on the DBA. In order to ensure that these tasks can be completed on time, it is necessary to add more DBAs, which will undoubtedly increase the cost of the project. At the same time, DBA is a very technical position. They should invest their energy in more valuable areas instead of spending it on frequent and monotonous changes.
Suppose there is such a platform. When a change is initiated, it is first controlled by students who understand the business, then approved by the business person in charge, then checked by the database person in charge, and then reviewed by the security person in charge. When it is finally executed, this change Are the risks of change effectively controlled? The answer is yes, and ODC is such a platform.
Users with permission to create new projects can create new projects based on business lines after logging into ODC. After the project is successfully created, the project leader needs to add all project team members to the project and assign roles to everyone. At the same time, the project leader also needs to add all the databases used by the business line to the project. At this point, a collaboration unit has basically been successfully constructed.
When a project member initiates a change, it will first go through the SQL specification check. After passing the check, it will be approved by the nodes defined in the approval flow. Only after all nodes pass will it be actually executed. ODC has built-in development specifications covering DDL, DML, and DQL statements. Users can adjust the development specifications according to actual business conditions. SQL specification checking can automatically intercept changes that do not comply with specifications, reduce human intervention, and greatly improve the work efficiency of change participants. At the same time, ODC supports defining the approval process based on the environment where the change is initiated, task type, database name, SQL check results, etc. ODC's change control covers all possible scenarios, including SQL windows, data changes, data export, etc., to ensure the comprehensiveness of change control.
ODC provides full-scenario risk control capabilities. Through built-in comprehensive risk control, it covers all scenarios of the SQL development process. It implements progressive risk control at three levels: access permissions, change review, and stable changes to ensure that the production database change process is stable and efficient. , traceable. For enterprise-level change control, ODC also provides project-based collaboration and change approval processes. Through SQL check rules, SQL window specifications, and risk level identification, it can customize control rules that meet actual business conditions, realize the productization of database R&D specifications, and help users reduce costs. Configuration costs improve collaboration efficiency.
▋Optimize data processes: automatic data lifecycle management
As the business runs for a long time, there will be more and more historical data in the business database. More and more historical data will have the following two negative impacts on the business database: First, the performance of the business database may drop sharply due to the large amount of business table data. Second, storage costs will continue to increase as data grows. To solve this problem, ODC offers three solutions.
The first solution is data cleaning. If the expired historical data business is no longer needed, users can initiate a data cleaning work order on ODC and define a cleaning strategy. After the task is successfully initiated, ODC will regularly clean the data in the table according to the user's requirements.
The second solution is data archiving. If the expired historical data business is still needed, then hot and cold data need to be separated. Periodically migrate cold data in tables to the archive library. Users need to prepare the archive library in advance, then initiate a data archiving work order on the ODC and define the archiving strategy. After the task is successfully initiated, ODC will separate hot and cold data according to user requirements.
A third solution is a zoning plan. This solution is suitable for range/range column type partition tables. ODC provides partition creation and partition cleanup capabilities. Users initiate a partition planning work order on ODC, select tables to be processed in batches, and define partition strategies. After the task is successfully initiated, ODC will periodically clean up and create partitions as defined by the user.
ODC is committed to optimizing enterprise data processes and realizing automated data lifecycle management, thereby improving data processing efficiency and quality. In actual operation, ODC provides a comprehensive hot and cold data separation strategy. Through intuitive interfaces and tools, developers can easily clean or archive data according to the retention period. In just 5 minutes, you can build your Hot and cold data separation system. In addition, ODC also supports efficient automated partition maintenance functions, which can clean and create partitions in batches, effectively reducing the workload of the database administrator (DBA), making the maintenance process of automated partition tables more efficient, and ensuring the availability and performance of the database. Stay in top shape.
Not only that, ODC also provides powerful SQL scheduled task capabilities, which is especially suitable for tasks such as data archiving, data cleaning, and report generation in enterprise scenarios. ODC allows you to easily create, manage and automatically execute SQL scheduled tasks, thereby ensuring the smooth execution of users' periodic calculation tasks and ensuring the stability and accuracy of enterprise data processing. Through these capabilities, ODC is committed to providing integrated data lifecycle management solutions to bring greater efficiency and optimization to enterprise data processes.
▋ Ensure security compliance: reliable all-scenario sensitive data protection
In recent years, there have been more and more national bills targeting personal information security, and accordingly operational auditing and privacy data protection have become increasingly important. For operation auditing, ODC has provided operation recording capabilities a long time ago, and all user operations on ODC will be recorded in this module.
Common private data include the user’s ID number, mobile phone number, home address, etc. These private data are the basis for the normal operation of the business and cannot be stripped from the business library. In order to ensure that user data is not leaked, access to the business library must be strictly restricted. But if it is really completely restricted, it will inevitably lead to collaboration efficiency problems. How to ensure business development efficiency while ensuring privacy data security? ODC offers a solution: data desensitization.
ODC supports desensitization of sensitive fields in the database. For example, the real data of the user's phone number is 1300685abcd. After using ODC desensitization, it can be 130****abcd. ODC's desensitization technology is applied in two aspects: the first is the user's real-time query. The sensitive data that the user accesses on ODC displays the results after desensitization. The second is data export. When users export data from ODC, if sensitive data is involved, desensitized data will be exported by default.
Currently, ODC supports single and batch scanning to add sensitive columns. If there are only a few columns of sensitive data in the business database, you can choose to add each column manually. If there is a lot of sensitive data in the business database, you can configure identification rules and scan out sensitive columns in batches, effectively improving the adding efficiency. In the future, ODC will also provide more intelligent sensitive data protection capabilities, such as automatically discovering sensitive data, so stay tuned.
ODC prioritizes the protection of sensitive data and provides reliable, full-scenario sensitive data protection for enterprise-level scenarios to ensure user data security and compliance. Through the data privacy protection capability, security administrators can configure sensitive data rules and desensitization algorithms as needed to ensure the absolute security of sensitive data. Even database administrators (DBAs) and developers cannot directly access sensitive data after configuration, thus greatly reducing the risk of data leakage. This strict data privacy protection mechanism enables enterprises to maximize the protection of their sensitive information at all times.
Full-scenario desensitization coverage is another highlight of ODC in data protection. ODC's data desensitization is not only suitable for static scenes, but also supports dynamic scenes. Whether you are performing table data viewing, SQL query, result set export, or data export, ODC can provide efficient data protection. This means that sensitive data can be protected during the entire process of data processing, thus ensuring the security of corporate data in all aspects. Through these capabilities, ODC provides enterprises with full-scenario sensitive data protection, reduces the risk of data leakage, and ensures data security and compliance.
▋ Integration of processes and tools: 0 code to seamlessly connect ODC to your collaborative processes
In terms of the integration of collaborative processes and tools, ODC can help users seamlessly connect it with existing collaborative processes through zero code, helping enterprises achieve more efficient collaborative work. ODC provides a variety of system integration capabilities, including single sign-on (SSO, OAuth2/OIDC), approval integration, SQL audit integration, bastion host integration, audit integration, etc. Through these integration functions, ODC can cover almost all scenarios of enterprise management and control integration, ensuring that your existing processes and strategies can be seamlessly integrated with ODC to achieve high efficiency and consistency in collaborative work.
In addition, ODC also provides flexible and easy-to-use integrated configuration functions, making it easier to integrate ODC into your current database development collaborative workflow. You can easily adapt to existing system APIs through ODC without making any code changes. This means that you can efficiently introduce the advantages and functions of ODC into existing processes without changing the existing system architecture, injecting new impetus into your collaborative work. Through this flexible and easy-to-use integrated configuration, ODC brings greater convenience and efficiency to enterprises' collaborative work.
▋ Full-scenario collaboration: the entire life cycle from database development to data management
With critical needs in database development and data management, ODC solves the pain points that many customers experience when working together. Through its full-scenario collaboration capabilities, ODC helps users achieve seamless connections throughout the entire life cycle from database development to data management, providing enterprises with all-round support and collaborative management. In terms of the full life cycle, ODC covers all scenarios of enterprise database development and data management, from SQL development to change risk management and control, to data life cycle management and data security. This means that enterprises can use ODC to achieve efficient and seamless collaboration in terms of development, change, data protection or data management, ensuring smooth collaboration in all aspects.
At the same time, ODC focuses on collaborative participation of multiple roles. Developer, DBA, data analyst, architect, and data security officer, each role has different concerns and needs. ODC fully understands and integrates the special needs of each role, making collaboration in the database development and data management process smoother. By integrating the needs of different roles into collaborative work, ODC reduces the deployment and use costs of enterprise database collaboration platforms and provides efficient and flexible solutions for collaborative work.
Many outstanding open source projects have emerged in the field of database development. ODC hopes to use open source to work with the community to create an enterprise-level database collaborative development tool that helps Devs and DBAs truly realize "ONE team". In the future, we will continue to polish our existing product capabilities and try out some new directions.
-
More data sources: Gradually introduce MySQL and Oracle data source support to more comprehensively manage data from different sources.
-
More powerful changes: Build stable change modules, including lock-free structure changes and lock-free data changes.
-
Difference architecture management: Achieve comprehensive comparison and synchronization of database structures to ensure consistent processing of object structure changes between libraries and tables.
We are always on the road to helping users solve enterprise-level database management and control collaboration problems. If you are interested in this open source project, you can click "Read the original text" at the end of the article and go to GitHub to learn more details. We also welcome participating in the ODC community in any form. Whether you are contributing code, asking questions, sharing insights, or simply expressing your love and support for this open source project, it will become the driving force for the development of the ODC project.