The thing that has always caused headaches for DBAs or the team responsible for operating and maintaining the database is how to manage database access reasonably . On the one hand, the database operation and maintenance team hopes to centrally control database access, but on the other hand, the development team using the database hopes to be able to access the database anytime, anywhere.
The extreme of the former is that all access to the database needs to be approved by the database team, which will cause the database team to become a bottleneck, dragging down the progress of product delivery, and being constantly complained by the R&D team.
If the latter goes to the extreme, the access rights of the database are scattered everywhere, and data leaks; there is no standardized change process, which leads to vicious online failures such as #delete database and run away.
Is there an ideal solution that can take into account both control and efficiency ?
Let us first disassemble the database access scenario, and then provide solutions.
Database access scenario
There are two types of database access scenarios: one is program access (Machine to Database), and the other is human access (Human to Database).
The program access is divided into :
- In response to the request (end user initiated) sent by the user from the web/mobile end, the addition, deletion, modification, and query operations are performed on the database through the application server (Application Server). order record.
- Database changes when software is deployed (change on deploy). Sometimes after the software version is upgraded, it is also necessary to change the dependent database schema or data. The practice of many software is to make changes before the new version runs for the first time. Many language application development frameworks and ORMs also have built-in related capabilities, such as Active Record Migrations provided by Ruby on Rails and Migrations provided by django .
- The background timing service accesses the database (cron). For example, the financial reconciliation system performs automatic reconciliation operations on a regular basis, and finally generates reconciliation records in the database. For example, the downstream data warehouse synchronizes data from the database.
- One-time execution (one-shot / on-demand). For example, data cleaning/migration/correction involving complex business logic. These actions are packaged into individual scripts or command-line programs (CLI) that are executed only when needed.
Manual access can be divided into:
- Interactive data query (interactive query). Connect directly to the database and use SQL language for data query.
- Data correction (ad-hoc data correction). Connect directly to the database and use SQL language for data correction.
- Database schema change (offline schema change). Users sometimes directly connect to the database to make schema changes. The so-called deletion of the database and running away refers to the user mistakenly executing the DROP TABLE / Database schema change operation, resulting in serious or even irreparable losses.
- Database operation and maintenance management (admin operations). The DBA connects to the database and performs administrator operations, such as terminating SQL statements that take too long to execute.
It is a high-risk behavior to directly connect to the database manually for operation , so usually the organization will set up a jump server (Bastion / Jump server). On the jump server, database operations can be controlled and audited to a certain extent. However, setting up the springboard also brings more inconvenience to users and introduces additional management burdens. For example, it is necessary to consider whether to set up a separate springboard machine for database access, or to share the springboard machine for accessing other production systems.
Solutions
First, we need to clarify a few points.
- Accessing databases, especially online production databases, is a high-risk operation. Even if it's not a change, a single inadvertent SELECT statement can bring down an entire database, causing catastrophic failure. Therefore, whenever possible, control is necessary.
- Minimize or even avoid the operation of directly connecting to the database manually. Humans make mistakes, and manually connecting to the database also means that the access information of the database needs to be given to individuals, which greatly increases the hidden danger of data leakage.
- It is unavoidable for the application to access the database, otherwise the program cannot run normally. However, the schema change of the database can be separated separately, and there is no need to change it at the first run after the new version is deployed. Moreover, decoupling schema changes and application deployment can better control the entire change process and deal with possible rollbacks.
Based on the above points, we give the ideal division of database access scenarios:
Here we introduce Bytebase, which can not only cover all scenarios of manual access to the database, but also unify schema changes into a set of processes with control capabilities.
- Bytebase defines a two-tier authority model, one is the global authority at the entire workspace level, and the other is the specific business development project authority at the project level. The authority system of these two layers corresponds to the central DBA team and the development team of each application project group respectively.
- Bytebase provides a visual interface to manage database schema and data change review process. At the same time, it also supports integration with GitLab and GitHub, and provides the GitOps process of Database-as-Code.
- Bytebase provides SQL Editor, developers can query SELECT operations. For DBAs, you can enable the administrator mode in SQL Editor to execute database management commands.
- Users can configure SQL Review audit rules. As shown in the figure below, in the data change process, the rules prevent users from executing DELETE statements without WHERE. And for different environments such as dev, test, and prod in the R&D process, users can also configure different rules.
- You can also configure different audit policies, such as dev environment, DBA audit can be skipped; prod environment, DBA audit is required or project owner audit can also be required.
Summarize
The operation of the database needs to be cautious, but while maintaining the necessary caution, the R&D team also hopes to improve efficiency. As the only Database CI/CD solution included in CNCF Landscape, Bytebase provides enterprise-level one-stop database development lifecycle management capabilities.
For DBAs or teams responsible for database operation and maintenance, Bytebase can close all database access paths except application access, and perform global database control and operation auditing on top of it. For application developers who need to use databases, Bytebase provides a developer-friendly interface to assist them in completing database development work efficiently and safely.
Guarantee safe production and take into account development efficiency.