Prepare transparent transmission under Proxy makes GaussDB (for MySQL) more stable and performs better

From data to large model application, on November 25th, Hangzhou Yuanchuang Conference was held to share development tips.

This article is shared from Huawei Cloud Community " Prepare transparent transmission under Proxy, making GaussDB (for MySQL) more stable and performant ", author: GaussDB database.

1. Introduction

In many business scenarios, database applications process a large number of the same SQL statements - just changing the text or variable values in the SQL statements. For example: use the same SQL template for WHERE query, SET update and VALUES insert operations. After receiving the SQL statement inside the database, it needs to parse the statement, that is, translate it into a machine-executable language. A large number of similar statements must be translated repeatedly. GaussDB (for MySQL) supports the Prepare protocol to reduce the workload of repeated translation. The Prepare protocol utilizes an efficient client/server binary protocol and uses placeholders instead of parameter values in prepared statements, so that each prepared statement only needs to be parsed once, thereby reducing database overhead.

In addition, due to the uneven level and experience of programmers, a considerable number of programmers do not consider the risk of SQL injection when writing code, giving criminals an opportunity to implant malicious SQL to attack the database. SQL injection inserts malicious SQL queries or update statements into the input parameters of the application, and the backend database will be attacked when doing SQL parsing. Since the Prepare protocol has completed SQL pre-compilation before inputting parameters, when malicious SQL is inserted as a parameter, the database will not be recompiled, thus preventing SQL injection attacks and providing higher security.

In summary, the Prepare protocol can bring double benefits in terms of database performance and stability.

2.Prepare execution process

The Prepare protocol is divided into two phases. The first phase is to submit the precompiled SQL with placeholders. The second phase is to transmit the data and replace the placeholders with data for execution.

Prepare protocol application code example:

3. Prepare protocol in database proxy (Proxy) scenario

In the article " Highly available read-only, making RDS for MySQL more stable ", we mentioned the solution of using database proxy (Proxy) to achieve read-write separation and high reliability. GaussDB (for MySQL) also supports multiple Proxy to achieve high reliability and high performance solutions such as read-write separation, automatic load balancing, and automatic routing to other instances when a read-only instance fails. ——GaussDB (for MySQL) also supports multiple Proxy to achieve read-write separation, automatic load balancing and automatic routing to other instances when a read-only instance fails, with high reliability and high performance capabilities.

The database proxy is a network proxy service located between the database server and the application server. It receives connections from applications and automatically distributes SQL requests to GaussDB (for MySQL) nodes for execution based on routing conditions. The database proxy (hereinafter referred to as proxy) has the characteristics of high availability, high performance, operation and maintenance, and simplicity and ease of use. It provides functions such as automatic read and write separation, connection pooling, transaction splitting, and session consistency. The detailed architecture diagram is as follows:

3.1 Implementation plan of Prepare protocol under Proxy

When Proxy is not used, the two phases of the Prepare protocol (precompilation phase and execution phase) are executed directly on the database node. After Proxy is added, since the database proxy will distribute different SQL on the same connection to multiple back-end databases, the database proxy needs to save the correspondence between the precompiled SQL and execution parameters, and pass it to the database node during execution. execute on. Currently, there are two implementation solutions for the Prepare protocol in the Proxy scenario:

Text mode (the earlier mode)
Transparent transmission mode (currently popular mode)

3.1.1 Text mode:

The Proxy internally saves the relevant information of the precompiled SQL, converts the binary Prepare protocol sent by the client into a normal text protocol for transmission, that is, completes the parsing and replacement on the Proxy side, and sends the parsed and replaced SQL to the back-end database without any dependencies. The database performs parsing and conversion.

Advantage:

The implementation is simple, any SQL is unconditionally encapsulated into text SQL and executed directly.
Can be routed arbitrarily.
When the connection between the database agent and the back-end database is disconnected, the client has no awareness and is automatically compatible with disaster recovery scenarios.

Disadvantages :

It is necessary to parse the type structure and assemble the original SQL, which is strongly dependent on the type sent. For example: execute SELECT * FROM t1 LIMIT ?, when sending String type data, it will be spliced into a quoted String value: SELECT * FROM t1 LIMIT '10', the database does not support this SQL syntax, resulting in an execution error.
Since each piece of SQL needs to be spliced on the database proxy side, and during splicing, the proxy side needs to handle SQL injection issues based on the data type. The proxy side has serious performance losses and can easily become a bottleneck.

3.1.2 Transparent transmission mode

In transparent transmission mode, the Proxy side only does the precompilation phase, and the execution phase sends the precompiled statements and parameter data directly to the back-end database. That is, the Proxy side does not perform parsing and compilation. There are two ways to implement transparent transmission mode:

Broadcast transparent transmission (the mode used by most database manufacturers)

Broadcast transparent transmission is to send the precompiled statement sent by the client to all back-end database nodes, that is, all back-end database nodes can parse, compile and execute the statement after receiving the parameters. Therefore, when sending parameters, you only need to arbitrarily select a node that has sent the precompiled statement for transmission.

Advantage:

Proxy does not need to assemble and parse SQL.
Proxy does not need to maintain the mapping relationship between precompiled statements and back-end database nodes. It only needs to be responsible for broadcasting and forwarding, which is simple to implement.

Disadvantages :

All database nodes are under equal pressure. The same precompiled statement is executed by multiple database nodes, which wastes resources. When a certain node is under high pressure, it will also bear the pressure of compiled statements.
When multiple nodes recover after an exception, the Proxy needs to re-execute precompiled statements on all recovered database nodes to ensure the correspondence between parameters and precompiled statements. This will cause excessive pressure on the proxy.

Single node transparent transmission (mode used by GaussDB (for MySQL) agent)

Single-node transparent transmission means that the database agent sends the precompiled statement to one of the nodes in the database according to the route, and maintains the mapping relationship between the precompiled statement and the database node. When the parameter data is executed, the parameters are sent to the specified node for execution according to the mapping relationship. .

Advantage:

Proxy does not need to assemble and parse SQL.
It will not cause waste of back-end database resources and excessive overall pressure.

Disadvantages :

The precompiled statements stored on the client are transparently transmitted to a certain database node at a single point. During the execution phase, the parameters input by the client need to be routed to the corresponding node. Proxy needs to handle this relationship, and the function implementation is complex. It is coded by the GaussDB (for MySQL) team . bring greater challenges .

4.Performance comparison

We conducted a performance test and comparison of the Prepare protocol of GaussDB (for MySQL) in text mode and transparent transmission mode in the Proxy scenario. Tests show that the performance in transparent transmission mode is better than text mode, with a performance improvement of about 28%.

The hardware and software specifications for performing the tests are shown in the table below:

Due to current conditions, although we did not test the performance comparison between the broadcast transparent transmission mode and the single-point transparent transmission mode, it is not difficult to predict from the previous solution comparison that the single-point transparent transmission mode has higher stability because of its It is less likely to consume resources on the Proxy side and the database node side, causing bottlenecks.

5. Summary

The Prepare protocol under GaussDB (for MySQL) Proxy optimizes the original text mode processing method into a transparent transmission mode, and its performance is greatly improved, solving the problem of strong dependence on data types in text mode. At present, the industry mostly uses broadcast mode for transparent transmission, while the GaussDB (for MySQL) Proxy side optimizes the broadcast mode into a single-point transmission mode, reducing the overall performance loss of the database.

Click to follow and learn about Huawei Cloud’s new technologies as soon as possible~