Author: Wang Zhibin, won China PostgreSQL database management engineer (PGCE), PostgreSQL is officially certified instructor, Pangu cloud classroom guest lecturer gold medal.
PgBouncer serves as the connection pool middleware for PostgreSQL database. Unlike other connection pool middlewares that exist in PostgreSQL, PgBouncer only serves as a connection pool and proxy layer to provide services between PostgreSQL and applications.
Pgbouncer has many important features such as connection pool mode, connection type, port reuse, application scenarios, user authentication, network authentication, etc. The following will describe them one by one, and hope to provide readers with a configuration guide used in the implementation process.
Database connection pool includes session connection pool, transaction connection pool, and statement connection pool in Pgbouncer.
1. Session connection pool The
official explanation is the most polite method. When the client connects, the server connection will be allocated to it during the entire process of keeping it connected. When the client disconnects, the server connection will be put back into the pool. This mode supports all PostgeSQL functions.
2. Transaction connection pool
Server connections are only allocated to clients during the transaction. When PgBouncer finds that the transaction has ended, the server connection will be put back into the pool. This mode destroys some of the session-based functions of PostgreSQL. It can be used only when the application uses the non-disruptive function through collaboration. About incompatible features.
3. Sentence connection pool The
official interpretation is the most radical method. Multi-statement transactions are not allowed. Essentially in order to enforce the "auto-commit" mode on the client, it is mainly aimed at PL/Proxy.
Other features supported include:
- High performance, because Pgbouncer itself does not need to view the entire data packet, so the network overhead is only 2k (default), and the memory requirement of the system is small.
- Flexible deployment: Pgbouncer is not bound to a back-end server. The target database can reside on a different host.
- Strong maintainability: support online reconfiguration of most configuration items; and support online restart/upgrade without disconnecting the client.
- Flexible authentication: User authentication supports file-based authentication, as well as database query authentication; network connection authentication is consistent with the Postgresql database and supports multiple modes of authentication.
- Flexible connection number: It supports the combination of global, database, user and client connection number settings.
(Note: For parts that are not described in detail in the text, please refer to the relevant documents on the official website of Pgbouncer[1], such as the configuration manual, user manual, FAQ and other official documents).
Some features of Pgbouncer are briefly introduced above. For detailed features, please refer to (Pgbouncer official website). The following will explain some configuration precautions when using Pgbouncer, and provide a guide for users of Pgbouncer to make full use of Pgbouncer when meeting complex business needs Features to achieve specific business scenario requirements.
In the process of configuring Pgbouncer, you need to pay special attention to the connection pool mode , as well as the number of data connections , connection methods , and finally the Pgbouncer deployment form for different business scenarios .
First discuss why the connection pool [2] is used, the performance difference between using and not using it, and discuss the workflow, details and some precautions of the connection pool mode, and finally provide a suitable connection pool recommendation.
When we are getting started with Postgresql, we usually see this introduction "PostgreSQL server can handle multiple concurrent connections from clients. To this end, it starts ("fork") a new process for each connection. From then on, The client communicates with the new server process without the intervention of the original postgres process. Therefore, the main server process is always running, waiting for the client to connect, and the client and the associated server process come and go." But that means Each new connection forks a new process, remains in memory, and may become too busy in multiple sessions. In the case of small business volume, this method can basically meet the requirements, but when the business volume rapidly increases, we may need to constantly change max_connections to meet the needs of the client. At that time, it also brought great problems, such as the memory overhead caused by frequent closing and connection creation, and the management of a large number of connections that have been generated, which eventually led to the slow response of the server and the inability to provide database services to the outside world. In this context, database connection pools have been proposed. For the use of Postgresql databases, they are generally divided into client connection pools, such as c3p0, druid, etc.; the other is server-side connection pools, such as pgbouncer, odyssey, pgpoolII, etc.
Figure 1 Directly connected to the database server
This is the life cycle of a PostgreSQL connection without a connection pool:
- The client starts a new session by requesting and verifying the connection with the server.
- The server forks a new system process to handle connections and work sessions. The session state is initialized through a combination of server-level, database-level, and user-level configuration parameters.
- The customer completes the required work by performing one or more transactions. Examples include:
- Perform read and write for relationships (tables, views, etc.)
- Use the SET command to change the session or transaction state
- Prepare and execute prepared statements
- When the client disconnects, the session ends.
- The server destroys the session process.
A database session includes all the work done through the life cycle of a single connection. The length of the database session is variable, and the amount of resources consumed on the client and server is variable.
The key points are:
- The process of creating, managing and destroying connections takes time and consumes resources.
- As the number of connections to the server increases, the resources required to manage these connections also increase. In addition, as the client performs processing on the server, the memory usage of each process of the server will continue to grow.
- Since a single session only serves a single client, the client can change the state of the database session and hope that these changes will continue to exist in subsequent transactions.
A connection pool is located between the client and the server. The client connects to the pool manager, and the pool manager connects to the server. The introduction of the connection pool program will change the connection model to the client proxy server architecture:
Figure 2 uses the connection pool to connect to the database
This decouples the client connection lifetime from the server connection and process lifetime. The role of the connection pool:
- Accept and manage connections from clients
- Establish and maintain a connection with the server
- Assign server connections to client connections
Features:
- A single server connection can handle sessions, transactions and statements from different clients
- The transactions and/or statements of a single client session can be run on different server connections
Obviously, the use of connection pool can reduce the memory overhead of the server, and effectively reuse database connections, providing good database connection performance management.
For more PostgreSQL hotspot information, news trends, and exciting activities, please visit the official website of PostgreSQL in China
To solve more PostgreSQL-related knowledge, technology, and work issues, please visit the official PostgreSQL Q&A community in China
To download more PostgreSQL related information, tools, and plug-in issues, please visit the official PostgreSQL download site in China