Database connection pooling is a technology used to optimize database connections. It manages and reuses database connections by establishing a connection pool between applications and databases to improve database access efficiency and performance. A database connection pool usually contains the following parameters:
- Initial number of connections (initialSize): the number of connections initially established by the connection pool;
- Minimum number of connections (minIdle): the minimum number of connections kept in the connection pool;
- Maximum number of connections (maxActive): the maximum number of connections allowed in the connection pool;
- Maximum waiting time (maxWait): The maximum waiting time for obtaining a connection, and an exception will be thrown when the timeout expires;
- Whether the connection pool prepares statements (poolPreparedStatements): whether to enable prepared statements, which can improve the efficiency of database access;
- Prepared statement cache size (maxOpenPreparedStatements): the number of cached prepared statements;
- Whether the connection automatically commits the transaction (defaultAutoCommit): whether the connection automatically commits the transaction, which can avoid transaction management errors;
- Connection validity check (testOnBorrow): When obtaining a connection from the connection pool, whether to check the validity of the connection;
- Connection idle time (minEvictableIdleTimeMillis): The minimum idle time of the connection in the pool, beyond which time it will be cleared;
- Interval for clearing connections (timeBetweenEvictionRunsMillis): Periodic interval for clearing connections.
The principle of the database connection pool is to establish a connection pool between the application program and the database. The application program obtains a connection from the connection pool, and after accessing the database to complete the business operation, the connection is returned to the connection pool. When the application needs to access the database again, the available connection can be obtained from the connection pool, which avoids frequently establishing and closing connections, and improves the efficiency and performance of database access.
The implementation principle of connection pool usually includes the following steps:
- Initialize the connection pool: establish the initial number of connections and save the connections in the connection pool;
- The application obtains the connection: the application obtains the connection from the connection pool;
- Check the validity of the connection: the connection pool checks whether the connection is valid, if it is invalid, close the connection and re-establish the connection;
- Check whether the connection pool is full: the connection pool checks whether the current number of connections has reached the maximum number of connections, if it is full, it will block or throw an exception;
- Return an available connection: If there is an available connection in the connection pool, return the connection to the application;
- Application uses connections: applications use connections to access databases to complete business operations;
- Application Releases Connection: The application releases the connection back to the connection pool so it can be used by other applications.
database tuning
Database tuning is a method to optimize database performance. By adjusting database parameters, database access efficiency and performance can be improved. The following is a detailed introduction to the common tuning parameters of the database:
- cache size parameter
(1) Buffer pool size (innodb_buffer_pool_size): The size of the data and indexes cached by the InnoDB storage engine, generally set to 60%-80% of the physical memory.
(2) Cache index size (key_buffer_size): The size of the MyISAM storage engine cache index data, generally set to 10% of the physical memory.
- Thread pool parameters
(1) Maximum number of connections (max_connections): The maximum number of connections allowed to connect to the MySQL database at the same time, generally set to 500-1000.
(2) Thread pool size (thread_pool_size): The size of the connection thread pool, generally set to twice the number of CPU cores.
(3) Waiting for connection timeout (wait_timeout): If the waiting time for connection exceeds this time, it will be forcibly disconnected.
- Query Optimization Parameters
(1) Query cache (query_cache_size): MySQL can cache the query results, and the next time the same query can be directly obtained from the cache, generally set to 10% of the physical memory.
(2) Maximum number of connections (max_connections): The maximum number of connections allowed to connect to the MySQL database at the same time, generally set to 500-1000.
(3) Connection timeout (connect_timeout): The timeout period for establishing a connection.
(4) Query cache size (query_cache_size): MySQL can cache the query results, and the same query can be obtained directly from the cache next time.
- log parameters
(1) Slow query log (slow_query_log): Record the log of slow query, and you can set the threshold time of query.
(2) Error log (log_error): record the database error log.
(3) Binary log (binlog): records all modification operations on the database, which can be used for database backup and recovery.
- optimizer parameters
(1) Query optimizer (optimizer_switch): MySQL's query optimizer has many parameters that can be adjusted, and can be adjusted according to specific scenarios.
- InnoDB storage engine parameters
(1) Log size (innodb_log_file_size): The log size of the InnoDB storage engine, generally set to 1GB-2GB.
(2) Log cache size (innodb_log_buffer_size): The log cache size of the InnoDB storage engine, generally set to 32MB-64MB.
(3) Maximum number of transactions (innodb_max_dirty_pages_pct): The maximum number of transactions of the InnoDB storage engine, generally set to 70%-80%.
The above is a detailed introduction to the common tuning parameters of the database. According to the actual situation and performance requirements, the values of these parameters can be adjusted appropriately to improve the performance and response speed of the database. It should be noted that the adjustment of different database parameters needs to be adjusted according to the actual scene, and cannot be adjusted blindly. When adjusting database parameters, it is recommended to use performance testing tools to test whether the adjusted performance has improved, so as to adjust parameters in time and optimize database performance.