- Mainstream database connection pools
The popular database connection pools at the current technological frontier are: DBCP, Tomcat Jdbc Pool, BoneCP, Druid, C3P0, etc. - DBCP : A Java database connection pool project developed by Apache, Jakarta commons-pool object pool mechanism, the connection pool component used by Tomcat is DBCP. Using dbcp alone requires 3 packages: common-dbcp.jar, common-pool.jar, common-collections.jar. The database connection is placed in the memory in advance. When the application needs to establish a database connection, it can directly apply for one in the connection pool. , put it back after use. Single thread, low concurrency, poor performance, suitable for small systems.
- Tomcat Jdbc Pool : Tomcat used common-dbcp as a connection pool component before 7.0, but dbcp is a single thread. To ensure thread safety, the entire connection pool will be locked, and the performance is poor. There are more than 60 classes in dbcp, which is relatively complex. . Tomcat has introduced a new connection pool module called Tomcat jdbc pool since 7.0. It is based on Tomcat JULI, uses the Tomcat log framework, is fully compatible with dbcp, obtains connections through asynchronous methods, supports high concurrent application environments, and has only 8 super simple core files. Support JMX, supports XA Connection.
- BoneCP : Officially, BoneCP is an efficient, free and open source Java database connection pool implementation library. The original intention of the design is to improve the performance of database connection pools. According to some test data, BoneCP is the fastest, about 25 times faster than the second fastest connection pool at that time. It is perfectly integrated into some persistent products such as Hibernate and DataNucleus. middle. BoneCP features: highly extensible and fast; callback mechanism for connection state switching; allowing direct access to connections; automatic reset capability; JMX support; lazy loading capability; support for XML and property file configuration methods; better Java code organization, 100% Unit test branch code coverage; about 40KB of code.
- Druid : Druid is the best database connection pool in the Java language. Druid can provide powerful monitoring and expansion functions. It is a highly fault-tolerant, high-performance open source distributed system that can be used for real-time query and analysis of big data. Druid can still maintain 100% normal operation when code deployment, machine failure, and other production systems encounter downtime. Main features: designed for analysis and monitoring; fast interactive query; high availability; extensibility; Druid is an open source project, and the source code is hosted on github.
- C3p0 : An open source JDBC connection pool, which implements data source and JNDI binding, and supports JDBC3 specification and JDBC2 standard extension. Currently, open source projects using it include Hibernate, Spring, etc. Single thread, poor performance, suitable for small systems, the code is about 600KB.
- comparison between them
- Look at a group of HikariCP :
- HikariCP performance analysis
1. HikariCP improves concurrent read and write efficiency by optimizing (concurrentBag, fastStatementList) collections.
2. HikariCP uses the mechanism of threadlocal cache connection and extensive use of CAS to avoid locks to the greatest extent. Single may bring about an increase in cpu usage.
3. Optimize the code from the dimension of bytecode. (default inline threshold for a JVM running the server Hotspot compiler is 35 bytecodes ) Let the method use as little as 35 bytecodes to improve the processing efficiency of JVM. - Database connection pool faster than HikariCP :
A colleague told me that he also used and studied the connection pool faster than Hikari https://github.com/mauricio/postgresql-async This is something of the scala ecosystem. The MySQL protocol is implemented with netty, and the official MySQL connector is not used. It is purely asynchronous. Its connection pool is written casually, but the performance is still very good. - Looking at the future from the evolution of Sharding-jdbc architecture :
Database Mesh, an emerging vocabulary derived from the wave of Service Mesh. As the name suggests, Database Mesh uses a meshing layer to unify the databases scattered in every corner of the system. The interaction network between the application and the database, gathered together by meshing layers, is as complex and orderly as a spider web. Its primary goal is not to mesh the data stored in the database, but to mesh the interaction between the application and the database.
The focus of Database Mesh is how to organically connect distributed data access applications and databases. It focuses more on interaction, which is to effectively sort out the interaction between disorganized applications and databases.
Using Database Mesh, applications and databases that access the database will eventually form a huge grid system. Applications and databases only need to be seated in the grid system. They are all objects managed by the meshing layer.
Sharding-JDBC has always taken JDBC layer sharding as its core concept. Its architecture diagram is as follows:
Sharding-JDBC will implement three different versions of Driver, Server and Sidecar, together to form the Sharding-JDBC ecosystem, providing more targeted and differentiated services for different needs and environments.
Due to the emergence of Sharding-JDBC-Server, the original DBA's inability to operate data through Sharding-JDBC-Driver has been compensated. Since Sharding-JDBC-Driver does not require secondary forwarding through the proxy layer, it has better online performance. Sharding-JDBC can be used through the following hybrid deployment solutions:
Online applications use Sharding-JDBC-Driver to directly connect to the database for optimal performance, and use MySQL command line or UI client to connect to Sharding-JDBC-Server to easily query data and execute various DDL statements. They use the same registry cluster and configure the data in the registry through the management side, so that the registry can automatically push configuration changes to the Driver and Server applications. If the number of connections will skyrocket due to too many database splits, you can consider using Sharding-JDBC-Server directly online to effectively control the number of connections.
In the near future, Sharding-JDBC-Sidecar will also come out. Its deployment architecture is as follows:
Database Mesh and Service Mesh based on Sharding-JDBC do not interfere with each other and complement each other. Interaction between services is taken over by Service Mesh Sidecar, and SQL-based database access is taken over by Sharding-JDBC-Sidecar.
For business applications, whether it is RPC or access to the database, there is no need to pay attention to its real physical deployment structure, so as to achieve real zero intrusion. Since Sharding-JDBC-Sidecar is created and dies with the life cycle of the host,
Therefore, it is not a static IP, but a completely dynamic and elastic existence, and there is no central node in the entire system. For operations such as data operation and maintenance, you can still start a Sharding-JDBC-Server process as the entry of static IP, and operate through various command lines or UI clients.