Commonly used mainstream open source database connection pools include C3P0, DBCP, Tomcat Jdbc Pool, BoneCP, Druid, etc.
C3p0 : An open source JDBC connection pool that implements data source and JNDI binding, and supports JDBC3 specification and JDBC2 standard extension. Currently, open source projects using it include Hibernate, Spring, etc. Single thread, poor performance, suitable for small systems, the code is about 600KB.
DBCP (Database Connection Pool) : A Java database connection pool project developed by Apache, Jakarta commons-pool object pool mechanism, the connection pool component used by Tomcat is DBCP. Using dbcp alone requires 3 packages: common-dbcp.jar, common-pool.jar, common-collections.jar. The database connection is placed in the memory in advance. When the application needs to establish a database connection, it can directly apply for one in the connection pool. , put it back after use. Single thread, low concurrency, poor performance, suitable for small systems.
Tomcat Jdbc Pool : Tomcat used common-dbcp as a connection pool component before 7.0, but dbcp is a single thread. To ensure thread safety, the entire connection pool will be locked, and the performance is poor. There are more than 60 classes in dbcp, which is relatively complex. . Tomcat has introduced a new connection pool module called Tomcat jdbc pool since 7.0. It is based on Tomcat JULI, uses the Tomcat log framework, is fully compatible with dbcp, obtains connections through asynchronous methods, supports high concurrent application environments, and has only 8 super simple core files. Support JMX, supports XA Connection.
BoneCP : Officially, BoneCP is an efficient, free and open source Java database connection pool implementation library. The original intention of the design is to improve the performance of database connection pools. According to some test data, BoneCP is the fastest, about 25 times faster than the second fastest connection pool at that time. It is perfectly integrated into some persistent products such as Hibernate and DataNucleus. middle. BoneCP features: highly extensible and fast; callback mechanism for connection state switching; allowing direct access to connections; automatic reset capability; JMX support; lazy loading capability; support for XML and property file configuration methods; better Java code organization, 100% Unit test branch code coverage; about 40KB of code.
Druid : Druid is the best database connection pool in the Java language. Druid can provide powerful monitoring and expansion functions. It is a highly fault-tolerant, high-performance open source distributed system that can be used for real-time query and analysis of big data. Druid can still maintain 100% normal operation when code deployment, machine failure, and other production systems encounter downtime. Main features: designed for analysis and monitoring; fast interactive query; high availability; extensibility; Druid is an open source project, and the source code is hosted on github.
The functions of the mainstream connection pool are compared as follows :
HikariCP performance analysis
:
HikariCP improves concurrent read and write efficiency by optimizing (concurrentBag, fastStatementList) collections.
HikariCP uses threadlocal cache connection and extensive use of CAS mechanism to avoid locks to the greatest extent. Single may bring about an increase in cpu usage.
Optimize code from the dimension of bytecode. (default inline threshold for a JVM running the server Hotspot compiler is 35 bytecodes ) Let the method use as little as 35 bytecodes to improve the processing efficiency of JVM.
The optimizations made by HikariCP are as follows :
The ping command is used in the source code of mysql connecter
Faster database connection pool than HikariCP
A colleague told me that he has also used and researched the connection pool faster than hikari.
https://github.com/mauricio/postgresql-async
This is something in the scala ecosystem. The MySQL protocol is implemented with netty, and the official MySQL connector is not used. It is purely asynchronous. Its connection pool is written casually, but the performance is still very good.
Looking ahead, will the future be HikariCP or Druid?
Many people are asking, which is the second-generation connection pool HikariCP and Druid standing on the shoulders of giants? In fact, I think this is an issue that need not be discussed.
Let's take a look at the future trend first: the stand-alone operating system will be abandoned and replaced by a cloud operating system with container scheduling and orchestration. Bare metal or virtual machine runtimes will also be replaced by containers. For communication, Service Mesh will be used.
That is to say, the final trend of middleware must be weakened to no perception, which is the final direction from avenue to simplicity. For those maven dependency problems, the second-party library is written in the pom, and the hard-coding of monitoring and other codes into the application will gradually weaken to the point where they no longer exist. Instead, those java agents (such as pinpoint, skywalking, etc.), or service mesh This side car mode can be used to monitor middleware (including connection pools).
A friend who has praised told me that after replacing durid with HikariCP in the core application of Youzan, RT experienced a cliff-like decline (1.5ms ~ 1.2ms) and continued to stabilize with few glitches. After the performance test and stress test, the performance of a core system is about doubled compared with that of Druid.
Afei has done the following statistical work, all based on the latest tag statistics, only the java files and xml files are counted, the total number of lines of druid (alibaba-druid): 430289, the total number of lines of HikariCP (brettwooldridge-HikariCP): 18372.
Only the java code is counted, the total number of rows of druid (alibaba-druid): 428749, and the total number of rows of HikariCP (brettwooldridge-HikariCP): 17556.
Filter the test directory again, (alibaba-druid) total number of lines: 215232, (brettwooldridge-HikariCP) total number of lines: 7960.
A DruidDataSource alone has 3000 lines, not to mention performance, druid is based on jdbc, and its own coding has been enhanced.
If so, Druid is living in the process-oriented era of the first and second generation connection pools to be precise. Druid may have forgotten the concept of loose coupling, and making monitoring and database connection pools in one project is itself tightly coupled. Since microservices advocate business isolation, shouldn't this be isolated? Wouldn't it be nice to just let the component tools do one thing at a time? After all, monitoring matters in the future of service mesh will have other natural monitoring methods instead of being hard-coded in a small connection pool. To sum up, if you put it into the current or future trend, it is highly likely that it cannot compare to the HikariCP that embraces springboot 2.0 and the simplicity of the road to the extreme.
The future middleware must be the same as the spring ecosystem and servich mesh. The road is simple, and it is getting thinner and thinner. Upgrading middleware no longer requires users to forcibly upgrade maven dependencies to resolve dependency conflicts, but to achieve the ultimate upgrade through mesh. Let the business side be unaware. Therefore, those compromise methods such as hot deployment, Pandora boot, and container isolation to resolve dependency conflicts will likely be replaced with a high probability.
Looking at the future from the evolution of Sharding-jdbc architecture
Database Mesh, an emerging vocabulary derived from the wave of Service Mesh. As the name suggests, Database Mesh uses a meshing layer to unify the databases scattered in every corner of the system. The interaction network between the application and the database, gathered together by meshing layers, is as complex and orderly as a spider web. Its primary goal is not to mesh the data stored in the database, but to mesh the interaction between the application and the database.
The focus of Database Mesh is how to organically connect distributed data access applications and databases. It focuses more on interaction, which is to effectively sort out the interaction between disorganized applications and databases.
Using Database Mesh, applications and databases that access the database will eventually form a huge grid system. Applications and databases only need to be seated in the grid system. They are all objects managed by the meshing layer.
Sharding-JDBC has always taken JDBC layer sharding as its core concept. Its architecture diagram is as follows:
Sharding-JDBC will implement three different versions, Driver, Server, and Sidecar, to form the Sharding-JDBC ecosystem and provide more targeted and differentiated services for different needs and environments.
Due to the emergence of Sharding-JDBC-Server, the original DBA's inability to operate data through Sharding-JDBC-Driver has been compensated. Since Sharding-JDBC-Driver does not need to perform secondary forwarding through the proxy layer, the online performance is better. Sharding-JDBC can be used through the following hybrid deployment solutions:
Online applications use Sharding-JDBC-Driver to directly connect to the database to obtain optimal performance Performance, use MySQL command line or UI client to connect to Sharding-JDBC-Server to easily query data and execute various DDL statements. They use the same registry cluster and configure the data in the registry through the management side, so that the registry can automatically push configuration changes to the Driver and Server applications. If the number of connections will skyrocket due to too many database splits, you can consider using Sharding-JDBC-Server directly online to effectively control the number of connections.
In the near future, Sharding-JDBC-Sidecar will also come out, and its deployment architecture is as follows:
这篇文章转载自公众号 工匠小猪猪的技术世界， 原作者的第一段文字是引用自 梁朋举 同学的博客文章，望周知