Notes for personal use, unfinished.
1. Principle
1) Why do you need DB connection pool
https://github.com/onlyliuxin/coding2017/issues/451 (compare this explanation bar)
https://vladmihalcea.com/the-anatomy-of-connection-pooling/
DB connection overhead once more expensive
- create / destroy a TCP connection of I / O overhead
- DB connection requires server is connected to the distribution buffer cache frequently connected / closed causes the I / O overhead
- JVM GC pressure
2) Pool Technology
(Start here without thinking of typing
Optimizing operating efficiency for a large amount of request system, reducing system overhead frequently established connection. Thread pool, object pooling, database connection pools are applied similar techniques.
Usually provide the following configuration
- Minimum / maximum number of connections
- blocking queues, etc.
In addition there will be live explore mechanisms, mandatory recycling, monitoring a class supporting function to ensure database security.
2. DB connection pool
1) JAVA implemented connection pool
Here is a complete life cycle of connection:
1) a data layer requests DataSource, be connected
2) DataSource open connection request Diver
3) create a connection, and opens a TCP socket connection
4) using the connection
5) closes the connection
6) Close socket
Through connection pooling, JAVA connection pool can significantly increase the connection time (from this article..600 times ...), why?
Because it after 2) above step, requesting the connection pool, the pool if and only if there is no available connection / not connected to a maximum limit creates a new connection. And close () method will return connector connected to the pool, to wait for its next call.
2) Golang database / sql package implements
https://mp.weixin.qq.com/s/vSZpF55u4O343B4e0_TOvw
Reform package to achieve the internal connection pool management, only the lower driver exposed simple driver interface, which means that drivers of different databases only based on a single interface calls, without the need to care implement connection pool.
DETAILED hierarchy as follows: application -> API -> [including connection pooling] -> driver API -> go-sql-driver -> MYSQL
Overall, Golang JAVA may correspond with the interface, Driver, Conn, Stmt, Tx, Rows
Call relationship:
1) Open to make new Conn
2) Conn acquisition Prepare method
3) get into SQL Stmt,
4) call Exec, result set Result / call Query, to give Rows
Connection Pool Design
1) to establish a connection
Connected in the implementation of SQL established, there are two strategies to establish connections
- cachedOrNewConn: From freeConn idle connection queue to get a connection, if the acquired connection has not expired can be used normally, if not New Connection. If the maximum number of connections, then block the request.
- alwaysNewConn: Forever New Connection
To establish a connection to connect calls from Connect interface library in Driver
2) release the connection
After use, the connection needs to return the thread pool, usually determines the reliability of the connection, if there occurs an abnormal close the connection, the new connection needs to be replaced. Inspection methods: check ErrBadConn from err msg, if not nil, send a signal to new a connection stead.
3) clean-up connection
maxIdle, maxOpen, macLifeTime. MYSQL forced to scavenge connection than 8h is Idle, maxlifetime set a maximum time multiplexed connection is established to the recovered time is connected (non-idle time!), So as to ensure ligation activity. Each second asynchronous connection idle check freeConn thereof, whether or not combined standard, sub subject connected added Closing arr, then asynchronously Close.
So,
Can know from this article, the connection pool time Golang sql package is relatively simple (possibly article written in relatively clear and simple ..), there is a free connection pool (array) in the DB struct, the blocked request (map [ ] channel), the number of connections / maximum number of connections, maximum life cycle, as well as channel clean-up connection. I did not read the source code (TODO) preliminary judgment is based on the channel to achieve a simple database connection pool.
In the implementation of sql, trying to get a connection from the connection pool kind of idle, check whether the connection failed when acquired, if the failure will continue to get the next one, if you can not get a new connection (question:? What failure means that if will establish a new connection? ), and the current connection pool is full, the connection will enter the blocked request types (Question:? wake-up mechanism, is to start channel in obtaining in return a new available connections, or execution request from the inside), if not full, then, Happy ending: create a new connection.
Get connected after the implementation, connection recovery mechanism, an error msg Golang returned checks, check that the connection is available, if not available to establish a new connection alternatives, then put into the free connection pool.
At the same time clean-up connection mechanism Golang also asynchronous calls, check all connections per second, if the connection time exceeds the maximum life cycle, it is cleared away.