Analysis of the principle of golang database connection pool database/sql realization

Golang’s request to the database abstracts out a set of universal connection pools. With the go mechanism, golang only needs to provide a driver interface, and the underlying different database protocols are driven by users according to their own database. can.

This article explores the details here and the pits that need to be avoided from the perspective of source code implementation. Based on the 1.14 code analysis, some bugs have been fixed or optimized in 1.15, which will also be mentioned here.

golang version: 1.14

Directory structure description

└── sql
    ├── convert.go           # 结果行的读取与转换
    ├── convert_test.go
    ├── ctxutil.go           # 绑定上下文的一些通用方法
    ├── doc.txt
    ├── driver               # driver 定义来实现数据库驱动所需要的接口
    │   ├── driver.go
    │   ├── types.go         # 数据类型别名和转换
    │   └── types_test.go
    ├── example_cli_test.go
    ├── example_service_test.go
    ├── example_test.go
    ├── fakedb_test.go
    ├── sql.go               # 通用的接口和类型，包括事物，连接等
    └── sql_test.go

Main data structure

1. sql.DB

type DB struct {
    // Atomic access only. At top of struct to prevent mis-alignment
    // on 32-bit platforms. Of type time.Duration.
    waitDuration int64          // 等待新的连接所需要的总时间
    connector driver.Connector  // 数据库驱动自己实现
    // numClosed is an atomic counter which represents a total number of
    // closed connections. Stmt.openStmt checks it before cleaning closed
    // connections in Stmt.css.
    numClosed uint64           // 关闭的连接数

    mu           sync.Mutex // protects following fields
    freeConn     []*driverConn
    connRequests map[uint64]chan connRequest
    nextRequest  uint64 // Next key to use in connRequests.
    numOpen      int    // number of opened and pending open connections
    // Used to signal the need for new connections
    // a goroutine running connectionOpener() reads on this chan and
    // maybeOpenNewConnections sends on the chan (one send per needed connection)
    // It is closed during db.Close(). The close tells the connectionOpener
    // goroutine to exit.
    openerCh          chan struct{}      // 用于通知需要创建新的连接
    // resetterCh        chan *driverConn  // 已废弃
    closed            bool
    dep               map[finalCloser]depSet // map[一级对象]map[二级对象]bool，一个外部以来，用于自动关闭
    lastPut           map[*driverConn]string // stacktrace of last conn's put; debug only
    maxIdle           int                    // zero means defaultMaxIdleConns(2); negative means 0
    maxOpen           int                    // <= 0 means unlimited
    maxLifetime       time.Duration          // maximum amount of time a connection may be reused
    cleanerCh         chan struct{}          // 用于通知清理过期的连接，maxlife时间改变或者连接被关闭时会通过该channel通知
    waitCount         int64 // Total number of connections waited for.   // 这些状态数据，可以通过db.Stat() 获取
    maxIdleClosed     int64 // Total number of connections closed due to idle.
    maxLifetimeClosed int64 // Total number of connections closed due to max free limit.

    stop func() // stop cancels the connection opener and the session resetter.
}

sql.DB is not a connection, it is an abstract interface of the database, and also a handle to the entire connection pool. It is safe for multiple goroutines. It can open and close database connections according to the driver, and manage the connection pool. This is the same for different databases.

2. sql.driverConn

// driverConn wraps a driver.Conn with a mutex, to
// be held during all calls into the Conn. (including any calls onto
// interfaces returned via that Conn, such as calls on Tx, Stmt,
// Result, Rows)
type driverConn struct {
   db        *DB
   createdAt time.Time

   sync.Mutex  // guards following
   ci          driver.Conn  // 由不同的驱动自己实现，对应一条具体的数据库连接
   needReset   bool         // The connection session should be reset before use if true.
   closed      bool         // 当前连接的状态，是否已经关闭
   finalClosed bool         // ci.Close has been called
   openStmt    map[*driverStmt]bool

   // guarded by db.mu
   inUse      bool
   onPut      []func() // code (with db.mu held) run when conn is next returned  // 归还连接的时候调用
   dbmuClosed bool     // same as closed, but guarded by db.mu, for removeClosedStmtLocked
}

Encapsulation of a single connection, including the actual database connection and related status information, etc.

3. driver.Conn

// Conn is a connection to a database. It is not used concurrently
// by multiple goroutines.
//
// Conn is assumed to be stateful.
type Conn interface {
   // Prepare returns a prepared statement, bound to this connection.
   Prepare(query string) (Stmt, error)

   // Close invalidates and potentially stops any current
   // prepared statements and transactions, marking this
   // connection as no longer in use.
   //
   // Because the sql package maintains a free pool of
   // connections and only calls Close when there's a surplus of
   // idle connections, it shouldn't be necessary for drivers to
   // do their own connection caching.
   Close() error

   // Begin starts and returns a new transaction.
   //
   // Deprecated: Drivers should implement ConnBeginTx instead (or additionally).
   Begin() (Tx, error)
}

A specific database connection needs to be implemented by different drivers themselves

4. driver.Driver

type Driver interface {
    Open(name string) (Conn, error)
}

Driver contains only one function. Open() is used to return an available connection, which may be a newly established connection or a previously cached closed connection.

5. driver.DriverContext

type DriverContext interface {
// OpenConnector must parse the name in the same format that Driver.Open
// parses the name parameter.
    OpenConnector(name string) (Connector, error)
}

The purpose of DriverContext is to maintain the drievr context information, avoiding the need to parse the dsn every time a new connection is created. The Driver object needs to be implemented by itself.

6. driver.Connector

type Connector interface {
// Connect returns a connection to the database.
// Connect may return a cached connection (one previously
// closed), but doing so is unnecessary; the sql package
// maintains a pool of idle connections for efficient re-use.
//
// The provided context.Context is for dialing purposes only
// (see net.DialContext) and should not be stored or used for
// other purposes.
//
// The returned connection is only used by one goroutine at a
// time.
    Connect(context.Context) (Conn, error)
// Driver returns the underlying Driver of the Connector,
// mainly to maintain compatibility with the Driver method
// on sql.DB.
    Driver() Driver
}

driver.Connector is the socket of the driver, an object of interface type, which is implemented by different types of databases.
driver.Connector contains two functions.

• Connect is used to establish a connection
• Driver is used to return a Driver object. Driver is also an interface type object and needs to be implemented by different databases.

Main operation process

1. Register the driver

import (
    _ "github.com/go-sql-driver/mysql"
)

var (
    driversMu sync.RWMutex
    drivers   = make(map[string]driver.Driver)
)
func Register(name string, driver driver.Driver) {
    driversMu.Lock()
    defer driversMu.Unlock()
    if driver == nil {
        panic("sql: Register driver is nil")
    }
    if _, dup := drivers[name]; dup {
        panic("sql: Register called twice for driver " + name)
    }
    drivers[name] = driver
}

/database/sql provides a general database connection pool. When we connect to different databases, we only need to register the corresponding database driver to use it.

The registration here is actually to add the database name and the corresponding database driver (database connection wrapper) to a map. Each imported library needs to be implemented by calling the registration function in the init function.

2. Create a connection pool handle sql.Open()

func Open(driverName, dataSourceName string) (*DB, error) {
    driversMu.RLock()
    driveri, ok := drivers[driverName]  // 1
    driversMu.RUnlock()
    if !ok {
        return nil, fmt.Errorf("sql: unknown driver %q (forgotten import?)", driverName)
    }
    if driverCtx, ok := driveri.(driver.DriverContext); ok {  // 2
        connector, err := driverCtx.OpenConnector(dataSourceName)
        if err != nil {
            return nil, err
        }
        return OpenDB(connector), nil  // 3
    }
    return OpenDB(dsnConnector{dsn: dataSourceName, driver: driveri}), nil  // 4
}

func OpenDB(c driver.Connector) *DB {
   ctx, cancel := context.WithCancel(context.Background())
   db := &DB{
      connector:    c,
      openerCh:     make(chan struct{}, connectionRequestQueueSize),
      lastPut:      make(map[*driverConn]string),
      connRequests: make(map[uint64]chan connRequest),
      stop:         cancel,
   }

   go db.connectionOpener(ctx)  // 通过channel通知来创建连接
   // go db.connectionResetter(ctx) // 用于重置连接，1.14废弃
   return db
}

The Open function is usually interpreted as initializing the db. Here, the corresponding driver is obtained through the driver name, and a series of initialization operations are performed on the driver. It should be noted that Open does not establish a connection with the db, but is only operating these data structures To start actions such as background coroutines.

The dataSourceName here is referred to as dsn, which contains the necessary parameters for connecting to the database, user name, password, ip port and other information. Different drivers implement the analysis by themselves. Of course, some drivers also support configuring some database parameters in dsn, such as autocommit. Because the information obtained by parsing the string will consume a certain amount of resources, it also provides the function of caching the parsed result, avoiding the need to parse it every time a new connection is established. To do this, it needs to be driven. driver.DriverContext interface.

At this time you have such a structure, but there is no connection in the connection pool at this time, which means that there is no real access to db

3. Set database connection parameters

The maximum number of idle connections. If the number of idle connections exceeds this value, they will be closed. The default is defaultMaxIdleConns(2)

func (db *DB) SetMaxIdleConns(n int) {} 

The maximum number of open connections allowed. After this number is exceeded, no new connections are allowed to be established, and the work coroutine can only block waiting for the release of the connection

func (db *DB) SetMaxOpenConns(n int) {}

The maximum time that a connection can be reused, in other words, how long will a connection be closed after it is closed, but it will be closed after the current request is completed, be closed lazily, a very tasteless parameter

func (db *DB) SetConnMaxLifetime(d time.Duration) {
    // 通过启动一个单独的协程 connectionCleaner 来实现 
    startCleanerLocked {
        go db.connectionCleaner(db.shortestIdleTimeLocked())
    }
}

After 1.15, a new parameter is added, the maximum idle time of the connection, the idle time will be closed if the idle time exceeds this value, but it will be closed after the current request is completed, be closed lazily

func (db *DB) SetConnMaxIdleTime(d time.Duration) {
    // 1.15 实现了对空闲连接的超时回收，复用了SetConnMaxLifetime的部分逻辑，也是在connectionCleaner协程中实现的
}

SetConnMaxLifetime and SetConnMaxIdleTime detailed implementation

• 1.14 Implementation

func (db *DB) startCleanerLocked() {
   if db.maxLifetime > 0 && db.numOpen > 0 && db.cleanerCh == nil {
      db.cleanerCh = make(chan struct{}, 1)
      go db.connectionCleaner(db.maxLifetime)
   }
}

func (db *DB) connectionCleaner(d time.Duration) {
   const minInterval = time.Second

   if d < minInterval {
      d = minInterval
   }
   t := time.NewTimer(d)

   for {
      // 当maxlife时间到达
      // 或者maxlife发生改变及db被close
      select {
      case <-t.C:
      case <-db.cleanerCh: // maxLifetime was changed or db was closed.
      }

      db.mu.Lock()
      d = db.maxLifetime
      if db.closed || db.numOpen == 0 || d <= 0 {
         db.cleanerCh = nil
         db.mu.Unlock()
         return
      }

      // 循环处理free状态的连接
      expiredSince := nowFunc().Add(-d)
      var closing []*driverConn
      for i := 0; i < len(db.freeConn); i++ {
         c := db.freeConn[i]
         if c.createdAt.Before(expiredSince) {
            closing = append(closing, c)
            last := len(db.freeConn) - 1
            db.freeConn[i] = db.freeConn[last]
            db.freeConn[last] = nil
            db.freeConn = db.freeConn[:last]
            i--
         }
      }
      db.maxLifetimeClosed += int64(len(closing))
      db.mu.Unlock()

      for _, c := range closing {
         c.Close()
      }

      // 如果maxlife被重置，需要更新定时器时间
      if d < minInterval {
         d = minInterval
      }
      t.Reset(d)
   }
}

• 1.15 Implementation

func (db *DB) startCleanerLocked() {
  if (db.maxLifetime > 0 || db.maxIdleTime > 0) && db.numOpen > 0 && db.cleanerCh == nil {
    db.cleanerCh = make(chan struct{}, 1)
    go db.connectionCleaner(db.shortestIdleTimeLocked())  // maxidle和maxlife取较小值
  }
}

func (db *DB) connectionCleaner(d time.Duration) {
  const minInterval = time.Second

  if d < minInterval {
    d = minInterval
  }
  t := time.NewTimer(d)

  for {
    select {
    case <-t.C:
    case <-db.cleanerCh: // maxLifetime was changed or db was closed.
    }

    db.mu.Lock()
    d = db.shortestIdleTimeLocked()
    if db.closed || db.numOpen == 0 || d <= 0 {
      db.cleanerCh = nil
      db.mu.Unlock()
      return
    }

    closing := db.connectionCleanerRunLocked()
    db.mu.Unlock()
    for _, c := range closing {
      c.Close()
    }

    if d < minInterval {
      d = minInterval
    }
    t.Reset(d)
  }
}

// 对idle超时和life超时的连接分别收集，统一返回
func (db *DB) connectionCleanerRunLocked() (closing []*driverConn) {
  if db.maxLifetime > 0 {
    expiredSince := nowFunc().Add(-db.maxLifetime)
    for i := 0; i < len(db.freeConn); i++ {
      c := db.freeConn[i]
      if c.createdAt.Before(expiredSince) {
        closing = append(closing, c)
        last := len(db.freeConn) - 1
        db.freeConn[i] = db.freeConn[last]
        db.freeConn[last] = nil
        db.freeConn = db.freeConn[:last]
        i--
      }
    }
    db.maxLifetimeClosed += int64(len(closing))
  }

  if db.maxIdleTime > 0 {
    expiredSince := nowFunc().Add(-db.maxIdleTime)
    var expiredCount int64
    for i := 0; i < len(db.freeConn); i++ {
      c := db.freeConn[i]
      if db.maxIdleTime > 0 && c.returnedAt.Before(expiredSince) {
        closing = append(closing, c)
        expiredCount++
        last := len(db.freeConn) - 1
        db.freeConn[i] = db.freeConn[last]
        db.freeConn[last] = nil
        db.freeConn = db.freeConn[:last]
        i--
      }
    }
    db.maxIdleTimeClosed += expiredCount
  }
  return
}

The implementation logic of 1.14 and 1.15 is basically the same, but it adds compatibility with the judgment of idle timeout

4. Access the database

After we have done the above initialization actions, according to our habits, we usually try to connect to the db to determine whether the connection parameters are normal, such as whether the user name and password are correct, but not sending user requests. The general approach is to call db.Ping(),

func (db *DB) Ping() error {
   return db.PingContext(context.Background())
}

func (db *DB) PingContext(ctx context.Context) error {
   var dc *driverConn
   var err error

   // 获取一个可用连接，后面会看到一样的逻辑，这里先跳过细节
   for i := 0; i < maxBadConnRetries; i++ {
      dc, err = db.conn(ctx, cachedOrNewConn)
      if err != driver.ErrBadConn {
         break
      }
   }
   if err == driver.ErrBadConn {
      dc, err = db.conn(ctx, alwaysNewConn)  // db.conn 是来获取可用连接的，是数据库连接池较为核心的一部分
   }
   if err != nil {
      return err
   }

   // 发送ping命令
   return db.pingDC(ctx, dc, dc.releaseConn)
}

func (db *DB) pingDC(ctx context.Context, dc *driverConn, release func(error)) error {
   var err error
   if pinger, ok := dc.ci.(driver.Pinger); ok {
      withLock(dc, func() {
         err = pinger.Ping(ctx)  // 这里需要驱动自己去实现，对应mysql来说，发送的是sql_type=14(COM_PING)的请求包
      })
   }
   release(err)   // 将该连接放回到free池
   return err
}

5. Send sql request

Here are some of the simplest ways to send sql

// 没有结果集，值返回ok/error包
func (db *DB) Exec(query string, args ...interface{}) (Result, error) {}
func (db *DB) ExecContext(ctx context.Context, query string, args ...interface{}) (Result, error) {}

// 返回大于0条结果集
func (db *DB) Query(query string, args ...interface{}) (*Rows, error) {}
func (db *DB) QueryContext(ctx context.Context, query string, args ...interface{}) (*Rows, error) {}

// 预期结果集只有一行，没有结果集Scan时报ErrNoRows，Scan结果如果有多行，只取第一行，多余的数据行丢弃
func (db *DB) QueryRow(query string, args ...interface{}) *Row {}
func (db *DB) QueryRowContext(ctx context.Context, query string, args ...interface{}) *Row {}

Here are a few notes:

• We can find that each method has another method with Context suffix at the same time. If you look at the calling relationship, you will find that the function without Context (Exec/Query/QueryRow) is actually the function with Context (ExecContext/ QueryContext/QueryRowContext), the Context here is the same as most library functions, used for signal synchronization, such as timeout limits, etc., generally do not need to be set separately
• We can find that each function parameter supports a variable parameter list, and the usage is the same as that of prepare. Use? As a placeholder, then we directly spell SQL or use a placeholder, which is better?

  rows1, err := db.Query("select * from t1 where a = 1”)
  rows2, err := db.Query("select * from t1 where a = ?", 1)

The results of these two SQL executions are the same, but the bottom layer is different, which is slightly different from the specific implementation of different drivers.

Take mysql as an example, the difference is that the first Query actually sends one sql (sql_type: 3), and the second Query actually sends two sql (sql_type: 22 and sql_tyep: 23), prepare first, then execute, Although the binary protocol is faster, it will send two sql each time. The prepare sent for the first time will only be executed once after that and the prepare information will not be actively recycled.

At the beginning of the design of this interface, it should be designed according to the idea of ​​prepare+execute. When the number of placeholder parameters is 0, whether it can optimize and send a sql directly depends on whether the underlying driver interface supports it, in other words, prepare+execute

Next, take Query as an example to see the specific implementation process

func (db *DB) Query(query string, args ...interface{}) (*Rows, error) {
   return db.QueryContext(context.Background(), query, args...)
}

func (db *DB) QueryContext(ctx context.Context, query string, args ...interface{}) (*Rows, error) {
   var rows *Rows
   var err error

   // 执行query，优先从连接池获取连接，如果获取到badconn（以及关闭的连接），重试，最多重试maxBadConnRetries(2)次
   for i := 0; i < maxBadConnRetries; i++ {
      rows, err = db.query(ctx, query, args, cachedOrNewConn)
      if err != driver.ErrBadConn {
         break
      }
   }

   // 一定创建新的连接执行query
   if err == driver.ErrBadConn {
      return db.query(ctx, query, args, alwaysNewConn)
   }
   return rows, err
}

func (db *DB) query(ctx context.Context, query string, args []interface{}, strategy connReuseStrategy) (*Rows, error) {
   // 获取连接
   dc, err := db.conn(ctx, strategy)
   if err != nil {
      return nil, err
   }

   // 使用获取的连接执行查询
   return db.queryDC(ctx, nil, dc, dc.releaseConn, query, args)
}

It can be found that to execute a normal SQL requires two steps. The first step is to obtain the connection (db.conn), and the second step is to execute the query (db.queryDC).

6. Get the connection

// Provides two strategies for obtaining connections, alwaysNewConn & cachedOrNewConn, literally, always create new & prioritize reuse of free connections

func (db *DB) conn(ctx context.Context, strategy connReuseStrategy) (*driverConn, error) {
   // 全局加锁 这里有个连接池的大锁，需要注意
   db.mu.Lock()
   if db.closed {
      db.mu.Unlock()
      return nil, errDBClosed
   }

   // context 超时检测
   select {
   default:
   case <-ctx.Done():
      db.mu.Unlock()
      return nil, ctx.Err()
   }
   lifetime := db.maxLifetime

   // 优先从free池中获取连接
   numFree := len(db.freeConn)
   if strategy == cachedOrNewConn && numFree > 0 {
      // 取第一个free连接
      conn := db.freeConn[0]
      // 切片拷贝
      copy(db.freeConn, db.freeConn[1:])
      // 调整切片长度
      db.freeConn = db.freeConn[:numFree-1]
      conn.inUse = true
      db.mu.Unlock()
      // 检查连接是否超时，超时则返回错误
      if conn.expired(lifetime) {
         conn.Close()
         return nil, driver.ErrBadConn
      }

      // 对连接状态进行重置，通常是使用过的连接需要重置，避免连接已经处于不可用状态
      if err := conn.resetSession(ctx); err == driver.ErrBadConn {
         conn.Close()
         return nil, driver.ErrBadConn
      }
      return conn, nil
   }

   // 已经没有free连接，或者策略要求创建一个新连接

   // 当前打开的连接已经达到了允许打开连接数的上限，需要阻塞等待
   if db.maxOpen > 0 && db.numOpen >= db.maxOpen {
      // Make the connRequest channel. It's buffered so that the
      // connectionOpener doesn't block while waiting for the req to be read.

      // 建立一个唯一key和请求连接connRequest channel的映射
      req := make(chan connRequest, 1)
      reqKey := db.nextRequestKeyLocked()
      db.connRequests[reqKey] = req
      db.waitCount++
      db.mu.Unlock()

      waitStart := time.Now()
      // Timeout the connection request with the context.
      select {
      // 如果超时，从map中删除该key，记录统计信息，并检查连接是否已经就绪
      case <-ctx.Done():
         // Remove the connection request and ensure no value has been sent
         // on it after removing.
         db.mu.Lock()
         delete(db.connRequests, reqKey)
         db.mu.Unlock()
         atomic.AddInt64(&db.waitDuration, int64(time.Since(waitStart)))
         // 如果已经生成了可用连接，将新连接放回到free池中
         select {
         default:
         case ret, ok := <-req:
            if ok && ret.conn != nil {
               db.putConn(ret.conn, ret.err, false)
            }
         }
         return nil, ctx.Err()

      case ret, ok := <-req:
         atomic.AddInt64(&db.waitDuration, int64(time.Since(waitStart)))

         if !ok {
            return nil, errDBClosed
         }
         // Only check if the connection is expired if the strategy is cachedOrNewConns.
         // If we require a new connection, just re-use the connection without looking
         // at the expiry time. If it is expired, it will be checked when it is placed
         // back into the connection pool.
         // This prioritizes giving a valid connection to a client over the exact connection
         // lifetime, which could expire exactly after this point anyway.
         // 对cachedOrNewConn策略的连接请求，需要判断连接是否过期
         // 如果是请求新连接，则不做判断，等连接被放回free池中时再回收
         if strategy == cachedOrNewConn && ret.err == nil && ret.conn.expired(lifetime) {
            ret.conn.Close()
            return nil, driver.ErrBadConn
         }
         if ret.conn == nil {
            return nil, ret.err
         }

         // Reset the session if required.
         if err := ret.conn.resetSession(ctx); err == driver.ErrBadConn {
            ret.conn.Close()
            return nil, driver.ErrBadConn
         }
         return ret.conn, ret.err
      }
   }

   // 由于未达到连接数上限，直接创建新连接
   db.numOpen++ // optimistically
   db.mu.Unlock()
   ci, err := db.connector.Connect(ctx)
   if err != nil {
      db.mu.Lock()
      db.numOpen-- // correct for earlier optimism
      db.maybeOpenNewConnections()
      db.mu.Unlock()
      return nil, err
   }
   db.mu.Lock()
   dc := &driverConn{
      db:        db,
      createdAt: nowFunc(),
      ci:        ci,
      inUse:     true,
   }
   db.addDepLocked(dc, dc)
   db.mu.Unlock()
   return dc, nil
}

In summary, when we apply for a connection to the connection pool,

• If the strategy is cachedOrNewConn and there is in the free connection pool, it will be taken out directly;
• If there is no idle connection in the connection pool or the policy is alwaysNewConn, and the current connection does not exceed the upper limit, it will be created directly;
• Otherwise, the channel is used to create and establish asynchronously, and the call site blocks and waits for the connection.

7. Execute query

Query

// ctx 是调用sql设置的上下文
// txctx 是事务的上下文，如果有
// releaseConn 上层传递的函数句柄，连接使用完后，将该连接放回到连接池

func (db *DB) queryDC(ctx, txctx context.Context, dc *driverConn, releaseConn func(error), query string, args []interface{}) (*Rows, error) {
   queryerCtx, ok := dc.ci.(driver.QueryerContext)
   var queryer driver.Queryer
   if !ok {
      queryer, ok = dc.ci.(driver.Queryer)
   }
   if ok {
      var nvdargs []driver.NamedValue
      var rowsi driver.Rows
      var err error
      withLock(dc, func() {
         nvdargs, err = driverArgsConnLocked(dc.ci, nil, args)
         if err != nil {
            return
         }
         rowsi, err = ctxDriverQuery(ctx, queryerCtx, queryer, query, nvdargs)
      })
      // err要么为nil，要么为ErrSkip以外的其他错误
      // ErrSkip 通常为某些可选接口不存在，可以尝试其他接口
      if err != driver.ErrSkip {
         if err != nil {
            releaseConn(err)
            return nil, err
         }
         // err ！= nil
         // 数据库连接的所有权转交给了rows，rows需要主动Close，以将该连接放回到free连接池中
         rows := &Rows{
            dc:          dc,
            releaseConn: releaseConn,
            rowsi:       rowsi,
         }

         // 通过context，当收到上层事件或者事务关闭的消息，rows能够自动调用Close释放连接
         rows.initContextClose(ctx, txctx)
         return rows, nil
      }
   }

   // prepare
   var si driver.Stmt
   var err error
   withLock(dc, func() {
      si, err = ctxDriverPrepare(ctx, dc.ci, query)
   })
   if err != nil {
      releaseConn(err)
      return nil, err
   }

   // execute
   ds := &driverStmt{Locker: dc, si: si}
   rowsi, err := rowsiFromStatement(ctx, dc.ci, ds, args...)
   if err != nil {
      ds.Close()
      releaseConn(err)
      return nil, err
   }

   // Note: ownership of ci passes to the *Rows, to be freed
   // with releaseConn.
   rows := &Rows{
      dc:          dc,
      releaseConn: releaseConn,
      rowsi:       rowsi,
      closeStmt:   ds,
   }

   // 同上
   rows.initContextClose(ctx, txctx)
   return rows, nil
}

It can be found that at the sql package level, all connection management actions have been done. The specific sending and receiving package/package protocol logic is implemented by different drivers. When the query is executed, the ownership of the connection is transferred to rows Object means that rows actively call the Close() function to put the currently used connection back into the connection pool.

QueryRow

Similarly, QueryRow() and Query() actually use a set of methods at the bottom, and the return value is just a layer

func (db *DB) QueryRow(query string, args ...interface{}) *Row {
   return db.QueryRowContext(context.Background(), query, args...)
}

func (db *DB) QueryRowContext(ctx context.Context, query string, args ...interface{}) *Row {
   rows, err := db.QueryContext(ctx, query, args...)
   return &Row{rows: rows, err: err}
}

// Row 和 Rows 的关系
type Row struct {
   // One of these two will be non-nil:
   err  error // deferred error for easy chaining
   rows *Rows
}

If you are careful, you can find that Row only provides a Scan method, and even Close() is not. Compared with Rows, it looks a little thin, so how to release the connection?

In the Scan() method of Row, the first piece of data is read from rows, and at the end, the Close() method of rows is called

func (r *Row) Scan(dest ...interface{}) error {
   if r.err != nil {
      return r.err
   }

   defer r.rows.Close()
   for _, dp := range dest {
      if _, ok := dp.(*RawBytes); ok {
         return errors.New("sql: RawBytes isn't allowed on Row.Scan")
      }
   }

   if !r.rows.Next() {
      if err := r.rows.Err(); err != nil {
         return err
      }
      return ErrNoRows
   }
   err := r.rows.Scan(dest...)
   if err != nil {
      return err
   }
   // Make sure the query can be processed to completion with no errors.
   return r.rows.Close()
}

It means that when we use QueryRow(), we must use row.Scan() to get the result, otherwise the connection will not be put back into the connection pool.

Exec

Because Exec does not require a result set, the release of the connection is not as troublesome as the first two, and the processing flow is basically the same.

func (db *DB) execDC(ctx context.Context, dc *driverConn, release func(error), query string, args []interface{}) (res Result, err error) {
   // 调用 Exec 函数就不需要额外关心连接的release，在函数结束之前就放回free池中
   defer func() {
      release(err)
   }()
   execerCtx, ok := dc.ci.(driver.ExecerContext)
   var execer driver.Execer
   if !ok {
      execer, ok = dc.ci.(driver.Execer)
   }

   // 和Query一样，如果驱动有实现这两个接口，就直接调用，否则由sql包主动触发调用prepare+execute
   if ok {
      var nvdargs []driver.NamedValue
      var resi driver.Result
      withLock(dc, func() {
         nvdargs, err = driverArgsConnLocked(dc.ci, nil, args)
         if err != nil {
            return
         }
         resi, err = ctxDriverExec(ctx, execerCtx, execer, query, nvdargs)
      })
      if err != driver.ErrSkip {
         if err != nil {
            return nil, err
         }
         return driverResult{dc, resi}, nil
      }
   }

   var si driver.Stmt
   withLock(dc, func() {
      si, err = ctxDriverPrepare(ctx, dc.ci, query)
   })
   if err != nil {
      return nil, err
   }
   ds := &driverStmt{Locker: dc, si: si}
   defer ds.Close()

   // 从 statement 中保存结果
   return resultFromStatement(ctx, dc.ci, ds, args...)
}

8. Use stmt gracefully

As mentioned above, the direct use of placeholders to execute binary sql will actually send two sql each time, which does not improve the execution efficiency. What is the correct way to execute the statement?

stmt, err := db.Prepare("select * from t1 where a = ?”)   // prepare，sql_type=22
if err != nil {
   return
}
_, err = stmt.Exec(1)  // 第一次执行，sql_type=23
if err != nil {
   return
}
rows, err := stmt.Query(1)  // 第二次执行，连接所有权转交给rows，sql_type=23
if err != nil {
   return
}
_ = rows.Close()  // 归还连接的所有权

_ = stmt.Close()  // sql_type=25 

We know that db is a connection pool object, where prepare only needs to be called once, and then when stmt is executed, if a new connection is obtained or a connection that has not been prepared before, then it will first call prepare, and then execute execute Therefore, we don't need to worry about whether we will execute on a connection that has not been prepared.
Similarly, when stmt calls Close(), it will execute close on all connections and close this stmt. Therefore, before closing, ensure that this stmt will not be executed again.

9. Release the connection

As mentioned earlier, our connection needs to be returned to the freeConn connection pool in time after executing an ordinary query. Although the ownership of the intermediate connection will be transferred, it will eventually need to be recycled. In fact, the request to open a transaction is similar. The connection is released after the transaction is committed or rolled back. The method of connection release is continuously passed down from the upper layer, and all objects that may have connection ownership may receive the release of the connection to the method.

// 用来将使用完的连接放回到free连接池中

func (dc *driverConn) releaseConn(err error) {
   dc.db.putConn(dc, err, true)
}

func (db *DB) putConn(dc *driverConn, err error, resetSession bool) {
   // 检查连接是否还能复用
   if err != driver.ErrBadConn {
      if !dc.validateConnection(resetSession) {
         err = driver.ErrBadConn
      }
   }

   // debugGetPut 是测试信息
   db.mu.Lock()
   if !dc.inUse {
      db.mu.Unlock()
      if debugGetPut {
         fmt.Printf("putConn(%v) DUPLICATE was: %s\n\nPREVIOUS was: %s", dc, stack(), db.lastPut[dc])
      }
      panic("sql: connection returned that was never out")
   }

   if err != driver.ErrBadConn && dc.expired(db.maxLifetime) {
      err = driver.ErrBadConn
   }
   if debugGetPut {
      db.lastPut[dc] = stack()
   }
   dc.inUse = false

   // 在这个连接上注册的一些statement的关闭函数
   for _, fn := range dc.onPut {
      fn()
   }
   dc.onPut = nil

   // 如果当前连接已经不可用，意味着可能会有新的连接请求，调用maybeOpenNewConnections进行检测
   if err == driver.ErrBadConn {
      // Don't reuse bad connections.
      // Since the conn is considered bad and is being discarded, treat it
      // as closed. Don't decrement the open count here, finalClose will
      // take care of that.
      db.maybeOpenNewConnections()
      db.mu.Unlock()
      dc.Close()
      return
   }

   // hook 的一个函数，用于测试，默认为nil
   if putConnHook != nil {
      putConnHook(db, dc)
   }
   added := db.putConnDBLocked(dc, nil)
   db.mu.Unlock()

   if !added {
      dc.Close()
      return
   }
}

10. Connection Management

The connection management mainly includes connection application, connection recovery and reuse, and asynchronous release of overtime connections.

The entire process of connection management is as follows

11. How to fix a connection without opening the transaction

Through the previous content, it can be found that the connection completes a request without opening the transaction and is returned to the free pool, so even if two selects are executed consecutively, it is possible that the same actual database is not used Connection, for some special scenarios, such as when we finish executing a stored procedure and want to select an output result, this does not meet the requirements.

Simplify the requirements, in fact, we want to occupy a connection for a long time, opening a transaction is a solution, but the additional introduction of transactions may cause the delayed release of the lock (take mysql two-stage lock as an example), here you can use the Context method To achieve, usage examples

{
   var a int
   ctx := context.Background()
   cn, err := db.Conn(ctx)  // 绑定一个连接
   if err != nil {
      return
   }

   // 执行第一次查询，将连接所有权转交给rows1
   rows1, err := cn.QueryContext(ctx, "select * from t1")
   if err != nil {
      return
   }
   _ = rows1.Scan(&a)
   _ = rows1.Close() // rows1 close，将连接所有权交给cn 

   // 执行第二次查询，将连接所有权转交给rows2
   rows2, err = cn.QueryContext(ctx, "select * from t1")
   if err != nil {
      return
   }
   _ = rows2.Scan(&a)
   _ = rows2.Close() // rows1 close，将连接所有权交给cn

   // cn close，连接回收，放回free队列
   _ = cn.Close()
}

Regarding the sql.Conn object returned by db.Conn( ), it needs to be distinguished from driver.Conn. sql.Conn is another package of driverConn, which provides a continuous single database connection for the driver. Driver.Conn is a different driver to achieve Interface

// Conn represents a single database connection rather than a pool of database
// connections. Prefer running queries from DB unless there is a specific
// need for a continuous single database connection.
//
// A Conn must call Close to return the connection to the database pool
// and may do so concurrently with a running query.
//
// After a call to Close, all operations on the
// connection fail with ErrConnDone.

type Conn struct {
   db *DB

   // closemu prevents the connection from closing while there
   // is an active query. It is held for read during queries
   // and exclusively during close.
   closemu sync.RWMutex

   // dc is owned until close, at which point
   // it's returned to the connection pool.
   dc *driverConn

   // done transitions from 0 to 1 exactly once, on close.
   // Once done, all operations fail with ErrConnDone.
   // Use atomic operations on value when checking value.
   done int32
}

12. Monitor connection pool status

Because the mysql protocol is synchronous, when the client has a large number of concurrent requests, but the number of connections is less than the number of concurrent requests, some of the requests will be blocked and wait for other requests to release the connection. In some scenarios or improper use Under the circumstances, this may also become a bottleneck. However, the database does not record the connection waiting time of each request in detail. It only provides the sum of the accumulated waiting time and other monitoring indicators, which can be used as a reference when locating problems.

The library provides the db.Stats() method, which will obtain all the monitoring indicators from the db object and generate the object DBStats object

func (db *DB) Stats() DBStats {
   wait := atomic.LoadInt64(&db.waitDuration)

   db.mu.Lock()
   defer db.mu.Unlock()

   stats := DBStats{
      MaxOpenConnections: db.maxOpen,

      Idle:            len(db.freeConn),
      OpenConnections: db.numOpen,
      InUse:           db.numOpen - len(db.freeConn),

      WaitCount:         db.waitCount,
      WaitDuration:      time.Duration(wait),
      MaxIdleClosed:     db.maxIdleClosed,
      MaxLifetimeClosed: db.maxLifetimeClosed,
   }
   return stats
}

A simple usage example

func monitorConn(db *sql.DB) {
   go func(db *sql.DB) {
      mt := time.NewTicker(monitorDbInterval * time.Second)
      for {
         select {
         case <-mt.C:
            stat := db.Stats()
            logutil.Errorf("monitor db conn(%p): maxopen(%d), open(%d), use(%d), idle(%d), "+
               "wait(%d), idleClose(%d), lifeClose(%d), totalWait(%v)",
               db,
               stat.MaxOpenConnections, stat.OpenConnections,
               stat.InUse, stat.Idle,
               stat.WaitCount, stat.MaxIdleClosed,
               stat.MaxLifetimeClosed, stat.WaitDuration)
         }
      }
   }(db)
}

It should be noted that before 1.15, the statistics of the stat.MaxLifetimeClosed object will be abnormal, and it has been fixed after 1.15.

Attention

• Pay attention to the transfer relationship of the connection owner, and recycle it in time after use, such as rows.Close(), row.Scan(), etc. Failure to recycle will cause connection leakage, and new requests will be blocked all the time
• Try to avoid using placeholders to execute SQL. It is recommended to complete SQL splicing or use stmt normally.
• After 1.15, the restriction on the idle time of a single connection is supported
• db.Conn() can continue to occupy a connection, but in this connection, there is no way to call the stmt generated by the previous prepare, but it can be in the transaction, tx.Stmt() can generate stmt specific to the transaction
• Go provides a database connection pool recycling strategy, which is aimed at freeConn. In other words, if the connection is occupied all the time, even if it has exceeded the lifetime, it will not be recycled.
• We have noticed that every time a connection pool is operated, a global big lock must be added first. Therefore, when the number of connections is large (>1000) and the request volume is large, there will be more serious lock competition. One thing can be found through the top (sys) indicator and pprof, because a simple way is to split a large connection pool into multiple small connection pools. In general, the request is sent through simple polling. Scattered on multiple connection pools, can effectively reduce the granularity of the lock

---

【Finish】