[Realize the ORM framework from scratch in seven days｜Day02: database/sql basics] After learning to hang the interviewer

Today is the first day of the preparation of the TORM framework, mainly to complete several tasks:

• Simple use of Go-SQL-Driver driver
• Use Go language standard library database/sql to connect and operate MySQL database, and simple package.

Source code: Reply to the keyword "torm" in [Maimo coding] to get the github address link.
Follow-up will record video, article + video + code for [Realizing the TORM framework from zero in seven days]

Drive selection

There are many types of MYSQL drivers supported by Go. Here I check the information and list several mysql drivers. Some of them support the database/sql standard, but some of them use their own implementation interfaces. Common ones are as follows:

• https://github.com/Go-SQL-Driver/MySQL : Fully supports database/sql standards and is written in go language
• https://github.com/ziutek/mymysql : supports database/sql, also supports custom interfaces, and is written entirely in go language
• https://github.com/Philio/GoMySQL : does not support database/sql, it is completely written in go language

In this project, we use the first method as the mysql driver, and I recommend it to everyone here. The main reasons are:

• Fully support database/sql interface
• Support keepalive, keep long connection and connection stability
• High maintenance frequency

Install Go-SQL-Driver

Execute the following command:

go get -u github.com/go-sql-driver/mysql

Go-SQL-Driver driver combat

Go language provides the standard library database/sql to interact with the database, and then briefly familiarize yourself with the use of the following Go-SQL-Driver.

DML operation: add, delete and modify

He has two operation methods, one is to use the Exec function to add, the other is to first use Prepare to obtain stmt , and then call the Exec function to add.

• Use the Exec function to add

func (db *DB) Exec(query string, args ...interface{
    
    }) (Result, error) 

Sample code:

result, err := db.Exec("select * from user where user_name = ?", "迈莫coding")

• First use Prepare to get stmt, then call Exec function to add

func (db *DB) Prepare(query string) (*Stmt, error) {
    
    
  return db.PrepareContext(context.Background(), query) 
}

Sample code:

stmt,err:=db.Prepare("INSERT INTO user(user_name, age) values(?,?)")
//补充完整sql语句，并执⾏
result,err:=stmt.Exec("迈莫coding",1) 

Prepared Statement (Prepared Statement) provides many benefits, so we try to use it in development. The functions provided by using prepared statements are listed below:

• PreparedStatement Can realize the query of custom parameters
• PreparedStatement Usually more efficient than manually splicing string SQL statements
• PreparedStatement Can prevent SQL injection attacks

DQL operation: query

Single statement query

func (db *DB) QueryRow(query string, args ...interface{
    
    }) *Row 

Sample code:

var userName string
var age int
err := db.QueryRow("select user_name,age from user where user_name = ?", "迈莫coding")
err := row.Scan(&userName, &age) 

Multiple statement query

Query gets the data, for xxx.Next() traverses the data:
First use the Query() method to query, if the query is correct, return Rows, which is the information of all rows, similar to the result set.

func (db *DB) Query(query string, args ...interface{
    
    }) (*Rows, error)

We can use the Next() method to determine whether there is the next piece of data. If so, we can use the previous Scan() method to read a row, then continue to judge and continue to obtain. The essence of this process is iteration, so it is usually used in conjunction with loops.

func (rs *Rows) Next() bool

It is recommended to call rows.Close() after each db.Query operation.

Because db.Query() will get a connection from the database connection pool, the underlying connection will be marked as busy before the result set (rows) is closed. When the traversal reads the last record, an internal EOF error will occur, automatically calling rows.Close(), but if you exit the loop early, the rows will not be closed, the connection will not return to the connection pool, and the connection will also If it is not closed, the connection will always be occupied.

Therefore, we usually use defer rows.Close() to ensure that the database connection can be correctly put back into the connection pool.

Core underlying structure Session

First, use the Go-SQL-Driver driver to realize the interaction with the database, which is also the underlying code that TORM will eventually interact with the database. Today we will realize the interaction with the database through the original SQL statement. The code is stored in the raw.go file in the root directory.

raw.go

package session

import (
   "database/sql"
   _ "github.com/go-sql-driver/mysql"
   log "github.com/sirupsen/logrus"
   "strings"
)

// session类 旨于与数据库进行交互
type Session struct {
    
    
   // 数据库引擎
   db     *sql.DB
   // 事务
   tx     *sql.Tx
   // SQL动态参数
   sqlValues []interface{
    
    }
   // SQL语句
   sql strings.Builder
}

// DB return tx if a tx begins otherwise return *sql.DB
func (s *Session) DB() CommonDB {
    
    
   if s.tx != nil {
    
    
      return s.tx
   }
   return s.db
}

// 实例化 Session
func NewSession(db *sql.DB) *Session {
    
    
   return &Session{
    
    db: db}
}
func (s *Session) Clear() {
    
    
   s.sql.Reset()
   s.sqlValues = nil
}

func (s *Session) Raw(sql string, values ...interface{
    
    }) *Session {
    
    
   s.sql.WriteString(sql)
   s.sql.WriteString(" ")
   s.sqlValues = append(s.sqlValues, values...)
   return s
}

Code description:

• Lines 13-22: Create a session structure, which is used to interact with the database at the bottom of the torm framework. It contains several member variables, explained as follows:
  • db: database engine
  • tx: database transaction
  • sqlValues: used to store the parameters in the SQL statement
  • sql: The storage location of the SQL statement
• Line 25~30: Get the database engine
• Line 33~35: instantiate the session object
• Lines 36~39: After each SQL operation is executed, the sql statement and sql parameters will be cleared
• Line 41～46: The user of the Raw() function writes the SQL statement and request parameters into the sql and sqlValues ​​fields in the session structure

Next, you need to encapsulate, add, delete, modify, and check operation functions, the three functions of Exec() , QueryRow() , and Query() .

raw.go

// 定义统一接口 便于后续支持多个数据引擎
type CommonDB interface {
    
    
   Query(query string, args ...interface{
    
    }) (*sql.Rows, error)
   QueryRow(query string, args ...interface{
    
    }) *sql.Row
   Exec(query string, args ...interface{
    
    }) (sql.Result, error)
}

var _ CommonDB = (*sql.DB)(nil)
var _ CommonDB = (*sql.Tx)(nil)

// 增删改操作
func (s *Session) Exec() (result sql.Result, err error) {
    
    
   defer s.Clear()
   log.Info(s.sql.String(), s.sqlValues)
   if result, err = s.DB().Exec(s.sql.String(), s.sqlValues...); err != nil {
    
    
      log.Error(err)
   }
   return
}

// 单条查询操作
func (s *Session) QueryRow() *sql.Row {
    
    
   defer s.Clear()
   log.Info(s.sql.String(), s.sqlValues)
   return s.DB().QueryRow(s.sql.String(), s.sqlValues...)
}

// 多条查询操作
func (s *Session) Query() (rows *sql.Rows, err error) {
    
    
   defer s.Clear()
   log.Info(s.sql.String(), s.sqlValues)
   if rows, err = s.DB().Query(s.sql.String(), s.sqlValues...); err != nil {
    
    
      log.Error(err)
   }
   return
}

Code description:

• Lines 4 to 8: Define a unified interface (common operation method for adding, deleting, modifying, and checking) to facilitate subsequent adaptation to different database engines
• Lines 10~11: The user checks whether the sql.DB type and sql.Tx type implement all the methods in the Common interface. If they are not all implemented, an error will be reported during compilation.
• Lines 14~21: Adding, deleting, modifying the general function package
• Lines 24 to 28: single SQL statement query function package
• Line 31～38: Multiple SQL statement query function package

At this point, the function of interacting with the database (addition, deletion, modification, and check) is basically completed, and now it enters the testing phase.

test

TORM's unit testing is relatively complete. If you have not finished writing a module, you will write unit test code. Since we have finished writing the code for raw.go to interact with the database, let's test whether it is feasible. The code is stored in the raw_test.go file in the root directory. in.

raw_test.go

package session
import (
   "database/sql"
   "os"
   "testing"
   log "github.com/sirupsen/logrus"
)
var db *sql.DB
func TestMain(m *testing.M) {
    
    
   db, _ = sql.Open("mysql", "root:12345678@tcp(127.0.0.1:3306)/po?charset=utf8mb4")
   code := m.Run()
   _ = db.Close()
   os.Exit(code)
}
func New() *Session {
    
    
   return NewSession(db)
}
func TestSession_QueryRow(t *testing.T) {
    
    
   s := New()
   var userName string
   var age int
   s = s.Raw("select user_name,age from user where user_name = ?", "迈莫")
   res := s.QueryRow()
   if err := res.Scan(&userName, &age); err != nil {
    
    
      t.Fatal("failed to query db", err)
   }
   log.Info("userName--", userName)
   log.Info("age--", age)
}
func TestSession_Exec(t *testing.T) {
    
    
   s := New()
   key := "迈莫"
   s = s.Raw("insert into user(user_name, age) values(?, ?)", key, 22)
   _, err := s.Exec()
   if err != nil {
    
    
      t.Fatal("failed to insert db", err)
   }
}
func TestSession_Query(t *testing.T) {
    
    
   s := New()
   var userName string
   var age int
   s = s.Raw("select user_name, age from user")
   rows, err := s.Query()
   if err != nil {
    
    
      t.Fatal("fialed to query db", err)
   }
   for rows.Next() {
    
    
      err = rows.Scan(&userName, &age)
      if err != nil {
    
    
         t.Fatal("fialed to query db", err)
      }
      log.Info("userName--", userName)
      log.Info("age--", age)
   }
}

Test Results:

• TestSession_QueryRow(t *testing.T) Method test results

=== RUN   TestSession_QueryRow
time="2021-01-15T12:29:57+08:00" level=info msg="select user_name,age from user where user_name = ? [迈莫]"
time="2021-01-15T12:29:57+08:00" level=info msg="userName--迈莫"
time="2021-01-15T12:29:57+08:00" level=info msg=age--1
--- PASS: TestSession_QueryRow (0.00s)
PASS
 TestSession_Exec(t *testing.T) 方法测试结果
=== RUN   TestSession_Exec
time="2021-01-15T12:32:51+08:00" level=info msg="insert into user(user_name, age) values(?, ?) [迈莫 21]"
--- PASS: TestSession_Exec (0.00s)
PASS

• TestSession_Query(t *testing.T) Method test results

time="2021-01-15T12:33:35+08:00" level=info msg="select user_name, age from user []"
time="2021-01-15T12:33:35+08:00" level=info msg="userName--迈莫coding"
time="2021-01-15T12:33:35+08:00" level=info msg=age--1
time="2021-01-15T12:33:35+08:00" level=info msg="userName--迈莫"
time="2021-01-15T12:33:35+08:00" level=info msg=age--1
time="2021-01-15T12:33:35+08:00" level=info msg="userName--迈莫"
time="2021-01-15T12:33:35+08:00" level=info msg=age--21
--- PASS: TestSession_Query (0.00s)

Code directory

torm
|--raw.go                  // 底层与数据库交互语句
|--raw_test.go
|--go.mod

At this point, the tasks for the first day have been written. Looking back, the first day I mainly completed the code for interacting with the database, that is, the writing of raw.go code, which is the core function of interacting with the database in the TORM framework.

The article will also continue to be updated. You can search for "Maimo coding" on WeChat to read it for the first time, and reply to "1024" to receive learning go materials.