[Series] go-gin-api routing middleware - logging (c)

Outline

First synchronization under the project:

The articles shared, planning and project directory parameter validation, parameter validation which is used validator.v8 version has been updated to version validator.v9, github to see the latest code.

This article we share: routing middleware - logging.

Log is a particularly important thing to help us troubleshoot the problem, this article we will implement logging to a text file.

This is my plan, to be recorded parameters:

- request 请求数据
    - request_time
    - request_method
    - request_uri
    - request_proto
    - request_ua
    - request_referer
    - request_post_data
    - request_client_ip
    
- response 返回数据
    - response_time
    - response_code
    - response_msg
    - response_data
    
- cost_time 花费时间

Gin comes Logger middleware framework, the framework of our understanding comes Logger middleware meets our needs?

gin.Logger()

Let's use gin.Logger () to see results.

Increasing the code route.go SetupRouter method:

engine.Use(gin.Logger())

After running multiple requests several times, log output at the command line:

[GIN] 2019/08/30 - 21:24:16 | 200 |     178.072µs |             ::1 | GET      /ping
[GIN] 2019/08/30 - 21:24:27 | 200 |     367.997µs |             ::1 | POST     /product
[GIN] 2019/08/30 - 21:24:28 | 200 |    2.521592ms |             ::1 | POST     /product

First solve the first problem, how to output logs to the text?

Increasing the code route.go SetupRouter method:

f, _ := os.Create(config.AppAccessLogName)
gin.DefaultWriter = io.MultiWriter(f)
engine.Use(gin.Logger())

After running multiple requests several times, log output in the file:

[GIN] 2019/08/30 - 21:36:07 | 200 |     369.023µs |             ::1 | GET      /ping
[GIN] 2019/08/30 - 21:36:08 | 200 |      27.585µs |             ::1 | GET      /ping
[GIN] 2019/08/30 - 21:36:10 | 200 |      14.302µs |             ::1 | POST     /product

Although the recording to file successfully, but the record is not what we want the parameters of the way.

How to do it?

We need to customize a log middleware, recorded in accordance with the parameters we need.

Custom Logger ()

middleware/logger/logger.go

package logger

import (
    "bytes"
    "encoding/json"
    "fmt"
    "github.com/gin-gonic/gin"
    "go-gin-api/app/config"
    "go-gin-api/app/util"
    "log"
    "os"
)

type bodyLogWriter struct {
    gin.ResponseWriter
    body *bytes.Buffer
}
func (w bodyLogWriter) Write(b []byte) (int, error) {
    w.body.Write(b)
    return w.ResponseWriter.Write(b)
}
func (w bodyLogWriter) WriteString(s string) (int, error) {
    w.body.WriteString(s)
    return w.ResponseWriter.WriteString(s)
}

func SetUp() gin.HandlerFunc {
    return func(c *gin.Context) {
        bodyLogWriter := &bodyLogWriter{body: bytes.NewBufferString(""), ResponseWriter: c.Writer}
        c.Writer = bodyLogWriter

        //开始时间
        startTime := util.GetCurrentMilliTime()

        //处理请求
        c.Next()

        responseBody := bodyLogWriter.body.String()

        var responseCode int
        var responseMsg  string
        var responseData interface{}

        if responseBody != "" {
            response := util.Response{}
            err := json.Unmarshal([]byte(responseBody), &response)
            if err == nil {
                responseCode = response.Code
                responseMsg  = response.Message
                responseData = response.Data
            }
        }

        //结束时间
        endTime := util.GetCurrentMilliTime()

        if c.Request.Method == "POST" {
            c.Request.ParseForm()
        }

        //日志格式
        accessLogMap := make(map[string]interface{})

        accessLogMap["request_time"]      = startTime
        accessLogMap["request_method"]    = c.Request.Method
        accessLogMap["request_uri"]       = c.Request.RequestURI
        accessLogMap["request_proto"]     = c.Request.Proto
        accessLogMap["request_ua"]        = c.Request.UserAgent()
        accessLogMap["request_referer"]   = c.Request.Referer()
        accessLogMap["request_post_data"] = c.Request.PostForm.Encode()
        accessLogMap["request_client_ip"] = c.ClientIP()

        accessLogMap["response_time"] = endTime
        accessLogMap["response_code"] = responseCode
        accessLogMap["response_msg"]  = responseMsg
        accessLogMap["response_data"] = responseData

        accessLogMap["cost_time"] = fmt.Sprintf("%vms", endTime - startTime)

        accessLogJson, _ := util.JsonEncode(accessLogMap)

        if f, err := os.OpenFile(config.AppAccessLogName, os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0666); err != nil {
            log.Println(err)
        } else {
            f.WriteString(accessLogJson + "\n")
        }
    }
}

After running multiple requests several times, log output in the file:

{"cost_time":"0ms","request_client_ip":"::1","request_method":"GET","request_post_data":"","request_proto":"HTTP/1.1","request_referer":"","request_time":1567172568233,"request_ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36","request_uri":"/ping","response_code":1,"response_data":null,"response_msg":"pong","response_time":1567172568233}
{"cost_time":"0ms","request_client_ip":"::1","request_method":"GET","request_post_data":"","request_proto":"HTTP/1.1","request_referer":"","request_time":1567172569158,"request_ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36","request_uri":"/ping","response_code":1,"response_data":null,"response_msg":"pong","response_time":1567172569158}
{"cost_time":"0ms","request_client_ip":"::1","request_method":"POST","request_post_data":"name=admin","request_proto":"HTTP/1.1","request_referer":"","request_time":1567172629565,"request_ua":"PostmanRuntime/7.6.0","request_uri":"/product","response_code":-1,"response_data":null,"response_msg":"Key: 'ProductAdd.Name' Error:Field validation for 'Name' failed on the 'NameValid' tag","response_time":1567172629565}

OK, all the parameters we want all records!

Throw a few questions:

1, there is no open-source logging tools?

Of course, where logrus is the largest, with this powerful tool, and I have shared the original, you can look at the original article "Using logrus log collection" .

2. Why logging to text?

Because the log platform can be used ELK.

Logstash collected using a text file, use Elasticsearch search engine analysis, eventually showing up in Kibana platform.

3, when a large number of requests over, write to the file will not be a problem?

May be, this can use asynchronous, you can go under the chan, look at the code to achieve specific, and I will not put up.

Source Address

https://github.com/xinliangnote/go-gin-api

go-gin-api series

• 1. go modules initialize the project
• 2. Plan the project directory and parameter validation