introduce
Add API logging middleware to gorilla/mux microservices through a complete example .
What is a log interceptor/middleware?
The log interceptor will log every API request.
We will use rk-boot to start the gorilla/mux microservice .
Please visit the following address for the full tutorial: https://github.com/rookie-ninja/rk-mux
Install
go get github.com/rookie-ninja/rk-boot/mux
quick start
rk-boot integrates the following three open source libraries by default.
- uber-go/zap is used as the underlying logging library.
- logrus as log rolling.
- loki is stored remotely as a log.
1. Create boot.yaml
The boot.yaml file describes the original information of GoFrame framework startup, and rk-boot starts gorilla/mux by reading boot.yaml .
To verify, we started the commonService at the same time. CommonService contains a series of common APIs. Details: CommonService
---
mux:
- name: greeter # Required
port: 8080 # Required
enabled: true # Required
commonService:
enabled: true # Optional, enable common service
interceptors:
loggingZap:
enabled: true # Optional, enable logging interceptor
2. Create main.go
// Copyright (c) 2021 rookie-ninja
//
// Use of this source code is governed by an Apache-style
// license that can be found in the LICENSE file.
package main
import (
"context"
"fmt"
"github.com/rookie-ninja/rk-boot"
"github.com/rookie-ninja/rk-boot/mux"
"github.com/rookie-ninja/rk-mux/interceptor"
"net/http"
)
func main() {
// Create a new boot instance.
boot := rkboot.NewBoot()
// Register handler
entry := rkbootmux.GetMuxEntry("greeter")
entry.Router.NewRoute().Methods(http.MethodGet).Path("/v1/greeter").HandlerFunc(Greeter)
// Bootstrap
boot.Bootstrap(context.TODO())
boot.WaitForShutdownSig(context.TODO())
}
func Greeter(writer http.ResponseWriter, request *http.Request) {
rkmuxinter.WriteJson(writer, http.StatusOK, &GreeterResponse{
Message: fmt.Sprintf("Hello %s!", request.URL.Query().Get("name")),
})
}
type GreeterResponse struct {
Message string
}
3. Folder structure
$ tree
.
├── boot.yaml
├── go.mod
├── go.sum
└── main.go
0 directories, 4 files
4. Start main.go
$ go run main.go
2022-02-11T15:43:33.130+0800 INFO boot/mux_entry.go:643 Bootstrap muxEntry {"eventId": "1a7f1d5a-13d7-4796-8108-939285d3ec13", "entryName": "greeter", "entryType": "Mux"}
------------------------------------------------------------------------
endTime=2022-02-11T15:43:33.130747+08:00
startTime=2022-02-11T15:43:33.130545+08:00
elapsedNano=202290
timezone=CST
ids={"eventId":"1a7f1d5a-13d7-4796-8108-939285d3ec13"}
app={"appName":"rk","appVersion":"","entryName":"greeter","entryType":"Mux"}
env={"arch":"amd64","az":"*","domain":"*","hostname":"lark.local","localIP":"10.8.0.2","os":"darwin","realm":"*","region":"*"}
payloads={"commonServiceEnabled":true,"commonServicePathPrefix":"/rk/v1/","muxPort":8080}
counters={}
pairs={}
timing={}
remoteAddr=localhost
operation=Bootstrap
resCode=OK
eventStatus=Ended
EOE
5. Verify
We send the /rk/v1/healthy request that comes with CommonService.
$ curl -X GET localhost:8080/rk/v1/healthy
{
"healthy": true
}
Send a request to /v1/greeter.
$ curl "localhost:8080/v1/greeter?name=rk-dev"
{"Message":"Hello rk-dev!"}
EventLog will output to stdout by default.
The log format below is from rk-query , and users can also choose JSON format, which we will introduce later.
------------------------------------------------------------------------
endTime=2022-02-11T15:44:20.00081+08:00
startTime=2022-02-11T15:44:20.000749+08:00
elapsedNano=61165
timezone=CST
ids={"eventId":"c786ff55-78b4-4c44-a268-581aa16def16"}
app={"appName":"rk","appVersion":"","entryName":"greeter","entryType":"Mux"}
env={"arch":"amd64","az":"*","domain":"*","hostname":"lark.local","localIP":"10.8.0.2","os":"darwin","realm":"*","region":"*"}
payloads={"apiMethod":"GET","apiPath":"/v1/greeter","apiProtocol":"HTTP/1.1","apiQuery":"name=rk-dev","userAgent":"curl/7.64.1"}
counters={}
pairs={}
timing={}
remoteAddr=127.0.0.1:57341
operation=/v1/greeter
resCode=200
eventStatus=Ended
EOE
Modify log format
We can modify the log format by modifying boot.yaml. Currently supports json and console formats, the default is console.
By modifying the value of eventLoggerEncoding to json, we can output the log in JSON format.
mux:
- name: greeter # Required
port: 8080 # Required
enabled: true # Required
commonService:
enabled: true # Optional, enable common service
interceptors:
loggingZap:
enabled: true # Optional, enable logging interceptor
zapLoggerEncoding: "json" # Override to json format, option: json or console
eventLoggerEncoding: "json" # Override to json format, option: json or console
{
"endTime":"2022-02-11T15:45:27.481+0800",
"startTime":"2022-02-11T15:45:27.480+0800",
"elapsedNano":60615,
"timezone":"CST",
"ids":{
"eventId":"f497d8bb-578f-485a-977b-7de7fc5b560e"
},
"app":{
"appName":"rk",
"appVersion":"",
"entryName":"greeter",
"entryType":"Mux"
},
"env":{
"arch":"amd64",
"az":"*",
"domain":"*",
"hostname":"lark.local",
"localIP":"10.8.0.2",
"os":"darwin",
"realm":"*",
"region":"*"
},
"payloads":{
"apiMethod":"GET",
"apiPath":"/v1/greeter",
"apiProtocol":"HTTP/1.1",
"apiQuery":"name=rk-dev",
"userAgent":"curl/7.64.1"
},
"error":{},
"counters":{},
"pairs":{},
"timing":{},
"remoteAddr":"127.0.0.1:61646",
"operation":"/v1/greeter",
"eventStatus":"Ended",
"resCode":"200"
}
Modify log path
Output paths can be specified by modifying the value of eventLoggerOutputPaths.
Logs are cut and compressed after 1GB by default.
---
mux:
- name: greeter # Required
port: 8080 # Required
enabled: true # Required
commonService:
enabled: true # Optional, enable common service
interceptors:
loggingZap:
enabled: true # Optional, enable logging interceptor
zapLoggerOutputPaths: ["logs/app.log"] # Override output paths
eventLoggerOutputPaths: ["logs/event.log"] # Override output paths
.
├── boot.yaml
├── go.mod
├── go.sum
├── logs
│ └── event.log
└── main.go
Write directly to Loki (remote log storage)
loki is a cloud native log storage, search open source service. Lighter than ElasticSearch , storage cost is very low (can use cloud object storage). The search engine and storage engine of ElasticSearch are very advanced, but the difficulty of operation and maintenance, price, ease of use, and configuration threshold are high, and it is not suitable for simple services.
loki uses the traditional Agent mode to collect logs. rk-boot internally uses a loki-client to transfer logs directly to the Loki service. One aspect of this use is to eliminate the trouble of Agent configuration. Secondly, it is to eliminate the situation of multi-line log. For example, when printing Panic information, we must configure a regular expression in the Agent to tell the Agent to integrate the Panic into a Stream and send it to the Loki service. However, this configuration is very cumbersome.
In the case of a large number of logs, it may affect the speed. Although rk-boot sends logs to Loki asynchronously, the speed will inevitably be affected because of the locks involved.
So, if it is a large log situation, you can use the traditional Agent mode.
1.boot.yaml
Additional definitions for zapLogger, eventLogger, mux.logger.zapLogger & mux.logger.eventLogger.
zapLogger:
- name: zap-logger
loki:
enabled: true
eventLogger:
- name: event-logger
loki:
enabled: true
mux:
- name: greeter # Required
port: 8080 # Required
enabled: true # Required
logger:
zapLogger: zap-logger # Optional, reference of zapLogger entry name
eventLogger: event-logger # Optional, reference of eventLogger entry name
interceptors:
loggingZap:
enabled: true # Optional, enable logging interceptor
2. Start Loki locally
To verify, we start Loki locally using Docker.
$ docker run -d --name=loki -p 3100:3100 grafana/loki
3. Start Grafana locally
We use Grafana search to view logs.
$ docker run -p 3000:3000 --name grafana grafana/grafana
Adding Loki data sources in Grafana Grafana is just a web UI tool, in order to see the data report, we tell Grafana where to look for Loki.
Because Grafana runs in Docker, we don't use localhost:9090, but instead, host.docker.internal:9090.
4. Start main.go & send request
$ go run main.go
$ curl "localhost:8080/v1/greeter?name=rk-dev"
5. View logs in Grafana
6. Complete Loki Configuration
---
eventLogger:
- name: event-logger # Required
loki:
enabled: true # Optional, default: false
addr: localhost:3100 # Optional, default: localhost:3100
path: /loki/api/v1/push # Optional, default: /loki/api/v1/push
username: "" # Optional, default: ""
password: "" # Optional, default: ""
maxBatchWaitMs: 3000 # Optional, default: 3000
maxBatchSize: 1000 # Optional, default: 1000
insecureSkipVerify: false # Optional, default: false
labels: # Optional, default: empty map
my_label_key: my_label_value
zapLogger:
- name: zap-logger # Required
loki:
enabled: true # Optional, default: false
addr: localhost:3100 # Optional, default: localhost:3100
path: /loki/api/v1/push # Optional, default: /loki/api/v1/push
username: "" # Optional, default: ""
password: "" # Optional, default: ""
maxBatchWaitMs: 3000 # Optional, default: 3000
maxBatchSize: 1000 # Optional, default: 1000
insecureSkipVerify: false # Optional, default: false
labels: # Optional, default: empty map
my_label_key: my_label_value
concept
After verifying the log interceptor, let's talk about the functions of the log interceptor provided by rk-boot.
We need to understand two concepts in advance.
- EventLogger
- ZapLogger
ZapLogger
It is used to record error/detailed logs. Users can obtain the ZapLogger instance of this RPC call and write the log. The ZapLogger instance of each RPC contains the current RequestId.
2022-02-11T15:47:04.571+0800 INFO boot/mux_entry.go:643 Bootstrap muxEntry {"eventId": "7b995b92-d77b-4fed-861d-cbfd53738768", "entryName": "greeter", "entryType": "Mux"}
EventLogger
The RK enabler treats each RPC request as an Event and logs it using the Event type in rk-query.
| field | Details |
|---|---|
| endTime | End Time |
| startTime | Starting time |
| elapsedNano | Event time overhead (Nanoseconds) |
| timezone | Time zone |
| ids | Contains eventId, requestId and traceId. If the original data interceptor is activated, or event.SetRequest() is called by the user, the new RequestId will be used, and the eventId and requestId will be exactly the same. If the call chain interceptor is enabled, the traceId will be logged. |
| app | 包含 appName, appVersion, entryName, entryType。 |
| env | Contains arch, az, domain, hostname, localIP, os, realm, region. realm, region, az, domain fields. These fields come from system environment variables (REALM, REGION, AZ, DOMAIN). "*" means the environment variable is empty. |
| payloads | Contains RPC related information. |
| error | contains errors. |
| counters | Operate through event.SetCounter(). |
| pairs | Operate through event.AddPair(). |
| timing | Operation through event.StartTimer() and event.EndTimer(). |
| remoteAddr | RPC remote address. |
| operation | RPC name. |
| resCode | RPC return code. |
| eventStatus | Ended or InProgress |
------------------------------------------------------------------------
endTime=2022-02-11T15:44:20.00081+08:00
startTime=2022-02-11T15:44:20.000749+08:00
elapsedNano=61165
timezone=CST
ids={"eventId":"c786ff55-78b4-4c44-a268-581aa16def16"}
app={"appName":"rk","appVersion":"","entryName":"greeter","entryType":"Mux"}
env={"arch":"amd64","az":"*","domain":"*","hostname":"lark.local","localIP":"10.8.0.2","os":"darwin","realm":"*","region":"*"}
payloads={"apiMethod":"GET","apiPath":"/v1/greeter","apiProtocol":"HTTP/1.1","apiQuery":"name=rk-dev","userAgent":"curl/7.64.1"}
counters={}
pairs={}
timing={}
remoteAddr=127.0.0.1:57341
operation=/v1/greeter
resCode=200
eventStatus=Ended
EOE
Log middleware options
| name | describe | Types of | Defaults |
|---|---|---|---|
| mux.interceptors.loggingZap.enabled | Start log interceptor | boolean | false |
| mux.interceptors.loggingZap.zapLoggerEncoding | Log format: json or console | string | console |
| mux.interceptors.loggingZap.zapLoggerOutputPaths | log file path | []string | stdout |
| mux.interceptors.loggingZap.eventLoggerEncoding | Log format: json or console | string | console |
| mux.interceptors.loggingZap.eventLoggerOutputPaths | log file path | []string | stdout |
Get RPC log instance
Every time an RPC request comes in, the interceptor will inject the RequestId (when the original data interceptor is activated) into the log instance.
In other words, for every RPC request, there will be a new Logger instance. Let's see how to log ZapLogger for an RPC request.
Obtain the log instance of this request through the rkmuxctx.GetLogger(ctx) method.
func Greeter(writer http.ResponseWriter, request *http.Request) {
rkmuxctx.GetLogger(request, writer).Info("Request received")
rkmuxinter.WriteJson(writer, http.StatusOK, &GreeterResponse{
Message: fmt.Sprintf("Hello %s!", request.URL.Query().Get("name")),
})
}
The log prints out!
2022-02-11T15:49:33.513+0800 INFO mux/main.go:33 Request received
Modify Event
The logging interceptor will create an Event instance for each RPC request.
User can add pairs, counters, errors.
Get the Event instance of this RPC through rkmuxctx.GetEvent(ctx).
func Greeter(writer http.ResponseWriter, request *http.Request) {
event := rkmuxctx.GetEvent(request)
event.AddPair("key", "value")
rkmuxinter.WriteJson(writer, http.StatusOK, &GreeterResponse{
Message: fmt.Sprintf("Hello %s!", request.URL.Query().Get("name")),
})
}
Added pairs={"key":"value"} to Event!
------------------------------------------------------------------------
endTime=2022-02-11T15:50:19.286491+08:00
startTime=2022-02-11T15:50:19.286432+08:00
elapsedNano=59508
timezone=CST
ids={"eventId":"382e39dd-c258-4347-b833-c8e391b05777"}
app={"appName":"rk","appVersion":"","entryName":"greeter","entryType":"Mux"}
env={"arch":"amd64","az":"*","domain":"*","hostname":"lark.local","localIP":"10.8.0.2","os":"darwin","realm":"*","region":"*"}
payloads={"apiMethod":"GET","apiPath":"/v1/greeter","apiProtocol":"HTTP/1.1","apiQuery":"name=rk-dev","userAgent":"curl/7.64.1"}
counters={}
pairs={"key":"value"}
timing={}
remoteAddr=127.0.0.1:63859
operation=/v1/greeter
resCode=200
eventStatus=Ended
EOE