Article Directory
pprof module
Performance Optimization Go language projects mainly in the following areas:
- CPU profile: CPU usage report program, according to a certain frequency to collect data and application programs in the CPU registers above
- Memory usage reporting process: Memory Profile (Heap Profile)
- Block Profiling: Report goroutines is not running, and can be used to analyze and find performance bottlenecks deadlock
- Goroutine Profiling: report on the use of goroutines, which goroutine, their relationship is like calling
Go languages built-in tools to obtain operational data of the program, including the following two standard libraries:
runtime/pprof: Applications running collection tools to analyze the datanet/http/pprof: Analysis of the data collection service application runtime
After pprof open, (10ms) will be collected at regular intervals at the current stack information obtaining formatting function CPU and memory resources occupied; Finally, by analyzing these data samples, forming a performance analysis report.
Note that we should only be introduced until pprof in the code when performance testing.
Preparation Tool
Graphical tool
- Installation Kit pprof
go get -u github.com/google/pprof
Analysis of normal use go tool pprofwithout the need to install this package. But if you need the flame chart, it is still to be installed pprof package job.
- Graphviz installation, to support the flame FIG
this installation on the server side, each operating system can use the package management tools, or select source installation:
brew install graphviz # for macos
apt-get install graphviz # for ubuntu
yum install graphviz # for centos
Why you are prompted to install the server
if the server and the client does not exist on this issue in the same machine
, but if the service and the client are not on the same machine, you need to know where installed graphviz
Pressure measuring tool
Not described in detail, if necessary, you can use
https://github.com/wg/wrk or https://github.com/adjust/go-wrk
go-torch
uber go-torch is open source tools, mainly for a flame FIG. Official Address: https://github.com/uber/go-torch
but from Go 1.11, flame graphic visualization go tool pprofdirectly available! So go-torch has been abandoned and archiving recommendedgo tool pprof
Tools applications
If your application is running for some time on the exit end type. So the best way is when the application exit to save the profiling report to a file for analysis. In this case, you can use the runtime / pprof library. First introduced runtime / pprof tool in the code:
import "runtime/pprof"
Online sample code:
package main
import (
"flag"
"fmt"
"log"
"os"
"runtime/pprof"
)
var (
cpuprofile = flag.String("cpuprofile", "", "write cpu profile to file.")
)
func main() {
log.Println("begin")
flag.Parse()
if *cpuprofile != "" {
f, err := os.Create(*cpuprofile)
if err != nil {
log.Fatal(err)
}
pprof.StartCPUProfile(f) // 开启CPU性能分析
defer pprof.StopCPUProfile() // 停止CPU性能分析
}
for i := 0; i < 30; i++ {
nums := fibonacci(i)
fmt.Println(nums)
}
}
func fibonacci(num int) int {
if num < 2 {
return 1
}
return fibonacci(num-1) + fibonacci(num-2)
}
Note that these two analyzes CPU performance:
pprof.StartCPUProfile(f) // 开启CPU性能分析
defer pprof.StopCPUProfile() // 停止CPU性能分析
Create a file named test.go the go and write the above code, execute the command go build test.go. When running the program add a --cpuprofileparameter, for example ./test --cpuprofile=cpu.prof.
When such programs are running CPU profiling data will be recorded cpu.profin, and then you can use go tool pproftools CPU performance analysis.
Optimize the use of memory performance:
pprof.WriteHeapProfile(w io.Writer)
Service applications
If your application is always running, such as web applications, you can use the net/http/pproflibrary, it can be analyzed in the services provided HTTP.
If the default http.DefaultServeMux(usually the code directly http.ListenAndServe(“0.0.0.0:8000”, nil), just import as follows in your web server side codenet/http/pprof
import _ "net/http/pprof" //注意下划线
Online sample code:
package main
import (
_ "net/http/pprof"
"net/http"
)
func main() {
http.ListenAndServe("0.0.0.0:8000", nil)
}
Very simple, the file name test.go, run directly:
go run test.go
Your service will be extra HTTP /debug/pprofendpoint, and then access it in a browser, localhostcan be replaced with your own server IP:
http://localhost:8082/debug/pprof/
Under this path there are several sub-pages:
| Types of | description | Remark |
|---|---|---|
| instead | Sampling information memory allocation of | You can use the browser to open, but not high readability |
| blocks | Sampling information blocking operation conditions | You can use the browser to open, but not high readability |
| cmdline | Display startup command and parameters | You can open a browser |
| goroutine | The current stack information for all coroutines | You can use the browser to open, but not high readability |
| heap | Sample Information heap memory usage | You can use the browser to open, but not high readability |
| mutex | Lock contention sampling information with the situation | You can use the browser to open, but not high readability |
| profile | Sampling CPU usage conditions, continued 30s | Open the browser will download a file |
| threadcreate | System thread created situations sampling information | You can use the browser to open, but not high readability |
| trace | Running trace information | Open the browser will download a file |
Gin framework uses pprof
There are two packages can be used:
package main
import (
"github.com/gin-contrib/pprof"
"github.com/gin-gonic/gin"
)
func main() {
router := gin.Default()
pprof.Register(router)
router.Run(":8080")
}
package main
import (
"github.com/gin-gonic/gin"
"github.com/DeanThompson/ginpprof"
)
func main() {
router := gin.Default()
ginpprof.Wrap(router)
router.Run(":8080")
}
Note:
If you use router.Use (gzip.Gzip (gzip.DefaultCompression)) and other similar functions methods need to comment, or the browser opens the page will trigger download
and performance tested for binding pprof
go testCommand takes two parameters and pprof related, they are designated to generate the CPU and Memory profiling saved files:
- cpuprofile: cpu profiling data file to be saved addresses
- memprofile: memory profiling data to message file address
We can also choose to combine pprof and performance testing, for example:
while performing the following tests, will perform CPU profiling, and store the result in cpu.prof file:
go test -bench . -cpuprofile=cpu.prof
The following while performing the test, will perform Mem profiling, and store the result in cpu.prof file:
go test -bench . -memprofile=./mem.prof
Examples of two upper generated .profdocument is to go tool pprofbe used.
Note that, in general use with Profiling and performance testing, application in Profiling makes sense only under high load conditions.
go tool pprof command
Introduction
Previously wrote a lot of data generated collection of papers, but did not write how to use this data file.
This file is a binary file, can not directly open, you need go tool pprofcommand to parse.
After application, whether instrumental or service applications, we use the appropriate pprof library to obtain data, the next step should be to analyze these data, we can use the go tool pprofcommand-line tool.
go tool pprofThe easiest way is to use:
go tool pprof [binary] [source]
among them:
- application binary is a binary file, to resolve the symbols;
- source represents a source of profile data, can be local file, it can be an http address.
Note: Profiling data acquisition is dynamic, in order to obtain valid data, to ensure that the application is a large load (such as the service is being generated in the running, or by other means to access analog pressure). Otherwise, if the application is idle, the results might not make any sense.
Graphical display
I am now using Gin enable a framework of web services using github.com/gin-contrib/pprofpacket analysis.
This section focuses on the use of go tool pprof, and more project code, the code is not posted. If you want to speak in front of their own in accordance with the test can be done.
$ go run main.go
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
- using env: export GIN_MODE=release
- using code: gin.SetMode(gin.ReleaseMode)
[GIN-debug] GET /debug/pprof/ --> github.com/DeanThompson/ginpprof.IndexHandler.func1 (3 handlers)
[GIN-debug] GET /debug/pprof/heap --> github.com/DeanThompson/ginpprof.HeapHandler.func1 (3 handlers)
[GIN-debug] GET /debug/pprof/goroutine --> github.com/DeanThompson/ginpprof.GoroutineHandler.func1 (3 handlers)
[GIN-debug] GET /debug/pprof/allocs --> github.com/DeanThompson/ginpprof.AllocsHandler.func1 (3 handlers)
[GIN-debug] GET /debug/pprof/block --> github.com/DeanThompson/ginpprof.BlockHandler.func1 (3 handlers)
[GIN-debug] GET /debug/pprof/threadcreate --> github.com/DeanThompson/ginpprof.ThreadCreateHandler.func1 (3 handlers)
[GIN-debug] GET /debug/pprof/cmdline --> github.com/DeanThompson/ginpprof.CmdlineHandler.func1 (3 handlers)
[GIN-debug] GET /debug/pprof/profile --> github.com/DeanThompson/ginpprof.ProfileHandler.func1 (3 handlers)
[GIN-debug] GET /debug/pprof/symbol --> github.com/DeanThompson/ginpprof.SymbolHandler.func1 (3 handlers)
[GIN-debug] POST /debug/pprof/symbol --> github.com/DeanThompson/ginpprof.SymbolHandler.func1 (3 handlers)
[GIN-debug] GET /debug/pprof/trace --> github.com/DeanThompson/ginpprof.TraceHandler.func1 (3 handlers)
[GIN-debug] GET /debug/pprof/mutex --> github.com/DeanThompson/ginpprof.MutexHandler.func1 (3 handlers)
[GIN-debug] Listening and serving HTTP on :8080
I used to develop the service debug mode is activated, you can see that the system automatically adds a lot api / debug / pprof of.
Then open in your browser /debug/pprof:
This page should not be unfamiliar to you. Yes, this is a typical service-oriented application analysis.
Below describes other uses:
We then create a new terminal window, enter the following command, the current stack information for all coroutines:
go tool pprof --seconds 20 http://localhost:8080/debug/pprof/goroutine
This means, the data acquisition and for 20s cpu
may be used another way:
go tool pprof http://localhost:8080/debug/pprof/goroutine?second=20
Look at the results:
$ go tool pprof --seconds 20 http://localhost:8080/debug/pprof/goroutine
Fetching profile over HTTP from http://localhost:8080/debug/pprof/goroutine?seconds=20
Please wait... (20s)
Saved profile in /home/server/pprof/pprof.main.goroutine.001.pb.gz
File: main
Type: goroutine
Time: Dec 18, 2019 at 5:48pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof)
In fact, we have now entered the command line, the last line of (pprof)the input webwill automatically open the local browser and visit the page, of course, this is limited to graphical user interface systems such as MacOS, Ubuntu, Windows and so on. If your operating system is not, it may be the result of the following:
(pprof) web
Couldn't find a suitable web browser!
Set the BROWSER environment variable to your desired browser.
(pprof)
Then how to do it, please pay attention to the top of the fourth row Saved profile, behind /home/server/pprof/pprof.main.goroutine.001.pb.gzis the file we need. How to use this document?
We then create a new terminal window, enter the following command:
go tool pprof -http=0.0.0.0:8081 /home/server/pprof/pprof.main.goroutine.001.pb.gz
Note that at this time need a new port (8081), and the same port can not be before (8080).
Then access directly in your browser http://localhost:8081/, pay attention to modify ip:
If you have not installed graphviz, there will be Could not execute dot; may need to install graphviz.
At this point we see the function call stack information graphically. The upper right corner of the VIEWbar there are some options, you can take a look at each point to open, I do not introduced one by one, but the third Flame Graphyou should like it because it is the flame chart :
Command line interface
Next, a brief introduction go tool pprofusing the command line, the one where we encountered a command, that is web. This section will introduce several commonly used commands.
top
top default viewer occupies the first 10-bit CPU functions
top3 can see the CPU function before three programs occupation:
(pprof) top
Showing nodes accounting for 5, 100% of 5 total
Showing top 10 nodes out of 53
flat flat% sum% cum cum%
3 60.00% 60.00% 3 60.00% runtime.gopark
1 20.00% 80.00% 1 20.00% runtime/pprof.writeRuntimeProfile
1 20.00% 100% 1 20.00% syscall.Syscall
0 0% 100% 1 20.00% bufio.(*Reader).ReadLine
0 0% 100% 1 20.00% bufio.(*Reader).ReadSlice
0 0% 100% 1 20.00% bufio.(*Reader).fill
0 0% 100% 1 20.00% github.com/DeanThompson/ginpprof.GoroutineHandler.func1
0 0% 100% 1 20.00% github.com/gin-gonic/gin.(*Context).Next
0 0% 100% 1 20.00% github.com/gin-gonic/gin.(*Engine).Run
0 0% 100% 1 20.00% github.com/gin-gonic/gin.(*Engine).ServeHTTP
(pprof) top3
Showing nodes accounting for 5, 100% of 5 total
Showing top 3 nodes out of 53
flat flat% sum% cum cum%
3 60.00% 60.00% 3 60.00% runtime.gopark
1 20.00% 80.00% 1 20.00% runtime/pprof.writeRuntimeProfile
1 20.00% 100% 1 20.00% syscall.Syscall
among them:
- flat: the current function CPU-consuming
- flat :: consuming CPU-percentage of the current function
- sun%: cumulative percentage of CPU time-consuming function of occupancy
- cum: current function together called the current function of the CPU-total time
- Total time plus a percentage of the current function called the current function of the CPU-: cum%
- The last one: function name
In most cases, we can analyze these five results of an application operation, and program optimization.
list
We can also use listthe function name command to view specific analysis functions, such as executing list goparksee a detailed analysis of the function we write.
(pprof) list gopark
Total: 5
ROUTINE ======================== runtime.gopark in /usr/local/go/src/runtime/proc.go
3 3 (flat, cum) 60.00% of Total
. . 297: gp.waitreason = reason
. . 298: mp.waittraceev = traceEv
. . 299: mp.waittraceskip = traceskip
. . 300: releasem(mp)
. . 301: // can't do anything that might move the G between Ms here.
3 3 302: mcall(park_m)
. . 303:}
. . 304:
. . 305:// Puts the current goroutine into a waiting state and unlocks the lock.
. . 306:// The goroutine can be made runnable again by calling goready(gp).
. . 307:func goparkunlock(lock *mutex, reason waitReason, traceEv byte, traceskip int) {
ROUTINE ======================== runtime.goparkunlock in /usr/local/go/src/runtime/proc.go
0 1 (flat, cum) 20.00% of Total
. . 303:}
. . 304:
. . 305:// Puts the current goroutine into a waiting state and unlocks the lock.
. . 306:// The goroutine can be made runnable again by calling goready(gp).
. . 307:func goparkunlock(lock *mutex, reason waitReason, traceEv byte, traceskip int) {
. 1 308: gopark(parkunlock_c, unsafe.Pointer(lock), reason, traceEv, traceskip)
. . 309:}
. . 310:
. . 311:func goready(gp *g, traceskip int) {
. . 312: systemstack(func() {
. . 313: ready(gp, traceskip, true)
(pprof)
pdf pdf file visualization may be generated:
(pprof) pdf
Generating report in profile001.pdf
(pprof)
profile001.pdfFile is in the current directory
help
Pprof help perform other file formats, see the command description.
(pprof) help
Commands:
callgrind Outputs a graph in callgrind format
comments Output all profile comments
disasm Output assembly listings annotated with samples
dot Outputs a graph in DOT format
eog Visualize graph through eog
evince Visualize graph through evince
gif Outputs a graph image in GIF format
gv Visualize graph through gv
kcachegrind Visualize report in KCachegrind
list Output annotated source for functions matching regexp
pdf Outputs a graph in PDF format
peek Output callers/callees of functions matching regexp
png Outputs a graph image in PNG format
proto Outputs the profile in compressed protobuf format
ps Outputs a graph in PS format
raw Outputs a text representation of the raw profile
svg Outputs a graph in SVG format
tags Outputs all tags in the profile
text Outputs top entries in text form
top Outputs top entries in text form
topproto Outputs top entries in compressed protobuf format
traces Outputs all profile samples in text form
tree Outputs a text rendering of call graph
web Visualize graph through web browser
weblist Display annotated source in a web browser
o/options List options and their current values
quit/exit/^D Exit pprof
reference
Go pprof tuning
golang pprof combat
pprof (golang performance monitoring and analysis)
golang tuning (go-torch, go tool pprof )
