Using Golang program performance optimization and Pprof

The performance optimization of the program is nothing more than the optimization of the resources occupied by the program. For a server, the two most important resources are CPU and memory. Performance optimization means that we usually require the program's CPU memory usage to be as low as possible without affecting the program's data processing capabilities. Conversely, that is, when the CPU and memory usage of the program remain unchanged, the data processing capability or throughput of the program should be improved as much as possible.

Go's native toolchain provides a wealth of tools for developers to use, including pprof .

The use of pprof is divided into the following two parts.

Web programs use pprof

Write a simple Web service program first. The program receives requests on port 9876.

package main

import (
    "bytes" "io/ioutil" "log" "math/rand" "net/http" _ "net/http/pprof" ) func main() { http.HandleFunc("/test", handler) log.Fatal(http.ListenAndServe(":9876", nil)) } func handler(w http.ResponseWriter, r *http.Request) { err := r.ParseForm() if nil != err { w.Write([]byte(err.Error())) return } doSomeThingOne(10000) buff := genSomeBytes() b, err := ioutil.ReadAll(buff) if nil != err { w.Write([]byte(err.Error())) return } w.Write(b) } func doSomeThingOne(times int) { for i := 0; i < times; i++ { for j := 0; j < times; j++ { } } } func genSomeBytes() *bytes.Buffer { var buff bytes.Buffer for i := 1; i < 20000; i++ { buff.Write([]byte{'0' + byte(rand.Intn(10))}) } return &buff }

You can see that we simply introduced it net/http/pprof , and didn't use it explicitly.

starting program.

We wrk use to simply mock the request.

wrk -c 400 -t 8 -d 3m http://localhost:9876/test

When we open http://localhost:9876/debug/pprofit, the following page will be displayed:

Users can click on the corresponding link to browse the content. But that's not what we're talking about, and it doesn't seem intuitive.

We open the link http://localhost:9876/debug/pprof/profile a moment later and can download to the file profile.

Open it with the pprof tool that comes with Go. go tool pprof test profile. (The test followed by proof is the executable file compiled by the program)

Enter top the command to get:

You can see the top 10 cpu consuming functions, and we can optimize this analysis.

It's just that this may not be very intuitive.

We enter the command web(graphviz needs to be installed in advance, it can be done under macOS brew install graphviz), and the interface will be opened in the browser as follows:

You can see that main.doSomeThingOne takes 92.46% of the CPU time and needs to be optimized.

The CPU time graph in the form of the web is completely sufficient for optimization. Here we will introduce the generation of the flame graph. macOS recommended go-torch tools. The method of use is go tool pprof similar.

go-torch test profile A torch.svg file is generated. It can be opened with a browser, as shown in the figure.

I just talked about the generation and viewing of the CPU occupancy analysis file. In fact, the generation of memory snapshots is similar. http://localhost:9876/debug/pprof/heap, the file will be downloaded heap.gz .

We can also use go tool pprof test heap.gzit, and then enter top or web command to view the relevant content.

Generic program using pprof

The Go programs we write are not all Web programs, and it is not possible to use the above method at this time.

We can still use the pprof tool, but it was introduced at runtime/pprof .

Two functions are posted here as examples:

// 生成 CPU 报告
func cpuProfile() {
    f, err := os.OpenFile("cpu.prof", os.O_RDWR|os.O_CREATE, 0644) if err != nil { log.Fatal(err) } defer f.Close() log.Println("CPU Profile started") pprof.StartCPUProfile(f) defer pprof.StopCPUProfile() time.Sleep(60 * time.Second) fmt.Println("CPU Profile stopped") } // 生成堆内存报告 func heapProfile() { f, err := os.OpenFile("heap.prof", os.O_RDWR|os.O_CREATE, 0644) if err != nil { log.Fatal(err) } defer f.Close() time.Sleep(30 * time.Second) pprof.WriteHeapProfile(f) fmt.Println("Heap Profile generated") }

The two functions generate cpu.prof and heap.prof file respectively. The tools can still be used go tool pprof for analysis, so I won't go into details here.

Trace report

Paste the code directly:

// 生成追踪报告
func traceProfile() {
    f, err := os.OpenFile("trace.out", os.O_RDWR|os.O_CREATE, 0644) if err != nil { log.Fatal(err) } defer f.Close() log.Println("Trace started") trace.Start(f) defer trace.Stop() time.Sleep(60 * time.Second) fmt.Println("Trace stopped") }

Using the tool go tool trace to analyze, you will get a very detailed tracking report for more in-depth program analysis and optimization. Since the content of the report is more complicated and the usage method is similar, it will not be continued. Readers can try it for themselves.

Post a picture on the Internet for everyone to have a look at:

Reference: https://github.com/caibirdme/hand-to-hand-optimize-go

Category: [86]golang