go performance analysis pprof and trace

• runtime/pprof: Collect the running data of the program (non-Server) for analysis, used for endable code blocks, such as a codec operation, etc.
• net/http/pprof: Collect the runtime data of HTTP Server for analysis. Used for non-endable code blocks, such as web applications, etc.

scenes to be used

Analyze memory leaks'

Summarize

 Memory leaks in go are generally caused by goroutine leaks. The goroutine is not closed, or no timeout control is added, leaving the goroutine in a blocked state and unable to be GCed.

temporary memory leak

• Obtaining a segment of a long string results in the long string not being released
• Obtaining a section of a long slice causes the long slice to not be released
• Creating a new slice in a long slice causes leakage

Permanent memory leak

• goroutine leak
• time.Ticker is not closed causing leaks
• Finalizer causes leak
• Deferring Function Call causes leak

Common ways to use Pprof (including net/http/pprof ):

1. Network access method, go tool to view flame graph

In the program, start a separate coroutine, introduce the package _ "net/http/pprof", and listen to another port as pprof access, as shown in the figure below

func setDebug() error {
	if config.C.General.Debug != "" {
		port := strings.Split(config.C.General.Debug, ":")[1]
		log.WithFields(log.Fields{
			"profile":    "http://127.0.0.1:" + port + "/debug/pprof/profile",
			"goroutines": "http://127.0.0.1:" + port + "/goroutines",
		}).Info("open prof debug")
		go func() {
			http.HandleFunc("/goroutines", func(w http.ResponseWriter, r *http.Request) {
				num := strconv.FormatInt(int64(runtime.NumGoroutine()), 10)
				w.Write([]byte(num))
			})
			http.ListenAndServe(config.C.General.Debug, nil)
		}()
	}
	return nil
}

Or directly share the program listening port. Just introduce the package_ "net/http/pprof" package, as follows

import (
"fmt"
"net/http"
_ "net/http/pprof"
)

func HelloWorld(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintln(w, "hello world")
}

func main() {
	http.HandleFunc("/", HelloWorld)

	err := http.ListenAndServe(":8080", nil)
	if err != nil {
		fmt.Println(err)
	}
}

Execute the program first, then open a terminal window and execute the go tool command. Remember to install the flame graph tool first for easy viewing. The installation method is below.

go tool pprof -http=:9000 http://localhost:88/debug/pprof/goroutine?debug=1 (service address to be monitored)

9000: Indicates the port where you finally view the flame graph. After execution, the default browser will be used and the web page will be automatically opened.

#View stack traces of all current goroutines

go tool pprof http://127.0.0.1:8080/debug/pprof/goroutine

# wait 120s

go tool pprof http://127.0.0.1:8080/debug/pprof/profile?seconds=120

Flame graph installation address :

Download and install the visual graphics software tool graphviz: https://graphviz.org/download/. You can choose the corresponding version of Linux, Windows, Mac, etc., and remember to add the tool to the system environment variable, otherwise an error will be reported: failed to execute dot. Is Graphviz installed? Error: exec: "dot": executable file not found in %PATH% 

 

You can also perform a pressure test before collection, and then collect during the pressure test process, which is helpful for analysis :

• go-wrk -c=400 -t=8 -n=100000 http://127.0.0.1:8000/api/getReq
• 400 connections, 8 threads, simulate 100,000 requests

 2. gin framework, use encapsulated 

"github.com/gin-contrib/pprof"

The ones in the framework are encapsulated. After starting the program, you can access it directly through the browser, and there is no need to use the go tool to execute it (you can execute the stress test command first after the program is started)

Directly access the program http://localhost:88/debug/pprof/

​

The meaning of specific parameters

• allocs: abstraction of memory allocations
• block: Sampling information of blocking stack
• cmdline: program startup command and its parameters
• goroutine: stack information of the current coroutine
• heap: Sampling information of heap memory
• mutex: sampling information of lock competition
• profile: Sampling information of cpu usage
• threadcreate: Sampling information about system program creation
• trace: trace information of program execution

We can see graphical function call stack information in the browser. The bar in the upper right corner VIEWhas some options that you can click to view, Flame Graphwhich is the legendary flame graph.

​

​

Common ways to use Pprof (including runtime/pprof ):

Tool-type application calls are relatively simple, mainly using the runtime/pprof library to write data into files: and then view

package  main
import (
"flag"
"log"
"os"
"runtime/pprof"
"sync"
)

var (
	cpu string
	mem string
)

func init() {
	flag.StringVar(&cpu, "cpu", "", "write cpu profile to file")
	flag.StringVar(&mem, "mem", "", "write mem profile to file")
}

func main() {
	flag.Parse()

	//采样 CPU 运行状态
	if cpu != "" {
		f, err := os.Create(cpu)
		if err != nil {
			log.Fatal(err)
		}
		_ = pprof.StartCPUProfile(f)
		defer pprof.StopCPUProfile()
	}

	var wg sync.WaitGroup
	wg.Add(100)
	for i := 0; i < 100; i++ {
		go workOnce(&wg)
	}
	wg.Wait()

	//采样内存状态
	if mem != "" {
		f, err := os.Create(mem)
		if err != nil {
			log.Fatal(err)
		}
		_ = pprof.WriteHeapProfile(f)
		f.Close()
	}
}

func counter() {
	slice := make([]int, 0)
	var c int
	for i := 0; i < 10000; i++ {
		c = i + 1 + 2 + 3 + 4 + 5
		slice = append(slice, c)
	}
	_ = slice
}

func workOnce(wg *sync.WaitGroup) {
	counter()
	wg.Done()
}

go run main.go --cpu=cpu.pprof --mem=mem.pprof

go tool pprof cpu.pprof

 Click on the file to open. (Prerequisite is that the graphviz tool is installed)

 

 

graphvizIt will appear   if it is not installed Could not execute dot；may need to install graphviz. Here’s how to install graphviz:

brew install graphviz # for macos

apt-get install graphviz # for ubuntu

yum install graphviz # for centos

Trace performance tool usage

1. Use package "runtime/trace"

package main

import (
	"fmt"
	"log"
	"os"
	"runtime/trace"
	"sync"
)

func main() {
	//runtime.GOMAXPROCS(1)
	// 1. 创建trace持久化的文件句柄
	f, err := os.Create("trace.out")
	if err != nil {
		log.Fatalf("failed to create trace output file: %v", err)
	}
	defer func() {
		if err := f.Close(); err != nil {
			log.Fatalf("failed to close trace file: %v", err)
		}
	}()

	// 2. trace绑定文件句柄
	if err := trace.Start(f); err != nil {
		log.Fatalf("failed to start trace: %v", err)
	}
	defer trace.Stop()

	var wg sync.WaitGroup
	wg.Add(2)

	go func() {
		fmt.Println(`Hello`)
		wg.Done()
	}()

	go func() {
		fmt.Println(`World`)
		wg.Done()
	}()

	wg.Wait()
}

Open the window first and execute

go run main.go

Open another window and execute

 go tool trace -http=localhost:8080  trace.out

2023/01/03 11:01:57 Parsing trace...
2023/01/03 11:01:57 Splitting trace...
2023/01/03 11:01:57 Opening browser. Trace viewer is listening on http://127.0.0.1:8888

 View trace: View trace
Goroutine analysis: Goroutine analysis, view the specific Ctrip number
Network blocking profile: Network blocking profile
Synchronization blocking profile: Synchronization blocking profile
Syscall blocking profile: System call blocking profile
Scheduler latency profile: Scheduling delay profile, this is commonly used , used to analyze where the time is spent
User defined tasks: User-defined tasks
User defined regions: User-defined regions
Minimum mutator utilization: Minimum Mutator utilization

reference:

Go language memory leak_cqu_jiangzhou's blog-CSDN blog_go memory leak

http://liumurong.org/2019/12/gin_pprof/

Go pprof performance analysis tool - detailed usage diagram: _dreamer'~'s blog - CSDN blog_How to see the pprof diagram

golang performance analysis trace_raoxiaoya's blog-CSDN blog_golang trace