The "pit" that exists when time.After and select are used together

Last night the west wind withered green trees

In the four major usages of select/#timeout control in Golang , it is mentioned that select is matched time.Afterto realize timeout control. Actually, this is a problem.

Since this writing method will initialize a new time.After every time, when the waiting time is long, such as 1 minute, a memory leak will occur (of course, the problem is not limited to this, continue to read)

I don't know who brought it. Using select and time.After for timeout control in Go has almost become a de facto standard. Examples such as
Golang time.After() usage and code examples abound on the Internet

In many large company code repositories, <- time.Afterthere are a lot of keywords for a search, and many of the following time are a few minutes.

It is not difficult to find out with pprof, this is a textbook-level error... It is initialized every time, but it will not be recycled before execution, causing the memory to skyrocket.

It is recommended to search when you have time to see if there is a lot of such usage in the code...

can refer to

Fengyun-Analyze the OOM problem of memory explosion caused by golang time.After

Using time.After with caution will cause memory leaks (golang)

These analysis, the actual verification

after.go：

package main

import (
	"fmt"
	"net/http"
	_ "net/http/pprof"
	"time"
)

/**
  time.After oom 验证demo
*/
func main() {

	go func() {
		// 开启pprof，监听请求
		if err := http.ListenAndServe(":6060", nil); err != nil { // 也可以写成 127.0.0.1:6060
			fmt.Printf("start pprof failed on %s,err%v \n", "6060", err)
		}
	}()

	ch := make(chan string, 100)
	go func() {
		for {
			ch <- "向管道塞入数据"
		}
	}()

	for {
		select {
		case <-ch:
		case <-time.After(time.Minute * 3):
		}
	}
}

run this program, then execute

go tool pprof -http=:8081 http://localhost:6060/debug/pprof/heap

This command line can be divided into three parts:

• -http=:8081It is specified in the form of web and starts on the local port 8081
  (if no -http=:8081parameters are added, it will enter the command line interaction, and entering it on the command line is equivalent webto using the -http=:8081parameters directly)

• http://localhost:6060/debug/pprof/heapIt is the address that specifies to obtain the profile file. Local programs running in real time can use this method. In more cases (such as on the server, there is no port for pprof open to the outside world), you can go to the machine first, use curl http://127.0.0.1:6060/debug/pprof/heap -o heap_cui.outit to get the profile file, and then find a way to get it locally. , using go tool pprof --http :9091 heap_cui.outthe analysis

And over time, the memory occupied by the program will continue to increase

从调用图可发现， 程序不断调用time.After，进而导致计时器 time.NerTimer 不断创建和内存申请

// After waits for the duration to elapse and then sends the current time
// on the returned channel.
// It is equivalent to NewTimer(d).C.
// The underlying Timer is not recovered by the garbage collector
// until the timer fires. If efficiency is a concern, use NewTimer
// instead and call Timer.Stop if the timer is no longer needed.

//After 等待持续时间过去，然后在返回的通道上发送当前时间。
//它相当于 NewTimer(d).C。
//在定时器触发之前，垃圾收集器不会恢复底层定时器。 如果效率是一个问题，请改用 NewTimer 并在不再需要计时器时调用 Timer.Stop。

func After(d Duration) <-chan Time {
	return NewTimer(d).C
}

在select里面虽然没有执行到time.After，但每次都会初始化，会在时间堆里面，定时任务未到期之前，是不会被gc清理的

• 在计时器触发之前，垃圾收集器不会回收Timer

• 如果考虑效率，需要使用NewTimer替代

衣带渐宽终不悔

使用NewTimer 或NewTicker替代：

package main

import (
	"fmt"
	"net/http"
	_ "net/http/pprof"
	"time"
)

/**
  time.After oom 验证demo
*/
func main() {

	go func() {
		// 开启pprof，监听请求
		if err := http.ListenAndServe(":6060", nil); err != nil { // 也可以写成 127.0.0.1:6060
			fmt.Printf("start pprof failed on %s,err%v \n", "6060", err)
		}
	}()

	ticker := time.NewTicker(time.Minute * 3)
    // 或
    //timer := time.NewTimer(3 * time.Minute)
    //defer timer.Stop()
    // 下方的case <-ticker.C:相应改为case <-timer.C:


	ch := make(chan string, 100)
	go func() {
		for {
			ch <- "向管道塞入数据"
		}
	}()

	for {
		select {
		case <-ch:
		case <-ticker.C:
			print("结束执行")
		}
	}

}

这篇Go 内存泄露之痛，这篇把 Go timer.After 问题根因讲透了！应该有点问题，不是内存孤儿，gc还是会去回收的，只是要在time.After到期之后

众里寻他千百度

如上是网上大多数技术文章的情况：

• 昨夜西风凋碧树，独上高楼，望断天涯路: select + time.After实现超时控制

• 衣带渐宽终不悔，为伊消得人憔悴: 这样写有问题，会内存泄露，要用NewTimer 或NewTicker替代time.After

其实针对本例，这些说法都没有切中肯綮

最初的代码仅仅是有内存泄露的问题吗？

实际上，即便3分钟后，第2个case也得不到执行 (可以把3min改成2s验证下)

只要第一个case能不断从channel中取出数据(在此显然可以)，那第二个case就永远得不到执行。这是因为每次time.After都被重新初始化了，而上面那个case一直满足条件，当然就是第二个case一直得不到执行， 除非第一个case超过3min没有从channel中拿到数据

所以其实在此例中NewTimer还是NewTicker，都不是问题本质，这个问题本质，就是个变量作用域的问题

在for循环外定义time.After(time.Minute * 3)，如下：

package main

import (
	"fmt"
	"net/http"
	_ "net/http/pprof"
	"time"
)

func main() {

	go func() {
		// 开启pprof，监听请求
		if err := http.ListenAndServe(":6060", nil); err != nil { // 也可以写成 127.0.0.1:6060
			fmt.Printf("start pprof failed on %s,err%v \n", "6060", err)
		}
	}()

	ch := make(chan string, 100)
	go func() {
		for {
			ch <- "向管道塞入数据"
		}
	}()

	timeout := time.After(time.Minute * 3)
	for {
		select {
		case <-ch:
		case <-timeout:
			fmt.Println("到了这里")
		}
	}
}

Put time.After outside the loop, you can see that there is no memory leak. After 3 minutes (maybe a little more), the second case is executed as scheduled.

So in this scenario, it's not time.After

The garbage collector will not recycle the Timer until the timer fires

The problem, but at the very least the most ignored variable scoping problem..

(The programmer's pot is not the problem of time.After... The reason why NewTimer or NewTicker will not leak memory is just because it is initialized outside the for loop...)

case <-time.After(time.Minute * 3)The previous writing in the for loop, the effect is similar to the following:

package main

import "time"

func main() {

	for {
		time.After(2 * time.Second)
	}

}

Verify whether it will become a so-called "memory orphan"

Modify the program to verify:

The garbage collector will not recycle the Timer until the timer fires

;

but in

After the timer fires, the garbage collector will recycle these timers

, does not cause "memory orphans"

package main

import (
	"fmt"
	"net/http"
	_ "net/http/pprof"
	"sync/atomic"
	"time"
)

func main() {

	go func() {
		// 开启pprof，监听请求
		if err := http.ListenAndServe(":6060", nil); err != nil { // 也可以写成 127.0.0.1:6060
			fmt.Printf("start pprof failed on %s,err%v \n", "6060", err)
		}
	}()

	after()

	fmt.Println("程序结束")

}

func after() {
	var i int32
	ch := make(chan string, 0)
	done := make(chan string) // 设定的时间已到，通知结束循环，不要再往channel里面写数据

	go func() {

		for {

			select {
			default:
				atomic.AddInt32(&i, 1)
				ch <- fmt.Sprintf("%s%d%s", "向管道第", i, "次塞入数据")
			case exit := <-done:
				fmt.Println("关闭通道", exit)
				return
			}

		}
	}()

	go func() {
		time.Sleep(time.Second)
		done <- "去给我通知不要再往ch这个channel里写数据了！"
	}()

	for {
		select {
		case res := <-ch:
			fmt.Println("res:", res)
		case <-time.After(2 * time.Second):
			fmt.Println("结束接收通道的数据")
			return
		}
	}

}

Remove the printed information and replace it with the current real-time memory information:

package main

import (
	"fmt"
	"net/http"
	_ "net/http/pprof"
	"runtime"
	"sync/atomic"
	"time"
)

func main() {

	go func() {
		// 开启pprof，监听请求
		if err := http.ListenAndServe(":6060", nil); err != nil { // 也可以写成 127.0.0.1:6060
			fmt.Printf("start pprof failed on %s,err%v \n", "6060", err)
		}
	}()

	after()

	fmt.Println("程序结束")

}

func after() {
	var ms runtime.MemStats
	runtime.ReadMemStats(&ms)
	fmt.Println("before, have", runtime.NumGoroutine(), "goroutines,", ms.Alloc, "bytes allocated", ms.HeapObjects, "heap object")

	var i int32
	ch := make(chan string, 0)
	done := make(chan string) // 设定的时间已到，通知结束循环，不要再往channel里面写数据

	go func() {

		for {

			select {
			default:
				atomic.AddInt32(&i, 1)
				ch <- fmt.Sprintf("%s%d%s", "向管道第", i, "次塞入数据")
			case exit := <-done:
				fmt.Println("关闭通道", exit)
				return
			}

		}
	}()

	go func() {
		time.Sleep(time.Second)
		done <- "去给我通知不要再往ch这个channel里写数据了！"
	}()

	for {
		select {
		case res := <-ch:
			runtime.GC()
			runtime.ReadMemStats(&ms)
			fmt.Printf("%s,now have %d goroutines,%d bytes allocated, %d heap object \n", res, runtime.NumGoroutine(), ms.Alloc, ms.HeapObjects)
		case <-time.After(2 * time.Second):
			runtime.GC()
			fmt.Println("当前结束接收通道的数据，准备返程")
			runtime.ReadMemStats(&ms)
			fmt.Printf("now have %d goroutines,%d bytes allocated, %d heap object \n", runtime.NumGoroutine(), ms.Alloc, ms.HeapObjects)
			return
		}
	}

}

More references:

Go time.NewTicker() and timer

tech-talk-time.After constantly initialized