The realization principle of defer in Golang

Preface

In Go language, you can use the keyword defer to register the exit call to the function, that is, when the main function exits, the function after defer is called. The function of the defer statement is to automatically execute the relevant code when the function exits regardless of whether the program is abnormal. Therefore, the function behind defer is usually called deferred function

Defer rule

1. The parameters of the delay function are determined when the defer statement appears

func a() {
    
    
    i := 0
    defer fmt.Println(i)
    i++
    return
}

Return result:

The variable i printed in the defer statement has been copied when defer appears, so the subsequent modification of the value of the variable i will not affect the printing result of the defer statement

Note: For pointer type parameters, the rules still apply, except that the parameter of the delayed function is an address value. In this case, the modification of the variable by the statement after defer may affect the delayed function

2. The execution order of defer is opposite to the declaration order

The simple understanding is: defining defer is similar to stacking operations, executing defer is similar to popping operations, first in and out

3. Defer can operate the return value of the main function

The function in the defer statement will be executed after the return statement updates the return value. Because the anonymous function defined in the function can access all the variables of the function including the return value.

func deferFuncReturn() (result int) {
    
    
    i := 1
    defer func() {
    
    
       result++
    }()
    return i
}

Return result:

So the above function actually returns the i++ value.

defer realization principle

Note: I will comment out the function of each method in the source code, you can refer to the comments for understanding.

data structure

Let's first look at the defer structure

src/src/runtime/runtime2.go:_defer

type _defer struct {
    
    
	siz     int32 //defer函数的参数大小
	started bool //是否被调用，默认为false。为true表示已经被调用
	sp      uintptr // sp at time of defer
	pc      uintptr //defer语句下一条语句的地址
	fn      *funcval //需要被延迟执行的函数
	_panic  *_panic //在执行 defer 的 panic 结构体
	link    *_defer //同一个goroutine所有被延迟执行的函数通过该成员链在一起形成一个链表
}

We know that in every goroutine structure there is a _defer pointer variable to store the defer singly linked list.
As shown below:

Creation and execution of defer

Let's first look at how the assembly translates the defer keyword

	0x0082 00130 (test.go:16)	CALL	runtime.deferproc(SB)
	0x0087 00135 (test.go:16)	TESTL	AX, AX
	0x0089 00137 (test.go:16)	JNE	155
	0x008b 00139 (test.go:19)	XCHGL	AX, AX
	0x008c 00140 (test.go:19)	CALL	runtime.deferreturn(SB)

Defer is translated into two processes. Runtime.deferproc is executed to generate a description structure of the println function and its related parameters, and then it is mounted on the _defer pointer of the current g.
Let's first look at the implementation of the deferproc function

deferproc

runtime\panic.go

// Create a new deferred function fn with siz bytes of arguments.
// The compiler turns a defer statement into a call to this.
//go:nosplit
func deferproc(siz int32, fn *funcval) {
    
     // arguments of fn follow fn
 	//用户goroutine才能使用defer
	if getg().m.curg != getg() {
    
    
		// go code on the system stack can't defer
		throw("defer on system stack")
	}
	//也就是调用deferproc之前的rsp寄存器的值
	sp := getcallersp()
	// argp指向defer函数的第一个参数
	argp := uintptr(unsafe.Pointer(&fn)) + unsafe.Sizeof(fn)
	// 存储的是 caller 中，call deferproc 的下一条指令的地址
	// deferproc函数的返回地址
	callerpc := getcallerpc()

	//创建defer
	d := newdefer(siz)
	if d._panic != nil {
    
    
		throw("deferproc: d.panic != nil after newdefer")
	}
	//需要延迟执行的函数
	d.fn = fn
	//记录deferproc函数的返回地址
	d.pc = callerpc
	//调用deferproc之前rsp寄存器的值
	d.sp = sp
	switch siz {
    
    
	case 0:
		// Do nothing.
	case sys.PtrSize:
		*(*uintptr)(deferArgs(d)) = *(*uintptr)(unsafe.Pointer(argp))
	default:
	 	//通过memmove拷贝defered函数的参数
		memmove(deferArgs(d), unsafe.Pointer(argp), uintptr(siz))
	}

	// deferproc通常都会返回0
	return0()

}

The key is newdefer

func newdefer(siz int32) *_defer {
    
    
	var d *_defer
	sc := deferclass(uintptr(siz))
	//获取当前goroutine的g结构体对象
	gp := getg()
	if sc < uintptr(len(p{
    
    }.deferpool)) {
    
    
		pp := gp.m.p.ptr()//与当前工作线程绑定的p
		if len(pp.deferpool[sc]) == 0 && sched.deferpool[sc] != nil {
    
    
			// Take the slow path on the system stack so
			// we don't grow newdefer's stack.
			systemstack(func() {
    
    //切换到系统栈
				lock(&sched.deferlock)
				//从全局_defer对象池拿一些到p的本地_defer对象池
				for len(pp.deferpool[sc]) < cap(pp.deferpool[sc])/2 && sched.deferpool[sc] != nil {
    
    
					d := sched.deferpool[sc]
					sched.deferpool[sc] = d.link
					d.link = nil
					pp.deferpool[sc] = append(pp.deferpool[sc], d)
				}
				unlock(&sched.deferlock)
			})
		}
		if n := len(pp.deferpool[sc]); n > 0 {
    
    
			d = pp.deferpool[sc][n-1]
			pp.deferpool[sc][n-1] = nil
			pp.deferpool[sc] = pp.deferpool[sc][:n-1]
		}
	}
	//如果p的缓存中没有可用的_defer结构体对象则从堆上分配
	if d == nil {
    
    
		// Allocate new defer+args.
		//因为roundupsize以及mallocgc函数都不会处理扩栈，所以需要切换到系统栈执行
		systemstack(func() {
    
    
			total := roundupsize(totaldefersize(uintptr(siz)))
			d = (*_defer)(mallocgc(total, deferType, true))
		})
		if debugCachedWork {
    
    
			// Duplicate the tail below so if there's a
			// crash in checkPut we can tell if d was just
			// allocated or came from the pool.
			d.siz = siz
			//把新分配出来的d放入当前goroutine的_defer链表头
			d.link = gp._defer
			gp._defer = d
			return d
		}
	}
	d.siz = siz
	d.link = gp._defer
	//把新分配出来的d放入当前goroutine的_defer链表头
	gp._defer = d
	return d
}

In functiondeferprocin,

• First obtain the value of the rsp register before calling deferproc, and then use this value to determine whether the defer to be executed belongs to the current caller when deferreturn is performed later
• bynewdefer The function allocates a _defer structure object and puts it into the head of the _defer linked list of the current goroutine
• Then copy the parameter part to the address immediately after the defer object:deferArgs(d)=unsafe.Pointer(d)+unsafe.Sizeof(*d)
• carried outreturn0Function, returns 0 under normal circumstances, and continues to execute business logic after testing %eax and %eax. Under abnormal conditions, it returns 1 and jumps directly todeferreturn

deferreturn

runtime\panic.go

// 编译器会在调用过 defer 的函数的末尾插入对 deferreturn 的调用
// 如果有被 defer 的函数的话，这里会调用 runtime·jmpdefer 跳到对应的位置
// 实际效果是会一遍遍地调用 deferreturn 直到 _defer 链表被清空
// 这里不能进行栈分裂，因为我们要该函数的栈来调用 defer 函数
func deferreturn(arg0 uintptr) {
    
    
	gp := getg()
	// defer函数链表
	// 也是第一个defer
	d := gp._defer
	if d == nil {
    
    
		//由于是递归调用，
		//递归终止
		return
	}
	//获取调用deferreturn时的栈顶位置
	sp := getcallersp()
	// 判断当前栈顶位置是否和defer中保存的一致
	if d.sp != sp {
    
    
		//如果保存在_defer对象中的sp值与调用deferretuen时的栈顶位置不一样，直接返回
        //因为sp不一样表示d代表的是在其他函数中通过defer注册的延迟调用函数，比如:
        //a()->b()->c()它们都通过defer注册了延迟函数，那么当c()执行完时只能执行在c中注册的函数
		return
	}

	//把保存在_defer对象中的fn函数需要用到的参数拷贝到栈上，准备调用fn
    //注意fn的参数放在了调用调用者的栈帧中，而不是此函数的栈帧中
	switch d.siz {
    
    
	case 0:
		// Do nothing.
	case sys.PtrSize:
		*(*uintptr)(unsafe.Pointer(&arg0)) = *(*uintptr)(deferArgs(d))
	default:
		memmove(unsafe.Pointer(&arg0), deferArgs(d), uintptr(d.siz))
	}
	fn := d.fn
	d.fn = nil
	// 指向 defer 链表下一个节点
	gp._defer = d.link
	 // 进行释放，归还到相应的缓冲区或者让gc回收
	freedefer(d)
	//执行defer中的func
	jmpdefer(fn, uintptr(unsafe.Pointer(&arg0)))
}

In function == deferreturn ==

• Determine whether there is still a bound defer on the current goroutine, if not, return directly. If so, get the defer at the head of the linked list
• By judging whether the sp stored in the current defer is consistent with the caller's sp, it is proved that the current defer is not declared in this calling function.
• Copy the parameters needed by the fn function saved in the _defer object to the stack, and prepare to call the function after defer
• Release defer
• byjmpdeferFunc after the function executes defer

to sum up

• In the compiling stage, the function deferproc() is inserted at the declaration defer, and the function deferreturn() is inserted before the function return.
• The deferred function parameters defined by defer have been determined when the defer statement is issued.
• Defer definition order is opposite to actual execution order
• Use the defer mechanism for anonymous functions to enable them to observe the return value of the function