The following content is reproduced from https://mp.weixin.qq.com/s/FF8YU8nXP9TKt0qvSuzmtw
Awa love to write original programs Zhang source Travels 2019-05-07
This article is the 13th chapter of the "Go Language Scheduler Source Code Scenario Analysis" series, and it is also the third subsection of Chapter 2.
In the previous section, we analyzed the initialization of the scheduler. In this section, we look at how the first goroutine in the program is created.
Create main goroutine
Continuing from the previous section, after schedinit completes the initialization of the scheduling system, it returns to the rt0_go function and starts calling newproc() to create a new goroutine to execute the runtime·main function corresponding to mainPC. Look at the following code:
runtime/asm_amd64.s : 197
# create a new goroutine to start program
MOVQ $runtime·mainPC(SB), AX # entry，mainPC是runtime.main
# newproc的第二个参数入栈，也就是新的goroutine需要执行的函数
PUSHQ AX # AX = &funcval{runtime·main},
# newproc的第一个参数入栈，该参数表示runtime.main函数需要的参数大小，因为runtime.main没有参数，所以这里是0
PUSHQ $0
CALL runtime·newproc(SB) # 创建main goroutine
POPQ AX
POPQ AX
# start this M
CALL runtime·mstart(SB) # 主线程进入调度循环，运行刚刚创建的goroutine
# 上面的mstart永远不应该返回的，如果返回了，一定是代码逻辑有问题，直接abort
CALL runtime·abort(SB)// mstart should never return
RET
DATA runtime·mainPC+0(SB)/8,$runtime·main(SB)
GLOB Lruntime·mainPC(SB),RODATA,$8In the subsequent analysis process, we will see that this runtime.main will eventually call the main.main function we wrote. Before analyzing runtime·main, we will focus on the newproc function.
The newproc function is used to create a new goroutine. It has two parameters. Let’s talk about the second parameter fn. The newly created goroutine will be executed from the function fn, and the fn function may also have parameters, the first of newproc Each parameter is the size of the parameter of the fn function in bytes. For example, there is the following Go code snippet:
func start(a, b, c int64) {
......
}
func main() {
go start(1, 2, 3)
}When the compiler compiles the above go statement, it will replace it with a call to the newproc function. The compiled code is logically equivalent to the following pseudo code
func main() {
push 0x3
push 0x2
push 0x1
runtime.newproc(24, start)
}When compiling, the compiler will first use several instructions to push the 3 parameters needed by the start function onto the stack, and then call the newproc function. Because the three int64 parameters of the start function account for a total of 24 bytes, the first parameter passed to newproc is 24, which means that the start function requires a parameter of 24 bytes.
Then why do we need to pass the parameter size of the fn function to the newproc function? The reason is that the newproc function will create a new goroutine to execute the fn function, and this newly created goroutine will use a different stack from the current goroutine, so you need to change the parameters that fn needs to use when creating the goroutine from the current one. The goroutine stack is copied to the stack of the new goroutine before it can be executed. The newproc function itself does not know how much data needs to be copied to the stack of the newly created goroutine, so it is necessary to specify how much data to copy by means of parameters.
After understanding the background knowledge, let's analyze the code of newproc. The newproc function is a wrapper for newproc1. There are two most important preparations here. One is to get the address of the first parameter of the fn function (argp in the code), and the other is to use the systemstack function to switch to the g0 stack. Of course, For our initialization scenario, it is already on the g0 stack, so there is no need to switch. However, this function is universal. A goroutine will also be created in the user's goroutine. At this time, a stack switch is required.
runtime/proc.go : 3232
// Create a new g running fn with siz bytes of arguments.
// Put it on the queue of g's waiting to run.
// The compiler turns a go statement into a call to this.
// Cannot split the stack because it assumes that the arguments
// are available sequentially after &fn; they would not be
// copied if a stack split occurred.
//go:nosplit
func newproc(siz int32, fn *funcval) {
//函数调用参数入栈顺序是从右向左，而且栈是从高地址向低地址增长的
//注意：argp指向fn函数的第一个参数，而不是newproc函数的参数
//参数fn在栈上的地址+8的位置存放的是fn函数的第一个参数
argp := add(unsafe.Pointer(&fn), sys.PtrSize)
gp := getg() //获取正在运行的g，初始化时是m0.g0
//getcallerpc()返回一个地址，也就是调用newproc时由call指令压栈的函数返回地址，
//对于我们现在这个场景来说，pc就是CALLruntime·newproc(SB)指令后面的POPQ AX这条指令的地址
pc := getcallerpc()
//systemstack的作用是切换到g0栈执行作为参数的函数
//我们这个场景现在本身就在g0栈，因此什么也不做，直接调用作为参数的函数
systemstack(func() {
newproc1(fn, (*uint8)(argp), siz, gp, pc)
})
}The first parameter fn of the newproc1 function is the function to be executed by the newly created goroutine. Note that the type of this fn is the funcval structure type, which is defined as follows:
type funcval struct {
fn uintptr
// variable-size, fn-specific data here
}The second parameter argp of newproc1 is the address of the first parameter of the fn function, and the third parameter is the size of the parameter of the fn function in bytes. We don't care about the latter two parameters. It should be noted here that newproc1 is executed on the stack of g0. This function is very long and important, so let's look at it in sections.
runtime/proc.go : 3248
// Create a new g running fn with narg bytes of arguments starting
// at argp. callerpc is the address of the go statement that created
// this. The new g is put on the queue of g's waiting to run.
func newproc1(fn *funcval, argp *uint8, narg int32, callergp *g, callerpc uintptr) {
//因为已经切换到g0栈，所以无论什么场景都有 _g_ = g0，当然这个g0是指当前工作线程的g0
//对于我们这个场景来说，当前工作线程是主线程，所以这里的g0 = m0.g0
_g_ := getg()
......
_p_ := _g_.m.p.ptr() //初始化时_p_ = g0.m.p，从前面的分析可以知道其实就是allp[0]
newg := gfget(_p_) //从p的本地缓冲里获取一个没有使用的g，初始化时没有，返回nil
if newg == nil {
//new一个g结构体对象，然后从堆上为其分配栈，并设置g的stack成员和两个stackgard成员
newg = malg(_StackMin)
casgstatus(newg, _Gidle, _Gdead) //初始化g的状态为_Gdead
//放入全局变量allgs切片中
allgadd(newg) // publishes with a g->status of Gdead so GC scanner doesn't look at uninitialized stack.
}
......
//调整g的栈顶置针，无需关注
totalSize := 4*sys.RegSize + uintptr(siz) + sys.MinFrameSize // extra space in case of reads slightly beyond frame
totalSize += -totalSize & (sys.SpAlign - 1) // align to spAlign
sp := newg.stack.hi - totalSize
spArg := sp
//......
if narg > 0 {
//把参数从执行newproc函数的栈（初始化时是g0栈）拷贝到新g的栈
memmove(unsafe.Pointer(spArg), unsafe.Pointer(argp), uintptr(narg))
// ......
}This code mainly allocates a g structure object from the heap and allocates a stack of 2048 bytes for this newg, and sets the stack member of newg, and then sets the parameters of the function that newg needs to execute from the stack that executes the newproc function. (It is the g0 stack at the time of initialization) Copy to the stack of newg, after completing these things, the state of newg is shown in the following figure:
We can see that after the previous code, there is an additional g structure object we call newg in the program. This object has also obtained a 2k stack space allocated from the heap, the stack of newg. hi and stack.lo respectively point to the start and end positions of their stack space.
Next we continue to analyze the newproc1 function.
runtime/proc.go : 3314
//把newg.sched结构体成员的所有成员设置为0
memclrNoHeapPointers(unsafe.Pointer(&newg.sched), unsafe.Sizeof(newg.sched))
//设置newg的sched成员，调度器需要依靠这些字段才能把goroutine调度到CPU上运行。
newg.sched.sp = sp //newg的栈顶
newg.stktopsp = sp
//newg.sched.pc表示当newg被调度起来运行时从这个地址开始执行指令
//把pc设置成了goexit这个函数偏移1（sys.PCQuantum等于1）的位置，
//至于为什么要这么做需要等到分析完gostartcallfn函数才知道
newg.sched.pc = funcPC(goexit) + sys.PCQuantum // +PCQuantum so that previous instruction is in same function
newg.sched.g = guintptr(unsafe.Pointer(newg))
gostartcallfn(&newg.sched, fn) //调整sched成员和newg的栈This code first initializes the sched member of newg. This member contains some information necessary for the scheduler code to schedule the goroutine to run on the CPU. The sp member of sched represents the stack that should be used when newg is scheduled to run. At the top of the stack, the pc member of sched indicates that instructions will be executed from this address when newg is scheduled to run. However, as can be seen from the above code, new.sched.pc is set to the address of the second instruction of the goexit function. Not fn.fn, why is this? To answer this question, you must go deep into the gostartcallfn function for further analysis.
// adjust Gobuf as if it executed a call to fn
// and then did an immediate gosave.
func gostartcallfn(gobuf *gobuf, fv *funcval) {
var fn unsafe.Pointer
if fv != nil {
fn = unsafe.Pointer(fv.fn) //fn: gorotine的入口地址，初始化时对应的是runtime.main
} else {
fn = unsafe.Pointer(funcPC(nilfunc))
}
gostartcall(gobuf, fn, unsafe.Pointer(fv))
}gostartcallfn first extracts the function address from the parameter fv (runtime.main during initialization), and then continues to call the gostartcall function.
// adjust Gobuf as if it executed a call to fn with context ctxt
// and then did an immediate gosave.
func gostartcall(buf *gobuf, fn, ctxt unsafe.Pointer) {
sp := buf.sp //newg的栈顶，目前newg栈上只有fn函数的参数，sp指向的是fn的第一参数
if sys.RegSize > sys.PtrSize {
sp -= sys.PtrSize
*(*uintptr)(unsafe.Pointer(sp)) = 0
}
sp -= sys.PtrSize //为返回地址预留空间，
//这里在伪装fn是被goexit函数调用的，使得fn执行完后返回到goexit继续执行，从而完成清理工作
*(*uintptr)(unsafe.Pointer(sp)) = buf.pc //在栈上放入goexit+1的地址
buf.sp = sp //重新设置newg的栈顶寄存器
//这里才真正让newg的ip寄存器指向fn函数，注意，这里只是在设置newg的一些信息，newg还未执行，
//等到newg被调度起来运行时，调度器会把buf.pc放入cpu的IP寄存器，
//从而使newg得以在cpu上真正的运行起来
buf.pc = uintptr(fn)
buf.ctxt = ctxt
}There are two main functions of the gostartcall function:
-
Adjust the stack space of newg, put the address of the second instruction of the goexit function on the stack, fake the goexit function to call fn, so that when the ret instruction is executed after the execution of fn is completed, return to goexit to continue execution and complete the final cleanup work;
-
Reset newg.buf.pc to the address of the function that needs to be executed, that is, fn, our scenario is the address of the runtime.main function.
After adjusting the stack and sched members of newg, return to the newproc1 function, we continue to look down,
newg.gopc = callerpc //主要用于traceback
newg.ancestors = saveAncestors(callergp)
//设置newg的startpc为fn.fn，该成员主要用于函数调用栈的traceback和栈收缩
//newg真正从哪里开始执行并不依赖于这个成员，而是sched.pc
newg.startpc = fn.fn
......
//设置g的状态为_Grunnable，表示这个g代表的goroutine可以运行了
casgstatus(newg, _Gdead, _Grunnable)
......
//把newg放入_p_的运行队列，初始化的时候一定是p的本地运行队列，其它时候可能因为本地队列满了而放入全局队列
runqput(_p_, newg, true)
......
}The last code of the newproc1 function is relatively intuitive. First, set several member variables that are not related to scheduling, and then modify the status of newg to _Grunnable and put it into the run queue. This is the first real goroutine in this program. It has been created.
At this time, the state of newg, which is the main goroutine, is shown in the following figure:
This picture looks more complicated, because there are too many arrows representing the pointer. Here is a brief explanation.
-
First of all, the sched member of the newg structure object corresponding to the main goroutine has been initialized. Only the pc and sp members are shown in the figure. The pc member points to the first instruction of the runtime.main function, and the sp member points to the top of the newg stack. The memory unit, which saves the return address after the execution of the runtime.main function is completed, which is the second instruction of the runtime.goexit function. It is expected that the runtime.main function will execute the CALL of the runtime.exit function after the execution of the runtime.main function returns. runtime.goexit1(SB) this instruction;
-
Secondly, newg has been placed in the local run queue of the p structure object bound to the current main thread, because it is the first real goroutine and there is no other goroutine, so it is placed at the head of the local run queue unit;
-
Finally, the m member of newg is nil, because it has not been scheduled to run, and it is not bound to any m.
In this section, we analyzed the creation of the first goroutine in the program, that is, the main goroutine. In the next section, we will continue to analyze how it is dispatched to the CPU by the main worker thread for execution.
Finally, if you think this article is helpful to you, please help me click on the “Looking” at the bottom right corner of the article or forward it to the circle of friends, thank you very much!
