In-depth understanding of Golang: Channel pipeline

Channel usage

Channel declaration method

• chInt := make(chan int) // unbuffered channel unbuffered channel
• chInt := make(chan int, 0) // unbuffered channel unbuffered channel
• chInt := make(chan int, 2) // buffered channel buffered channel

Channel basic usage

• ch <- x // channel receives data x
• x <- ch // channel sends data and assigns to x
• <- ch // channel sends data, ignore receiver

If a non-buffered channel is used, there needs to be a place to receive data immediately when stuffing data into the buffer at this time, otherwise it will be blocked consistently. The principle is that there is no data in the buffer (no buffering) at this time, and sending data to the channel is regarded as direct sending, that is, sending directly to the coroutine that is waiting for sleep.

func main() {
    
    
    ch := make(chan string)
    // 阻塞
    ch <- "ping"
    <-ch
}

Start the coroutine to get the data:

func main() {
    
    
    ch := make(chan string)

    // 程序能通过
    go func() {
    
    
        <-ch
    }()

    ch<-"ping"
}

memory and communication

Don't communicate by sharing memory, but share memory by communicating. Mainly for:

• Avoid coroutine races and data collisions.
• Higher-level abstraction reduces development difficulty and increases program readability.
• It is easier to decouple modules, enhancing scalability and maintainability.

Through the shared memory case:

func watch(p *int) {
    
    
    for {
    
    
        if *p == 1 {
    
    
            fmt.Println("go")
            break
        }
    }
}

func main() {
    
    
    i := 0
    go watch(&i)

    time.Sleep(time.Second)

    i = 1

    time.Sleep(time.Second)
}

The way of communication is as follows:

func watch(c chan int) {
    
    

    if <-c == 1 {
    
    
        fmt.Println("go")
    }
}

func main() {
    
    
    c := make(chan int)
    go watch(c)

    time.Sleep(time.Second)

    c <- 1

    time.Sleep(time.Second)
}

Channel design

Channel is represented as an hchan structure at the bottom of Go:

type hchan struct {
    
    
    /* 缓存区结构开始 */
    qcount   uint           // total data in the queue
    dataqsiz uint           // size of the circular queue
    buf      unsafe.Pointer // points to an array of dataqsiz elements
    elemsize uint16
    elemtype *_type // element type
    /* 缓存区结构结束*/

    // 发送队列
    sendx    uint   // send index
    sendq    waitq  // list of send waiters

    // 接收队列
    recvx    uint   // receive index
    recvq    waitq  // list of recv waiters

    lock mutex
    // 0：关闭状态；1：开启状态
    closed   uint32
}

type waitq struct {
    
    
    first *sudog
    last  *sudog
}

Data is stored in a ring buffer Ring Buffer, which can reduce memory/GC overhead

Regarding the syntactic sugar of c <- “x”, the principle of channel data transmission
in Go will compile c <- into the chansend1 method:

// %GOROOT%src/runtime/chan.go
//go:nosplit
func chansend1(c *hchan, elem unsafe.Pointer) {
    
    
    chansend(c, elem, true, getcallerpc())
}

3 situations for sending data:

1. There is no data in the buffer, sending data to the channel is regarded as direct sending, and the data is directly copied to the receiver of the coroutine waiting to receive, and wakes up the coroutine; if there is no waiting coroutine to receive, put the data into the buffer .
2. If there is data in the buffer but the buffer is not full, the data will be stored in the buffer.
3. If there is no dormant waiting coroutine in the receiving queue, and the buffer is full, the data will be packaged into sudog, put into the sendq queue for dormant waiting, and then unlocked by the channel.

func chansend(c *hchan, ep unsafe.Pointer, block bool, callerpc uintptr) bool {
    
    
    // ...
    if c.closed != 0 {
    
    
        unlock(&c.lock)
        panic(plainError("send on closed channel"))
    }
    // 1. 取出接收等待队列 recvq 的协程，将数据发送给它
    if sg := c.recvq.dequeue(); sg != nil {
    
    
        send(c, sg, ep, func() {
    
     unlock(&c.lock) }, 3)
        return true
    }

    // 接收等待队列中没有协程时
    if c.qcount < c.dataqsiz {
    
    
        // 2. 缓存空间还有余量，将数据放入缓冲区
        qp := chanbuf(c, c.sendx)
        if raceenabled {
    
    
            racenotify(c, c.sendx, nil)
        }
        typedmemmove(c.elemtype, qp, ep)
        c.sendx++
        if c.sendx == c.dataqsiz {
    
    
            c.sendx = 0
        }
        c.qcount++
        unlock(&c.lock)
        return true
    }
    // ...

    // 3. 休眠等待
    gp := getg()
    // 包装为 sudog
    mysg := acquireSudog()
    mysg.releasetime = 0
    if t0 != 0 {
    
    
        mysg.releasetime = -1
    }

    // 将数据，协程指针等记录到 mysq 中
    mysg.elem = ep
    mysg.waitlink = nil
    mysg.g = gp
    mysg.isSelect = false
    mysg.c = c
    gp.waiting = mysg
    gp.param = nil

    // 将 mysg 自己入队
    c.sendq.enqueue(mysg)

    // 休眠
    gp.parkingOnChan.Store(true)
    gopark(chanparkcommit, unsafe.Pointer(&c.lock), waitReasonChanSend, traceEvGoBlockSend, 2)
    KeepAlive(ep)

    // 被唤醒后再维护一些数据，注意，此时的记录的数据已经被拿走
    if mysg != gp.waiting {
    
    
    throw("G waiting list is corrupted")
    }
    gp.waiting = nil
    gp.activeStackChans = false
    closed := !mysg.success
    gp.param = nil
    // ...
}

Regarding rec <-c grammatical sugar, channel data receiving principle
Go will compile <-c into func chanrecv method, as follows:

• In the compilation stage, rec <- c is transformed intoruntime.chanrecv1()
• In the compilation phase, rec, ok <- c is transformed intoruntime.chanrecv2()
• will eventually call chanrecv()the method

//go:nosplit
func chanrecv1(c *hchan, elem unsafe.Pointer) {
    
    
    chanrecv(c, elem, true)
}

//go:nosplit
func chanrecv2(c *hchan, elem unsafe.Pointer) (received bool) {
    
    
    _, received = chanrecv(c, elem, true)
    return
}

4 cases of receiving data:

1. There is a sending coroutine (sendq) waiting, but the buffer is empty, receiving from the coroutine
2. There is a sending coroutine (sendq) waiting, but the buffer is not empty, and it is received from the cache
3. receive buffer
4. blocking receive

func chanrecv(c *hchan, ep unsafe.Pointer, block bool) (selected, received bool) {
    
    
    // ...
    if c.closed != 0 {
    
    
        // ...
    } else {
    
    
        // 1.，2. 接收数据前，已经有协程在休眠等待发送数据
        if sg := c.sendq.dequeue(); sg != nil {
    
    
            // 在 1 的情况下，缓存为空，直接从 sendq 的协程取数据
            // 在 2 的情况下，从缓存（缓冲）取走数据后，将 sendq 里的等待中的协程的数据放入缓存，并唤醒该协程
            // 这就是为什么 sendq 队列中的协程被唤醒后，其携带的数据已经被取走的原因
            recv(c, sg, ep, func() {
    
     unlock(&c.lock) }, 3)
            return true, true
        }
    }

    // 3. 直接从缓存接收数据
    if c.qcount > 0 {
    
    
        qp := chanbuf(c, c.recvx)
        if raceenabled {
    
    
            racenotify(c, c.recvx, nil)
        }
        if ep != nil {
    
    
            typedmemmove(c.elemtype, ep, qp)
        }
        typedmemclr(c.elemtype, qp)
        c.recvx++
        if c.recvx == c.dataqsiz {
    
    
            c.recvx = 0
        }
        c.qcount--
        unlock(&c.lock)
        return true, true
    }

    // ...
    // 缓冲区为空，同时 sendq 里也没休眠的协程，则休眠等待
    gp := getg()
    mysg := acquireSudog()
    mysg.releasetime = 0
    if t0 != 0 {
    
    
        mysg.releasetime = -1
    }
    mysg.elem = ep
    mysg.waitlink = nil
    gp.waiting = mysg
    mysg.g = gp
    mysg.isSelect = false
    mysg.c = c
    gp.param = nil
    c.recvq.enqueue(mysg)

    // 休眠
    gp.parkingOnChan.Store(true)
    gopark(chanparkcommit, unsafe.Pointer(&c.lock), waitReasonChanReceive, traceEvGoBlockRecv, 2)

    // ...
}

It is woken up during the process of receiving data from the Channel, indicating that it was dormant and waiting because there was no data before. When the sender sends data, it will actively copy the data to the receiver's local.