read catalog
Channel
Channel is a mechanism in Go language for passing data between Goroutines.
Channel implements shared memory through communication, can safely transfer data, and avoids competition and deadlock problems that occur when multiple Goroutines access shared memory.
Channels can be buffered or unbuffered.
An unbuffered Channel, also known as a synchronous Channel, must be ready for sending and receiving operations at the same time, otherwise it will be blocked.
Buffered Channel, also known as asynchronous Channel, the sending operation will return immediately if the Channel buffer is not full, and the receiving operation will return immediately if the Channel buffer is not empty, otherwise it will be blocked.
Define Channels
package main
import (
"fmt"
"time"
)
/*
定义 channel,
channel 是带有类型的管道，
可以通过信道操作符 <- 来发送或者接收值
*/
func main() {
// 信道在使用前必须通过内建函数 make 来创建
/*
make(chan T,size)
标识用内建函数 make 来创建
一个T类型的缓冲大小为 size 的 channel
*/
/*
如下: make(chan int) 用内建函数 make
来创建 一个 int 类型的缓冲大小为 0 的 channel
*/
c := make(chan int)
go func() {
// 从 c 接收值并赋予 num
num := <-c
fmt.Printf("recover:%d\n", num)
}()
// 将 1 发送至信道 c
c <- 1
<-time.After(time.Second * 3)
fmt.Println("return")
}
root@192:~/www/test# go run main.go
recover:1
return
root@192:~/www/test#
Firstly, an unbuffered Channel c of int type is created through the make function, namely: c := make(chan int).
Then an anonymous Goroutine is defined by the go keyword to receive data from Channel c.
In the anonymous Goroutine, use the <- syntax to receive the value from Channel c and assign it to the variable num. After receiving the value, use fmt.Printf to print out the received value.
Next, in the main function, use the <- syntax to send the integer value 1 to Channel c, ie: c <- 1.
Finally, in order to ensure that Goroutine has enough time to receive the value in Channel, after waiting for 3 seconds by <-time.After(time.Second * 3), print out "return". If <-time.After(time.Second * 3) is removed, the program may end before printing "return", because the Goroutine does not have enough time to receive the value in the Channel.
Unbuffered Channel
An unbuffered Channel is defined by:
make(chan T)
In an unbuffered Channel, send and receive operations are synchronous.
If a Goroutine sends data to an unbuffered Channel, it will block until another Goroutine receives data from the Channel.
Likewise, if a Goroutine receives data from an unbuffered Channel, it will block until another Goroutine sends data to the Channel.
package main
import (
"fmt"
"time"
)
// 发送端和接收端的阻塞问题
/*
发送端在没有准备好之前会阻塞,
同样接收端在发送端没有准备好之前会阻塞
*/
func main() {
c := make(chan string)
go func() {
<-time.After(time.Second * 10)
fmt.Println("发送端准备好了 send: ping")
c <- "ping" // 发送
}()
// 发送端10s后才准备好，所以阻塞在当前位置
fmt.Println("阻塞在当前位置，发送端发送数据后才继续执行")
num := <-c
fmt.Printf("recover: %s\n", num)
}
The above code creates an unbuffered Channel c of string type, and then starts a new Goroutine, which will send a string "ping" to Channel c after 10 seconds.
In the main main, the receive operation <- c will block until a value is received from Channel c.
Because the sender takes 10 seconds to send data, the receiver blocks at <-c for 10 seconds.
After receiving "ping", the main main continues to execute and outputs "recover: ping".
root@192:~/www/test# go run main.go
阻塞在当前位置，发送端发送数据后才继续执行
发送端准备好了 send: ping
recover: ping
root@192:~/www/test#
Calculate the sum of arrays through goroutine+channel
package main
import "fmt"
/*
对切片中的数进行求和，将任务分配给两个 Go 程。
一旦两个 Go 协程完成了它们的计算，它就能算出最终的结果。
*/
// sum 求和函数
func sum(s []int, c chan int) {
ans := 0
for _, v := range s {
ans += v
}
c <- ans // 将和送入 c
}
func main() {
s := []int{
1, 1, 1, 1, 1, 2, 2, 2, 2, 2}
c := make(chan int)
go sum(s[:len(s)/2], c)
go sum(s[len(s)/2:], c)
x, y := <-c, <-c // 从 c 中接收
fmt.Println(x, y, x+y)
}
root@192:~/www/test# go run main.go
10 5 15
root@192:~/www/test#
Buffer Channel
Buffer channel definition:make(chan T,size)
A buffered Channel is a Channel with a buffer, and the size of the buffer needs to be specified when creating it. For example, an integer Channel with a buffer size of 10 make(chan int, 10)is created .
In a buffered Channel, when the buffer is not full, the sending operation is non-blocking. If the buffer is full, the sending operation will block until a receiving operation receives a value before continuing to send.
Receive operations are non-blocking when the buffer is not empty, and block if the buffer is empty until a send operation sends a value.
package main
import (
"fmt"
"time"
)
func producer(c chan int, n int) {
for i := 0; i < n; i++ {
c <- i
fmt.Printf("producer sent: %d\n", i)
}
close(c)
}
func consumer(c chan int) {
for {
num, ok := <-c
if !ok {
fmt.Println("consumer closed")
return
}
fmt.Printf("consumer received: %d\n", num)
}
}
func main() {
c := make(chan int, 5)
go producer(c, 10)
go consumer(c)
time.Sleep(time.Second * 1)
fmt.Println("main exited")
}
root@192:~/www/test# go run main.go
producer sent: 0
producer sent: 1
producer sent: 2
producer sent: 3
producer sent: 4
producer sent: 5
consumer received: 0
consumer received: 1
consumer received: 2
consumer received: 3
consumer received: 4
consumer received: 5
consumer received: 6
producer sent: 6
producer sent: 7
producer sent: 8
producer sent: 9
consumer received: 7
consumer received: 8
consumer received: 9
consumer closed
main exited
root@192:~/www/test#
In the above code, we created an integer Channel with a buffer size of 5, and the producer sent 10 integers to the Channel,
The consumer receives these integers from the Channel and prints them out.
Since the buffer size is 5, the producer will only block if there are 5 or fewer elements in the Channel.
In this example, since the consumer is receiving elements from the Channel faster than the producer can send them, the producer will eventually block until the consumer has received all the elements and closes the Channel.
It should be noted that when the Channel is closed, the remaining elements can still be received from the Channel,
but no elements can be sent to the Channel.
So, in the consumer function, we use a for loop and the ok flag to check if the Channel has been closed.
Comparison of unbuffered channel and buffered channel
package main
import "fmt"
// 不带缓冲的 channel
func NoBufferChan() {
ch := make(chan int)
ch <- 1
/*
被阻塞,
执行报错 fatal error: all goroutines are asleep - deadlock!
*/
fmt.Println(<-ch)
}
// 带缓冲的 channel
func BufferChan() {
// channel 有缓冲、是非阻塞的，直到写满 cap 个元素后才阻塞
ch := make(chan int, 1)
ch <- 1
fmt.Println(<-ch)
}
func main() {
NoBufferChan()
// BufferChan()
}
NoBufferChan()An error is reported because the send operation and the receive operation must occur in different goroutines at the same time, otherwise the program will deadlock.
Because unbuffered channels are synchronous, if a send operation is performed without a receiver, the sender will wait until there is a receiver.
If there are no receivers, the sender will block until the program crashes.
Therefore, we need to ensure that there are receivers waiting to receive these values before sending them, or use a buffered channel so that the sender will not be blocked forever waiting for the receiver.
close channel
The Close function can be used to close the Channel. After closing a channel, you can read data from it, but the read data is all zero value of the current channel type, but you cannot write data to this channel and a panic will be sent.
package main
func main() {
ch := make(chan bool)
close(ch)
fmt.Println(<- ch)
//ch <- true // panic: send on closed channel
}
| operate | A nil channel with zero value | a non-zero but closed channel | a non-zero channel that has not been closed |
|---|---|---|---|
| closure | generate panic | generate panic | closed successfully |
| send data | permanently blocked | generate panic | block or send successfully |
| Receive data | permanently blocked | never block | blocked or successfully received |
Traversing Channels
You can continue to read the channel through the range until the channel is closed.
package main
import (
"fmt"
"time"
)
// 通过 range 遍历 channel, 并通过关闭 channel 来退出循环
/*
复制一个 channel 或用于函数参数传递时,
只是拷贝了一个 channel 的引用,
因此调用者和被调用者将引用同一个channel对象。
*/
func genNum(c chan int) {
for i := 0; i < 10; i++ {
c <- i
time.Sleep(1 * time.Second)
}
// 发送者可通过 close 关闭一个信道来表示没有需要发送的值了
close(c)
}
func main() {
c := make(chan int, 10)
go genNum(c)
/*
循环 for v := range c 会不断从信道接收值，直到它被关闭。
并且只有发送者才能关闭信道，而接收者不能,
向一个已经关闭的信道发送数据会引发程序恐慌（panic）
*/
for v := range c {
fmt.Println("receive:", v)
}
/*
接收者可以通过 v,ok := <- c 表达式接收第二个参数来测试信道是否被关闭：
若没有值可以接收且信道已被关闭,那么 v 为对应类型零值,ok 为 false
*/
v, ok := <-c
fmt.Printf("value:%d, ok:%t\n", v, ok)
fmt.Println("close")
}
Operate the channel by select
Through select-case, you can select a ready data channel for execution, and read or write data from this channel.
package main
import (
"fmt"
"time"
)
// 通过 channel+select 控制 goroutine 退出
func genNum(c, quit chan int) {
for i := 0; ; i++ {
/*
select 可以等待多个通信操作
select 会阻塞等待可执行分支。
当多个分支都准备好时会随机选择一个执行。
*/
select {
case <-quit:
// 发送者可通过 close 关闭一个信道来表示没有需要发送的值了。
close(c)
return
default:
/*等同于 switch 的 default。
当所有case都阻塞时如果有default则，
执行default*/
c <- i
time.Sleep(1 * time.Second)
}
}
}
func main() {
c := make(chan int)
quit := make(chan int)
go genNum(c, quit)
/*循环 for v := range c 会不断从信道接收值，直到它被关闭。
并且只有发送者才能关闭信道，而接收者不能。
向一个已经关闭的信道发送数据会引发程序恐慌（panic）。*/
for i := 0; i < 10; i++ {
fmt.Println("receive:", <-c)
}
// 通知 genNum() 退出
quit <- 1
/*
接收者可以通过 v,ok := <- c 表达式第二个参数来测试信道是否被关闭：
若没有值可以接收且信道已被关闭，那么在执行完。
*/
v, ok := <-c
fmt.Printf("value:%d, ok:%t\n", v, ok)
fmt.Println("close")
}
root@192:~/www/test# go run main.go
receive: 0
receive: 1
receive: 2
receive: 3
receive: 4
receive: 5
receive: 6
receive: 7
receive: 8
receive: 9
value:0, ok:false
close
root@192:~/www/test#
goroutine+channel counts the number of each word in a text file
package main
import (
"bufio"
"fmt"
"log"
"os"
"strings"
)
func main() {
// 打开文件
file, err := os.Open("test.txt")
if err != nil {
log.Fatal(err)
}
defer file.Close()
// 创建一个 map，用于存储每个单词的数量
wordCount := make(map[string]int)
// 创建一个 channel，用于在不同的 goroutine 中传递数据
words := make(chan string)
// 启动一个 goroutine 读取文件并将每个单词发送到 channel 中
go func() {
/*创建一个新的 Scanner 对象，用来读取文件 file。
Scanner 对象提供了一种方便的方式来逐行或逐词读取文件内容。
需要注意的是，file 参数需要是一个 *os.File 类型的文件对象，
该对象可以通过 os.Open 或 os.Create 等函数创建。*/
scanner := bufio.NewScanner(file)
/*设置 Scanner 对象的分割函数，这里我们使用 bufio.ScanWords 函数来实现按照单词进行分割。
具体来说，bufio.ScanWords 函数会将每个连续的非空白字符序列作为一个单词，
并将其作为一个字符串返回。在本例中，我们将使用 ScanWords 来逐个读取文件中的单词。*/
scanner.Split(bufio.ScanWords)
/*读取文件中的下一个单词或行，并将其保存在 Scanner 对象的缓冲区中。
如果成功读取到下一个单词或行，则返回 true，否则返回 false。
可以通过调用 scanner.Text() 方法来获取缓冲区中最后一次读取的内容。*/
for scanner.Scan() {
word := strings.ToLower(scanner.Text())
words <- word
}
close(words) // 关闭 channel
}()
// 启动多个 goroutine 统计单词数量
for i := 0; i < 3; i++ {
go func() {
for word := range words {
wordCount[word]++
}
}()
}
// 等待所有的 goroutine 完成并输出结果
for i := 0; i < 3; i++ {
<-words // 等待所有的单词被处理完
}
fmt.Println(wordCount)
}
map[again.:1 away:1 brown:3 dog:1 dog.:2 fox:3 from:1 jumps:2 lazy:3 over:2 quick:3 runs:1 the:5]
root@192:~/www/test# go run main.go
map[again.:1 away:1 brown:3 dog:1 dog.:2 fox:3 from:1 jumps:2 lazy:3 over:2 quick:3 runs:1 the:5]
root@192:~/www/test#
In this sample code, we first open the text file and create a map to store the count of each word.
Then, we create a channel for passing data between different goroutines.
Next, we start a goroutine that reads the file and sends each word to the channel.
In this goroutine, we use the Scanner type from the bufio package to split the file by word and convert each word to lowercase before sending it to the channel.
Finally, we close the channel.
Next, we start multiple goroutines to count the number of words.
In these goroutines we receive each word from the channel using the range keyword and store it in a map. Since a map is a concurrency-safe data structure in Go, multiple goroutines can update it simultaneously without data races.
Finally, we wait in the main goroutine for all goroutines to complete and output their results.
In this sample code, we use a for loop to wait for all words to be processed.
Since each goroutine receives a value from the channel, we need to wait for all goroutines to receive the same number of values from the channel to ensure that all words are processed.
Finally, we output the count of each word.
test.txt
The quick brown fox jumps over the lazy dog.
The quick brown fox jumps over the lazy dog again.
The quick brown fox runs away from the lazy dog.
In the Go language, map is a data structure that can be read and written in multiple goroutines at the same time, and does not require additional synchronization operations to ensure concurrency safety. Specifically, when multiple goroutines read and write the same map concurrently, the map will automatically handle the issue of concurrent access to ensure that there will be no concurrent security issues such as data competition.
In terms of implementation, map uses some complex technologies to achieve concurrency security, such as segment locks, concurrent copy-on-write, and so on. These technologies can effectively reduce the granularity of locks, improve the efficiency of concurrent access, and at the same time ensure the correctness of concurrent access.
It should be noted that although map is concurrently safe in implementation, some details still need to be paid attention to in actual use.
For example, when multiple goroutines read and write the same map at the same time, conflicts may occur, and appropriate synchronization operations are required, otherwise problems such as data errors or race conditions will result.
Solve conflicts when reading and writing the same map in goroutine at the same time
package main
import (
"bufio"
"fmt"
"os"
"sync"
)
func countWords(filename string, wg *sync.WaitGroup, mu *sync.Mutex, wordCountMap map[string]int) {
defer wg.Done()
file, err := os.Open(filename)
if err != nil {
fmt.Println("error opening file:", err)
return
}
defer file.Close()
scanner := bufio.NewScanner(file)
scanner.Split(bufio.ScanWords)
for scanner.Scan() {
word := scanner.Text()
mu.Lock()
wordCountMap[word]++
mu.Unlock()
}
}
func main() {
/*sync.WaitGroup 等待一组 goroutine 的执行完成。
它提供了三个方法：
Add(delta int)：WaitGroup 的计数器。
Done()：将 WaitGroup 的计数器减去 1。
Wait()：阻塞当前 goroutine，直到 WaitGroup 的计数器归零。
*/
var wg sync.WaitGroup
var mu sync.Mutex
wordCountMap := make(map[string]int)
for _, filename := range os.Args[1:] {
wg.Add(1)
go countWords(filename, &wg, &mu, wordCountMap)
}
/*主 goroutine 中调用 Add 来设置要等待的 goroutine 数量，
然后在每个子 goroutine 中调用 Done 来表示它已经执行完成了。
最后，主 goroutine 调用 Wait 来等待所有子 goroutine 执行完成。
*/
wg.Wait()
fmt.Println("word count:")
for word, count := range wordCountMap {
fmt.Printf("%s: %d\n", word, count)
}
}
root@192:~/www/test# go run main.go test.txt
word count:
The: 3
jumps: 2
away: 1
from: 1
quick: 3
over: 2
the: 3
lazy: 3
dog.: 2
dog: 1
runs: 1
brown: 3
fox: 3
again.: 1
root@192:~/www/test#
sync.MutexIt is a concurrency primitive in the Go language, which is used to protect the access of shared variables and avoid data competition caused by simultaneous access by multiple goroutines.
In Go language, multiple goroutines may access a shared variable at the same time, which causes data race problem.
To avoid this problem, we can use sync.Mutex to protect access to shared variables.
sync.Mutex provides two methods:
- Lock(): Lock Mutex to prevent other goroutines from accessing protected resources.
- Unlock(): Unlocks the Mutex, allowing other goroutines to access protected resources.
When a goroutine acquires the lock on the Mutex, other goroutines will be blocked until the goroutine releases the lock.
This ensures that only one goroutine can access the protected resource at a time.
When using sync.Mutex, you need to pay attention to the following points:
1. Before each access to the shared variable, the lock of the Mutex must be acquired.
2. After the access is completed, the lock of the Mutex must be released immediately to allow other goroutines to access the shared variable.
3. Do not perform time-consuming operations while holding a Mutex lock, because this will cause other goroutines to be blocked.
4. To sum up, sync.Mutex is an important concurrency primitive in Go language, which can effectively protect access to shared variables and avoid data competition problems.