The Go language supports concurrency at the language level, which is very different from other languages. Unlike before, we used the Thread library to create new threads, and we also used the thread-safe queue library to share data.
Below are my study notes for getting started.
First of all, parallel != concurrent, the two are different, you can refer to: http://concur.rspace.googlecode.com/hg/talk/concur.html
Goroutines, Channels and Deadlocks in Go There is a concept in Go called
goroutine
, which is similar to the familiar thread, but lighter.
In the following program, we execute the loop function twice in series:
func loop() {
for i := 0; i < 10; i++ {
fmt.Printf("%d ", i)
}
}
func main() {
loop()
loop()
}No doubt the output would be something like this:
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
Below we put a loop to run in a goroutine, we can use the keyword go to define and start a goroutine:
func main() {
go loop() // 启动一个goroutine
loop()
}This time the output becomes:
0 1 2 3 4 5 6 7 8 9
But why only one output? Obviously we ran the main line, and we also opened a goroutine to run it.
It turns out that the main function has already exited before the goroutine has time to run the loop.
The main function exits too quickly. We have to find a way to prevent it from exiting prematurely. One way is to make main wait for a while:
func main() {
go loop()
loop()
time.Sleep(time.Second) // 停顿一秒
}This time, I did output twice, and the purpose was achieved.
But the method of waiting is not good. If the goroutine is at the end, it tells the main thread to say "Hey, I'm finished!" That's the so-called method of blocking the main thread. Recall that we wait for all threads in Python to finish executing. of writing:
for thread in threads:
thread.join()Yes, we also need a join-like thing to block the main line. that is the channel
Channel What is a
channel ? Simply put, it is something that goroutines communicate with each other. Similar to pipes on our Unix (which can pass messages between processes), it is used to send and receive messages between goroutines. In fact, it is doing memory sharing between goroutines.
Use make to create a channel:
var channel chan int = make(chan int)// or
channel := make(chan int)How to store and retrieve messages from the channel? one example:
func main() {
var messages chan string = make(chan string)
go func(message string) {
messages <- message // 存消息
}(“Ping!”)
fmt.Println(<-messages) // 取消息
}
By default, the channel's message storage and retrieval are blocked (called an unbuffered channel, but the concept of buffering will be understood later, let's talk about blocking first).
That is, an unbuffered channel suspends the current goroutine when fetching and storing messages, unless the other end is ready.
For example, the following main function and foo function:
var ch chan int = make(chan int)
func foo() {
ch <- 0 // 向ch中加数据，如果没有其他goroutine来取走这个数据，那么挂起foo, 直到main函数把0这个数据拿走
}
func main() {
go foo()
<- ch // 从ch取数据，如果ch中还没放数据，那就挂起main线，直到foo函数中放数据为止
}So since the channel can block the current goroutine, then go back to the problem encountered in the previous part of "goroutine" "how to let the goroutine tell the main line that I have finished executing", just use a channel to tell the main line:
var complete chan int = make(chan int)
func loop() {
for i := 0; i < 10; i++ {
fmt.Printf("%d ", i)
}
complete <- 0 // 执行完毕了，发个消息
}
func main() {
go loop()
<- complete // 直到线程跑完, 取到消息. main在此阻塞住
}If the main line is not blocked by the channel, the main line will run out prematurely, and the loop line will not have a chance to execute,,,
In fact, an unbuffered channel will never store data, and is only responsible for the flow of data. Why do you say this?
To get data from an unbuffered channel, there must be a data stream coming in, otherwise the current line is blocked
The data flows into the unbuffered channel, if there is no other goroutine to take this data, the current line is blocked
So, you can test that, in any case, the size of the unbuffered channel we tested is 0 (len(channel))
If there is data flowing in the channel, we still need to add data, or if the channel is dry, we have been fetching data from the empty channel with no data flowing? will cause deadlock
Deadlock
An example of a deadlock:
func main() {
ch := make(chan int)
<- ch // 阻塞main goroutine, 信道c被锁
}Execute this program and you will see an error like this in Go:
fatal error: all goroutines are asleep - deadlock!What is deadlock? The operating system has said that all threads or processes are waiting for the release of resources. In the above program, there is only one goroutine, so when you add data or store data to it, it will lock the channel and block the current goroutine, that is, all goroutines (actually the main line) are waiting for the channel to open ( No one takes away the data channel will not open), that is, deadlock.
I find deadlocks to be an interesting topic, here are a few examples of deadlocks:
Only operating an unbuffered channel in a single goroutine will definitely deadlock. For example, if you only operate on the channel in the main function:
func main() {
ch := make(chan int)
ch <- 1 // 1流入信道，堵塞当前线, 没人取走数据信道不会打开
fmt.Println("This line code wont run") //在此行执行之前Go就会报死锁
}The following is also an example of a deadlock:
var ch1 chan int = make(chan int)
var ch2 chan int = make(chan int)
func say(s string) {
fmt.Println(s)
ch1 <- <- ch2 // ch1 等待 ch2流出的数据
}
func main() {
go say("hello")
<- ch1 // 堵塞主线
}Among them, the main line waits for the data in ch1 to flow out, and ch1 waits for the data in ch2 to flow out, but ch2 waits for the data to flow in, and both goroutines are waiting, that is, deadlock.
In fact, in summary, why is there a deadlock? If there is no inflow and no outflow, or no inflow or outflow on a non-buffered channel, a deadlock will result. Or to understand that the non-buffered channels in all goroutines started by Go must store data in one line and fetch data in one line, and they must be paired. So the following example must deadlock:
c, quit := make(chan int), make(chan int)
go func() {
c <- 1 // c通道的数据没有被其他goroutine读取走，堵塞当前goroutine
quit <- 0 // quit始终没有办法写入数据
}()
<- quit // quit 等待数据的写If you analyze it carefully, it is because: the main line waits for the data to flow out of the quit channel, quit waits for the data to be written, and the func is blocked by the c channel, all goroutines are waiting, so deadlock.
In simple terms, there are two lines in total. The data flowing into the c channel in the func line does not flow out in the main line, and it must be deadlocked.
But is it true that all unpaired accesses to the channel are deadlocks?
Here is a counter example:
func main() {
c := make(chan int)
go func() {
c <- 1
}()
}The program exits normally. It's very simple. It's not that our summary doesn't work, or it's because of a very embarrassing reason that main did not wait for other goroutines and finished running first, so no data flowed into the c channel, and it was executed in total. A goroutine, and no blocking occurs, so no deadlock errors.
So what about deadlock solutions?
The easiest way is to take away the data that has not been taken, and put the data that has not been put in, because the unbuffered channel cannot carry data, so take it away quickly!
Specifically, in the case of deadlock example 3, the deadlock can be avoided as follows:
c, quit := make(chan int), make(chan int)
go func() {
c <- 1
quit <- 0
}()
<- c // 取走c的数据！
<-quitAnother solution is to buffer the channel, i.e. set c to have a buffer size of the data:
c := make(chan int, 1)In this case, c can cache a data. That is to say, put a piece of data, c will not suspend the current line, and put another one will suspend the current line until the first data is taken away by other goroutines, that is, it will only block when the capacity is certain, and it will not reach Capacity is not blocked.
This is very similar to the queue Queue in our Python, isn't it?
Data in and out sequence of unbuffered channels
We already know that unbuffered channels never store data, and the incoming data must flow out.
Observe the following program:
var ch chan int = make(chan int)
func foo(id int) { //id: 这个routine的标号
ch <- id
}
func main() {
// 开启5个routine
for i := 0; i < 5; i++ {
go foo(i)
}
// 取出信道中的数据
for i := 0; i < 5; i++ {
fmt.Print(<- ch)
}
}We opened 5 goroutines, and then fetched data in turn. In fact, if the entire execution process is subdivided, the data of the 5 lines flows through the channel ch in turn, and the main prints it. Macroscopically, the data of the unbuffered channel is first come first out, but the unbuffered channel is not stored. Data, only responsible for the circulation of data
Buffered channel
Finally , this topic has come, in fact, the buffered channel is more expressive in English: buffered channel.
The word buffering means that a buffered channel can not only flow data, but also buffer data. It has capacity. If you store a piece of data, you can put it in the channel first, without blocking the current line and waiting for the data to be taken away.
When the buffer channel reaches the full state, it will show blocking, because it can no longer carry more data at this time, "you must take the data away before you can flow in the data".
When declaring a channel, we give make a second argument to specify its capacity (default is 0, i.e. unbuffered):
var ch chan int = make(chan int, 2) // Writing 2 elements will not block the current goroutine, and it will block when the number of storage reaches 2. In the
following example, the buffer channel ch can flow into 3 elements without buffering :
func main() {
ch := make(chan int, 3)
ch <- 1
ch <- 2
ch <- 3
}If you try to flow in another data, the channel ch will block the main line and report a deadlock.
That is, the buffer channel will be locked when it is full.
In fact, the buffer channel is first-in, first-out, and we can think of the buffer channel as a thread-safe queue:
func main() {
ch := make(chan int, 3)
ch <- 1
ch <- 2
ch <- 3
fmt.Println(<-ch) // 1
fmt.Println(<-ch) // 2
fmt.Println(<-ch) // 3
}Channel data reading and channel closing
You may find that the above code is too laborious to read channels one by one. Go allows us to use ranges to read channels:
func main() {
ch := make(chan int, 3)
ch <- 1
ch <- 2
ch <- 3
for v := range ch {
fmt.Println(v)
}
}If you execute the above code, a deadlock error will be reported, because the range will not end reading until the channel is closed. That is, if the buffer channel dries up, then the range will block the current goroutine, so deadlock.
Then, we try to avoid this situation, and it is easier to think of ending the reading when the channel is empty:
ch := make(chan int, 3)
ch <- 1
ch <- 2
ch <- 3
for v := range ch {
fmt.Println(v)
if len(ch) <= 0 { // 如果现有数据量为0，跳出循环
break
}
}The above method can be output normally, but note that the method of checking the channel size cannot be used to fetch all data when channel access is occurring. In this example, we only store data in ch, and now we fetch them one by one. , the channel size is decreasing.
Another way is to close the channel explicitly:
ch := make(chan int, 3)
ch <- 1
ch <- 2
ch <- 3
// 显式地关闭信道
close(ch)
for v := range ch {
fmt.Println(v)
}A closed channel will prevent data from flowing in and is read-only. We can still fetch data from the closed channel, but we can no longer write data.
Waiting for a multi-gorountine
solution Well, let's go back to the original problem, using the channel to block the main line and waiting for all the goroutines to run out.
This is a model that opens up many small goroutines, each of them runs their own, and finally reports to the main line when they are finished.
We discuss the following 2 versions of the scheme:
Block the main line with only a single unbuffered channel
Use a buffered channel with capacity equal to the number of goroutines
For scenario 1, the example code would look like this:
var quit chan int // 只开一个信道
func foo(id int) {
fmt.Println(id)
quit <- 0 // ok, finished
}
func main() {
count := 1000
quit = make(chan int) // 无缓冲
for i := 0; i < count; i++ {
go foo(i)
}
for i := 0; i < count; i++ {
<- quit
}
}For option 2, change the channel to buffer 1000:
quit = make(chan int, count) // 容量1000In fact, the only difference is that one is buffered and the other is non-buffered.
For this scenario, both can accomplish the task, both are ok.
An unbuffered channel is a batch of data "flowing in and out" one by one
The buffer channel is stored one by one and then streamed out together