Go with concurrent coroutines understand

Coroutine

Go language to create a coroutine very simple to use gokeywords to make a common method of coroutine:

package main

import (
    "fmt"
    "time"
)

func main(){
    fmt.Println("run in main coroutine.")

    for i:=0; i<10; i++ {
        go func(i int) {
            fmt.Printf("run in child coroutine %d.\n", i)
        }(i)
    }

    //防止子协程还没有结束主协程就退出了
    time.Sleep(time.Second * 1)
}

The following concepts may not be well understood and need to slowly understand. You can skip, go back and look.

concept:

协程It may be understood as user thread pure state, which is to be switched through collaboration not seize. Lower with respect to processes or threads, coroutine all operations can be completed in a user mode, and switched to create consumption.
Inside a process can run multiple threads, and each thread and can run many coroutines. Thread is responsible for co-scheduling process, to ensure that each has the opportunity to receive the coroutine execution. When a coroutine sleep, you want to run it right to give other threads coroutine to run, but can not continue to occupy this thread. The same thread inside at most, only a coroutine is running.
Coroutine can be simplified into three states: 运行态, 就绪态and 休眠态. There will be only up to a state of coroutine is running the same thread. 就绪态协程Are those that have the ability to run but has not been running coroutine chance, they will be scheduled to run at any time state; 休眠态的协程sleep time do not have the ability to run, they are waiting for the occurrence of certain conditions, such as the completion of IO operations end and so on.
Sub coroutine abnormal abnormal exit will spread to the main coroutine, will lead directly to the main coroutine followed hang.

Coroutine general use TCP / HTTP / RPC services, message push system, the chat system. Use coroutines, we can easily set up a TCP or HTTP server supports high concurrency.

aisle

English channels are Channels, short chan. When the channel to use it? It can first be simply interpreted as: 协程when needed cooperative communication will need to use the channel.

In GO, there are two ways of communication between different parallel coroutine, is through shared variables, the other is through the passage. Go encouraging the use of language in the form of channels to communicate.

As a simple example, we use the coroutine achieve concurrent calls a remote interface, and ultimately we need to come back each coroutine request to return data aggregated together, this time on the channel used.

Create a channel

Creating 通道(channel) can only use the makefunction:

c := make(chan int)

通道It is to distinguish between types, such as here int.

Go read-write channel design language is a special arrow syntactic sugar <-, so we are very convenient to use the channel. The arrow on the right channel variable write is to write channel, write the arrow on the left channel is read channel. You can only read one element.

c := make(chan bool)
c <- true //写入
<- c //读取

Buffer channel

Above us the non-default cache type of channel, but Go also allows the buffer size specified channel, is very simple, that is how many elements can be stored channel:

c := make(chan int, value)

When value = 0, the 通道unbuffered blocking read and write, it is equivalent to make(chan int); when value > 0, the 通道buffer is non-blocking, until it fills up valueelement does not block write. Specific instructions below:

Unbuffered channel
either transmit operation or receive operation, beginning execution will be blocked until the pairing operation is started will continue to pass. Thus, the unbuffered data transfer channel in a synchronous manner. That is, only the sender and receiver on the docking, the data will be delivered. The data are copied directly from the sender to the recipient, and will not make the intermediate transfer channels with non-buffered.

Buffer channel
buffer channel can be understood as message queues, when there is capacity, transmission and reception are not interdependent. Transmitting data asynchronously.

Let's use an example to understand this:

package main

import "fmt"

func main() {
    var c = make(chan int, 0)
    var a string

    go func() {
        a = "hello world"
        <-c
    }()

    c <- 0
    fmt.Println(a)
}

This example output must be hello world. But if you put the capacity of the channel is changed from 0 number greater than 0, the output is not necessarily hello world, and is likely to be empty. why?

When the channel is unbuffered channel, to execute c <- 0, the channel is full, the write operation will be blocked to live there until <-cunblocked, then the statement following the execution.

If changed to a non-blocking channels, to perform c <- 0, but also found written, the main coroutine will not be blocked, but when the output is an empty string or hello world, depending on the child Coroutine is fast and the main coroutine which the speed of operation .

Channel as the container, it can be like a slice, the use cap()and the len()number of elements in the overall capacity and function to obtain the current channel inside.

Analog Message Queuing

Examples on a "coroutine" is, we had a Riga in the main coroutine time.Sleep(), aimed at preventing child is not over coroutine main coroutine exits. But for most scenes of real life, the one seconds is not enough, and most of the time we can not predict the length of the inner for loop code running time. You can not use this time time.Sleep()to wait for the completion of the operation. Here we use channels to rewrite:

package main

import (
    "fmt"
)

func main() {
    fmt.Println("run in main coroutine.")

    count := 10
    c := make(chan bool, count)

    for i := 0; i < count; i++ {
        go func(i int) {
            fmt.Printf("run in child coroutine %d.\n", i)
            c <- true
        }(i)
    }

    for i := 0; i < count; i++ {
        <-c
    }
}

Unidirectional channel

The default channel is to support reading and writing, we can define a one-way channel:

//只读
var readOnlyChannel = make(<-chan int)

//只写
var writeOnlyChannel = make(chan<- int)

Here is an example, we simulate the consumer message queue, the producer:

package main

import (
    "fmt"
    "time"
)

func Producer(c chan<- int) {
    for i := 0; i < 10; i++ {
        c <- i
    }
}

func Consumer1(c <-chan int) {
    for m := range c {
        fmt.Printf("oh, I get luckly num: %v\n", m)
    }
}

func Consumer2(c <-chan int) {
    for m := range c {
        fmt.Printf("oh, I get luckly num too: %v\n", m)
    }
}

func main() {
    c := make(chan int, 2)

    go Consumer1(c)
    go Consumer2(c)

    Producer(c)

    time.Sleep(time.Second)
}

For producers, we hope that the channel is write-only property, and for consumers it is a read-only attribute, so avoid channel errors. Of course, if you will in this case the consumer, the producer of the channel is also possible to remove the one-way property, no problem:

func Producer(c chan int) {}
func Consumer1(c chan int) {}
func Consumer2(c chan int) {}

Indeed channelread-only or write-only no sense, so-called one-way channelis only when the method used in the statement, if the follow-up code, originally used to read channeldata is written, the compiler will prompt an error.

Close the channel

Read a passage has been closed immediately return channel type 零值, and the channel has been closed to write a will throw an exception. If the channel is an integer in the elements, the read operation can not return value to determine whether the channel is closed.

1, how to safely read channel, make sure to close the channel is not read 零值?
The answer is to use the for...rangesyntax. When the channel is empty, the loop obstruction; when the passage is closed, the cycle will stop. By stopping the cycle, we can say that the channel has been closed. Example:

package main

import "fmt"

func main() {
    var c = make(chan int, 3)

    //子协程写
    go func() {
        c <- 1
        close(c)
    }()

    //直接读取通道，存在不知道子协程是否已关闭的情况
    //fmt.Println(<-c)
    //fmt.Println(<-c)

    //主协程读取：使用for...range安全的读取
    for value := range c {
        fmt.Println(value)
    }
}

Output:

2, how to write secure channel, to ensure that no written closed channel?
Go language does not exist in a built-in function may determine whether the channel has been closed. The best way to ensure safe passage written by the write channel responsible for their own coroutine to close the channel, read channel coroutine not to go to close the channel.

However, this method can only be solved Write Once Read Many scenes. If you encounter case write-single read of have a problem: You can not know what other time of writing coroutines finished, then it can not determine when to close the channel. This time we have to use an additional channel dedicated to do this thing.

We can use the built-in sync.WaitGroup, which uses the count to wait for the completion of the specified event:

package main

import (
    "fmt"
    "sync"
    "time"
)

func main() {

    var ch = make(chan int, 8)

    //写协程
    var wg = new(sync.WaitGroup)

    for i := 1; i <= 4; i++ {
        wg.Add(1)
        go func(num int, ch chan int, wg *sync.WaitGroup) {
            defer wg.Done()
            ch <- num
            ch <- num * 10
        }(i, ch, wg)
    }

    //读
    go func(ch chan int) {
        for num := range ch {
            fmt.Println(num)
        }
    }(ch)

    //Wait阻塞等待所有的写通道协程结束,待计数值变成零，Wait才会返回
    wg.Wait()

    //安全的关闭通道
    close(ch)

    //防止读取通道的协程还没有关闭
    time.Sleep(time.Second)

    fmt.Println("finish")
}

Output:


3
30
2
20
1
10
4
40
finish

Multiple channel

Sometimes you encounter multiple producers, as long as there is a producer in place, consumers will be spending situation. You can use this time to go language provides selectstatements that can manage multiple channels simultaneously read and write, if all channels can not read and write, it blocked the whole, as long as there is a channel can read and write, it will continue . Example:

package main

import (
    "fmt"
    "time"
)

func main() {

    var ch1 = make(chan int)
    var ch2 = make(chan int)

    fmt.Println(time.Now().Format("15:04:05"))

    go func(ch chan int) {
        time.Sleep(time.Second)
        ch <- 1
    }(ch1)

    go func(ch chan int) {
        time.Sleep(time.Second * 2)
        ch <- 2
    }(ch2)

    for {
        select {
            case v := <-ch1:
                fmt.Println(time.Now().Format("15:04:05") + ":来自ch1:", v)
            case v := <-ch2:
                fmt.Println(time.Now().Format("15:04:05") + ":来自ch2:", v)
            //default:
                //fmt.Println("channel is empty !")
        }
    }
}

Output:

13:39:56
13:39:57:来自ch1: 1
13:39:58:来自ch2: 2
fatal error: all goroutines are asleep - deadlock!

The default selectstate of congestion, after 1s, 1 child coroutine writing is completed, the main data are read out coroutine; then 2 sub coroutine writing is completed, the main data are read out coroutine; coroutine then hang the main reasons the main coroutine to find a never waiting for the data, which is obviously not the result, simply direct quit.

If the comments section open, the program after printing out the data from ch1, ch2, and would have been executed defaultinside the program. This time the program will not quit. The reason is that when selectthe time for all channels have not read the statement, if the definition of defaulta branch, it will perform the defaultbranching logic.

Note: select{}is without any caseof selectit would have been blocked.

Chan application scenarios

golang chan application scenarios are summarized in Issue # 9 · nange · / Blog
https://github.com/nange/blog/issues/9

Channels Go language of the practical application of | canoeのwhite sail
https://www.s0nnet.com/archives/go-channels-practice

message queue
Concurrent Requests
Analog lock function
Analog sync.WaitGroup
Parallel Computing

Channel principle part can reference links at the end of the text given 《快学 Go 语言》第 12 课 —— 通道to see.

Concurrent lock

The mutex

go where the language mapis not thread-safe:

package main

import "fmt"

func write(d map[string]string) {
    d["name"] = "yujc"
}

func read(d map[string]string) {
    fmt.Println(d["name"])
}

func main() {
    d := map[string]string{}
    go read(d)
    write(d)
}

Go language data structure built 竞态检查tools to help if there is a thread unsafe code check our program, just in time to run plus the -raceparameters can be:

$ go run -race main.go 
==================
WARNING: DATA RACE
Read at 0x00c0000a8180 by goroutine 6:

...

yujc
Found 2 data race(s)
exit status 66

As can be seen, the presence of the above code security risks.

We can use sync.Mutexto protect mapthe principle is to use a mutex to protect before each read and write operations, to prevent other threads operate simultaneously:

package main

import (
    "fmt"
    "sync"
)

type SafeDict struct {
    data map[string]string
    mux  *sync.Mutex
}

func NewSafeDict(data map[string]string) *SafeDict {
    return &SafeDict{
        data: data,
        mux:  &sync.Mutex{},
    }
}

func (d *SafeDict) Get(key string) string {
    d.mux.Lock()
    defer d.mux.Unlock()
    return d.data[key]
}

func (d *SafeDict) Set(key string, value string) {
    d.mux.Lock()
    defer d.mux.Unlock()
    d.data[key] = value
}

func main(){
    dict := NewSafeDict(map[string]string{})

    go func(dict *SafeDict) {
        fmt.Println(dict.Get("name"))
    }(dict)

    dict.Set("name", "yujc")
}

Run the test:

$ go run -race main.go 
yujc

If you do not use the above code -raceto run, it would achieve good results, depending on the main coroutine sub coroutine which to run.

Note: sync.Mutexis a structure of the object, the object during use to avoid shallow copy, or no protection. It should be possible to use the pointer type.

The above code we use the many d.mux.Lock(), whether simplified into d.Lock()it? The answer is yes. We know that the structure can be automatically inherits all the methods anonymous internal structure:

type SafeDict struct {
    data map[string]string
    *sync.Mutex
}

func NewSafeDict(data map[string]string) *SafeDict {
    return &SafeDict{data, &sync.Mutex{}}
}

func (d *SafeDict) Get(key string) string {
    d.Lock()
    defer d.Unlock()
    return d.data[key]
}

This completes the simplified.

Read-Write Lock

For reading and writing little scenes, can be used 读写锁instead of 互斥锁, you can improve performance.

Write lock provides the following four methods:

Lock() Write lock
Unlock() Write lock release
RLock() Read lock
RUnlock() Read lock release

写锁Is 排它锁plus 写锁clog when coupled with other Coroutine 读锁and 写锁; 读锁is 共享锁, plus read locks may also allow other coroutine together 读锁, but will increase obstruction 写锁. 读写锁In the case of concurrent high write performance degradation as normal 互斥锁.

We mutex on the section into read-write locks:

package main

import (
    "fmt"
    "sync"
)

type SafeDict struct {
    data map[string]string
    *sync.RWMutex
}

func NewSafeDict(data map[string]string) *SafeDict {
    return &SafeDict{data, &sync.RWMutex{}}
}

func (d *SafeDict) Get(key string) string {
    d.RLock()
    defer d.RUnlock()
    return d.data[key]
}

func (d *SafeDict) Set(key string, value string) {
    d.Lock()
    defer d.Unlock()
    d.data[key] = value
}

func main(){
    dict := NewSafeDict(map[string]string{})

    go func(dict *SafeDict) {
        fmt.Println(dict.Get("name"))
    }(dict)

    dict.Set("name", "yujc")
}

After the change, using the detection means detects a race or pass the.

reference

Difference 1, make (chan int) and make (chan int, 1) of
https://www.jianshu.com/p/f12e1766c19f
2, Channel
https://www.jianshu.com/p/4d97dc032730
. 3, "Fast Go learn the language "Lesson 12 - channel
https://mp.weixin.qq.com/s?__biz=MzI0MzQyMTYzOQ==&mid=2247484601&idx=1&sn=97c0de2acc3127c9e913b6338fa65737
4," Go quickly learn the language "Lesson 13 - Simultaneous and security
https://mp.weixin.qq.com/s?__biz=MzI0MzQyMTYzOQ==&mid=2247484683&idx=1&sn=966cb818f034ffd4538eae7a61cd0c58