Run the program go to goroutine as a unit, and goroutine actual operating within a system thread. goroutine (very much) and system threads (relatively small) is not one to one. When scheduling, both os thread scheduling, there go the scheduler itself scheduling goroutine. In short, go native support for concurrency, go scheduler is responsible for scheduling each goroutine to a different operating system threads get executed.
Function is executed by starting a coroutine go Keywords:
go func()
Three definitions:
- G: goroutine, go is usually mentioned coroutine
- P: process, processor, some articles say on behalf of the context, can be understood as a token comes with context information
- M: thread machine, thread, operating system, which is usually referred to in
Go early model is GM, and later because of performance issues in favor of the use of GPM model.
Implementation mechanisms:
- M bound P, can continue to run G, preemptive scheduling (different executable) (by sysmon)
- P G has its own queue (no lock access, fast); at the same time, the program has a global queue G
- M perform some system calls, it may be unbound and P; M may be dormant
M, P, G three different number, M default 10000 (SetMaxThreads change, generally do not), P is the default machine CPU core number (designated by GOMAXPROCS), G does not explicitly limit (created through go command).
GM to GPM
Early, GM model has many problems, such as global lock, M cache memory footprint and waste, see "Scalable Go Scheduler Design".
More simply, G to run, a need to bind P (where P is placed in the local queue), and then actually executed by the operating system threads bind P M.
When G switching, but cut from G1 to G2 M only, is performed in the user mode, very lightweight, unlike heavy operating system handover M.
P local queue in the absence of G, will "steal" from the queue's number or other P taken from the global queue.
When the aid netpoller, the network initiates calls, blocking G, M is not blocked, G can be switched. And when initiating file IO and other operations, will perform (blocking) system call, (Note: You should now realize some of the poller for os package), this time M will be waiting to return to the system call. G and M together, will unbind to P. If there are other local queue of G P, another will bind idle M, and if not, a new M, and then continue to be performed G.
A preemptive scheduler implementation
This means that if a too long execution G, will be switched out.
This ensures that the entire program appears to be "concurrent" execution, rather than when a G can perform is to keep the implementation of the other G are starving to death.
But switching point needs to be a function call. Suppose G is not a pure infinite loop transfer function calculation, or can not be preempted.
When G is scheduled:
- It is set to seize sysmon
- channel blocked or network IO
- mutex and other synchronization causes obstruction
- Use keywords go create goroutine
- GC process of scheduling a variety of strategies caused
- runtime, the realization of network IO uses kqueue (MacOS), epoll (Linux) or iocp (Windows).
View various scheduling status:
When executing the command, set the environment variable GODEBUG. For example: GODEBUG = schedtrace = 1000, scheddetail = 1 godoc -http =: 6060
P benefits have local queue
In fact, the benefits of good points. Obvious that, GM model which, when the switch M G, from the need to take global queue inside need to lock. GPM, M is bound with P, M are switched in the local G G P of the queue, no lock.
P is the default machine logic auditor
Because the presence of Hyper-Threading Technology, the number of logical core will be different from the physical number of cores. The following statement can print out the number of logical cores, through GOMAXPROCS setting can not mistake.
fmt.Println(runtime.NumCPU())
M defaults 10000
M corresponds sched.maxmcount, default 10000. Can be modified by SetMaxThreads, if the program uses more than this number, it will automatically crash!
// 改动时也会检查，并不能随意设置值
if in > 0x7fffffff { // MaxInt32
sched.maxmcount = 0x7fffffff
} else {
sched.maxmcount = int32(in)
}
goroutine abnormal capture
When multiple goroutine, if one goroutine anomaly, and we did not for exception handling, the entire program will be terminated, so we had better run each function goroutine have done programming time exception handling, exception treatment using recover.
package main
import (
"fmt"
"time"
)
func addele(a []int ,i int) {
defer func() { //匿名函数捕获错误
err := recover()
if err != nil {
fmt.Println("add ele fail")
}
}()
a[i]=i
fmt.Println(a)
}
func main() {
Arry := make([]int,4)
for i :=0 ; i<10 ;i++{
go addele(Arry,i)
}
time.Sleep(time.Second * 2)
}
Operating results as follows:
add ele fail
[0 0 0 0]
[0 1 0 0]
[0 1 2 0]
[0 1 2 3]
add ele fail
add ele fail
add ele fail
add ele fail
add ele fail
Synchronization goroutine
Since goroutine is asynchronous, it is likely to quit when the main program there goroutine not been performed, this time goroutine will follow the exit appear. At this point if you want to wait until the task is completed before all goroutine withdraw, go bag and provide a sync channel to solve the synchronization problem.
Use the sync packet synchronization goroutine:
package main
import (
"fmt"
"sync"
)
func cal(a int , b int ,n *sync.WaitGroup) {
c := a+b
fmt.Printf("%d + %d = %d\n",a,b,c)
defer n.Done() //goroutinue完成后, WaitGroup的计数-1
}
func main() {
var go_sync sync.WaitGroup //声明一个WaitGroup变量
for i :=0 ; i<10 ;i++{
go_sync.Add(1) // WaitGroup的计数加1
go cal(i,i+1,&go_sync)
}
go_sync.Wait() //等待所有goroutine执行完毕
}
The results are:
9 + 10 = 19
2 + 3 = 5
3 + 4 = 7
4 + 5 = 9
5 + 6 = 11
1 + 2 = 3
6 + 7 = 13
7 + 8 = 15
0 + 1 = 1
8 + 9 = 17
Synchronization between goroutine by channel:
Implementation: to communicate through the channel between multiple groutine, when a goroutine complete when sending a signal to exit the channel, and all goroutine quit when using the signal for the channel to loop channe, if we take the data will not clog principle, wait for all goroutine is finished, this method has a premise that you already know how many you start goroutine.
package main
import (
"fmt"
"time"
)
func cal(a int , b int ,Exitchan chan bool) {
c := a+b
fmt.Printf("%d + %d = %d\n",a,b,c)
time.Sleep(time.Second*2)
Exitchan <- true
}
func main() {
Exitchan := make(chan bool,10) //声明并分配管道内存
for i :=0 ; i<10 ;i++{
go cal(i,i+1,Exitchan)
}
for j :=0; j<10; j++{
<- Exitchan //取信号数据，如果取不到则会阻塞
}
close(Exitchan) // 关闭管道
}
Communication between goroutine
Goroutine is the essence coroutines, can be understood from the kernel scheduler, and go by the thread scheduler management. Or that can communicate through shared data channel between goroutine, of course, you can also use global variables to share data.
Example: Using channel analog consumer and producer mode
package main
import (
"fmt"
"sync"
)
func Productor(mychan chan int,data int,wait *sync.WaitGroup) {
mychan <- data
fmt.Println("product data：",data)
wait.Done()
}
func Consumer(mychan chan int,wait *sync.WaitGroup) {
a := <- mychan
fmt.Println("consumer data：",a)
wait.Done()
}
func main() {
datachan := make(chan int, 100) //通讯数据管道
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
go Productor(datachan, i,&wg) //生产数据
wg.Add(1)
}
for j := 0; j < 10; j++ {
go Consumer(datachan,&wg) //消费数据
wg.Add(1)
}
wg.Wait()
}
channel
commonly known as the pipeline channel, for data transfer or data sharing, which is essentially a FIFO queue, goroutine + channel using a simple and efficient data communication, but also the security thread, a plurality of channel goroutine may be modified simultaneously, no locking .
channel can be divided into three types:
-
Read-only channel: channel which can only read data unwritable
-
Write only channel: can only write data unreadable
-
General channel: readable and writable
channel using:
var readOnlyChan <-chan int // 只读chan
var writeOnlyChan chan<- int // 只写chan
var mychan chan int //读写channel
//定义完成以后需要make来分配内存空间，不然使用会deadlock
mychannel = make(chan int,10)
//或者
read_only := make (<-chan int,10)//定义只读的channel
write_only := make (chan<- int,10)//定义只写的channel
read_write := make (chan int,10)//可同时读写
Read and write data
have to be aware of is:
- If the pipeline is not closed, the reading-out will lead to deadlock will be an exception
- If the pipeline is closed for writing data will pannic
- When the pipe when no data is read or re-read the default value, such as an int default value is 0
ch <- "wd" //写数据
a := <- ch //读取数据
a, ok := <-ch //优雅的读取数据
Circulation pipe
have to be aware of is:
Use range circulation pipe, if the pipe is not closed will lead to deadlock error.
If an infinite loop has been closed for the pipeline, the pipeline when there is no data, the data is read is the default pipeline, and the cycle will not quit.
package main
import (
"fmt"
)
func main() {
mychannel := make(chan int,10)
for i := 0;i < 10;i++{
mychannel <- i
}
close(mychannel) //关闭管道
fmt.Println("data lenght: ",len(mychannel))
for v := range mychannel { //循环管道
fmt.Println(v)
}
fmt.Printf("data lenght: %d",len(mychannel))
}
The output is:
data lenght: 10
0
1
2
3
4
5
6
7
8
9
data lenght: 0
Buffer channe with and without buffer channel
With a buffer channel: define statement when formulated buffer size (length), you can save more data.
Without the buffer channel: a data storage only, and can be able to save a data only when the data is extracted time.
ch := make(chan int) //不带缓冲区
ch := make(chan int ,10) //带缓冲区
Without Buffer Example:
package main
import "fmt"
func test(c chan int) {
for i := 0; i < 10; i++ {
fmt.Println("send ", i)
c <- i
}
}
func main() {
ch := make(chan int)
go test(ch)
for j := 0; j < 10; j++ {
fmt.Println("get ", <-ch)
}
}
The output is:
send 0
send 1
get 0
get 1
send 2
send 3
get 2
get 3
send 4
send 5
get 4
get 5
send 6
send 7
get 6
get 7
send 8
send 9
get 8
get 9
channel for job pool
We create a three channel, a channel for accepting a task, a channel for holding a result, there is a channel used to determine when the program exits.
Read-only channel and write-only channel
General definition of read-only and write-only pipeline meaningful, more often we can pass parameters specified in the pipeline when read or write, even if the current pipeline is readable and writable.
package main
import (
"fmt"
"time"
)
//只能向chan里写数据
func send(c chan<- int) {
for i := 0; i < 10; i++ {
c <- i
}
}
//只能取channel中的数据
func get(c <-chan int) {
for i := range c {
fmt.Println(i)
}
}
func main() {
c := make(chan int)
go send(c)
go get(c)
time.Sleep(time.Second*1)
}
select-case non-blocking channel
Principle select + case by addition of a set of conduits, when satisfied (satisfied meant here that the data read or write) select in a case when it returns this case, if the case is not satisfied, then take the default branch .
package main
import (
"fmt"
)
func send(c chan int) {
for i :=1 ; i<10 ;i++ {
c <-i
fmt.Println("send data : ",i)
}
}
func main() {
resch := make(chan int,20)
strch := make(chan string,10)
go send(resch)
strch <- "wd"
select {
case a := <-resch:
fmt.Println("get data : ", a)
case b := <-strch:
fmt.Println("get data : ", b)
default:
fmt.Println("no channel actvie")
}
}
Operating results as follows:
get data : wd
package main
import (
"fmt"
"time"
)
func send(c chan int) {
for i :=1 ; i<10 ;i++ {
c <-i
fmt.Println("send data : ",i)
}
}
func main() {
resch := make(chan int,20)
strch := make(chan string,10)
go send(resch)
strch <- "wd"
time.Sleep(time.Second * 2)
select {
case a := <-resch:
fmt.Println("get data : ", a)
case b := <-strch:
fmt.Println("get data : ", b)
default:
fmt.Println("no channel actvie")
}
}
The above operating results as follows:
send data : 1
send data : 2
send data : 3
send data : 4
send data : 5
send data : 6
send data : 7
send data : 8
send data : 9
get data : 1
frequency control channel
When reading and writing of the channel, Go provides a very user-friendly operation, that is, the frequency of the read and write control, achieved by time.Ticke
package main
import (
"time"
"fmt"
)
func main(){
requests:= make(chan int ,5)
for i:=1;i<5;i++{
requests<-i
}
close(requests)
limiter := time.Tick(time.Second*1)
for req:=range requests{
<-limiter
fmt.Println("requets",req,time.Now()) //执行到这里，需要隔1秒才继续往下执行，time.Tick(timer)上面已定义
}
}
Operating results as follows:
requets 1 2020-04-02 02:19:58.089127 +0800 CST m=+1.002352626
requets 2 2020-04-02 02:19:59.087781 +0800 CST m=+2.000975368
requets 3 2020-04-02 02:20:00.092095 +0800 CST m=+3.005257639
requets 4 2020-04-02 02:20:01.096952 +0800 CST m=+4.010083443
