Problem: Find all primes between 1-20000
conventional method: each number to do 2 <= i <= n - 1 times modulo cycle, if it is 2000000000, the computer crying.
Golang method: Things sure the process is still the process, if a concurrent increase in a parallel manner, rather than the traditional cycle, so that efficiency will be much higher, but also meet the needs of distributed.
Solutions
using concurrent or parallel manner , so that the task assigned to multiple primes statistics goroutine to complete, this time on the introduction of coroutines goroutine program.
Calculated on the assumption 1-20000, according to the distribution into four equal parts, is significantly faster, in line with the CPU multicore style, distributed cluster.
Processes and Threads
- Learned linux people must know the whole relationship between the procedures, processes and threads, this is not to say, roughly make some notes
- A description of the program instructions, data and organizational form.
- Process is a process of program execution in the operating system , the system is the smallest unit of resource allocation and scheduling , in a process different from each other namespaces so independently of each other.
- Thread execution instance may be understood as a process, is the minimum unit of the operating system may execute a standard thread by the thread ID, the current instruction pointer, register set and stack components.
- The process is the thread of "container" ****, thread directly affect the process leading to the collapse of the collapse of the process , such as storm video, video and audio synchronization (concurrent) can be understood as threads collaboration.
Concurrent and Parallel
- Multithreaded programs running on a single core, is complicated (time division multiplexed) CPU in the order of milliseconds to switch between tasks, appears to be running together in a plurality of tasks, to distinguish between concurrent web, web concurrently run true.
- Multithreaded programs running on multi-core, parallel is , true multi-tasking at the same time, in terms of efficiency is definitely higher than the concurrent parallel efficiency.
Coroutine goroutine
- The main process is a physical process, directly on the CPU, heavyweight consumption of resources, each process has at least one main thread.
- Coroutine main Chengkai Qi from lightweight threads, the logic state, resource consumption small.
- Golang coroutine mechanisms are important features, easy to open thousands of coroutines.
- Go the main thread (to say somewhat different, in fact, can be understood as a process), can play a more coroutine go on a thread, the thread is lightweight coroutines (compiler to do the underlying optimization).
- Go coroutine features :
independent stack space
have shared heap space
scheduling in user space level
coroutine is a lightweight thread
goroutine use
- General functions are for use only in the main thread before the end of the implementation, add go <function name> () to complete.
- The main thread ends regardless of the task coroutine has not been executed, the final will be forced to shut down (the main thread stack space is closed, no coroutine to his home).
Case presentation:
package main
import (
"fmt"
"time"
)
func echo() {
for i := 0 ; i <= 10 ; i++ {
fmt.Println("echo() 我是练习时长两年半的xx")
time.Sleep(500 * time.Millisecond)
}
}
func main() {
/*
一般都是根据函数走的，只需要在前面添加 go <函数名>()
*/
go echo() //开启协程
for i := 0 ; i <= 10 ; i++ {
fmt.Println("main() 喜欢唱，跳，rap，篮球")
time.Sleep(time.Second)
}
//go echo() //这种开启协程方式无效，因为主函数已经结束了，充分理解原理
} //输出效果可以echo()和main()中是同时执行的(for循环表现为穿插)
MPG goroutine scheduling model
M: a kernel thread directly related KSM <-> M, a plurality of M is run concurrently on a single CPU, a plurality of M is run in parallel on multiple CPU.
P: communication between contexts, G and M of the bridge, P present runqueue, queue plurality G.
G: is coroutine Goroutines, lightweight threads, and the state is a logical, easy to Go from tens of thousands coroutine.
Compared to multi-threaded Java and C, are generally kernel mode, relatively heavyweight, each thread is about 8M, thousands of threads may run out of CPU;
and goroutine resource consumption is very low, probably * 2k each coroutine .
Running the model: MPG
- G P is a runqueue, P one by one in the processing queue pop manner of G.
- G queue if a large file is read coroutine cause obstruction, where M0 creates additional M, is assumed to be M1 (operating system dependent, thread pool), the P continue to mount the pop wherein M1 G; M0 continues to be blocked G0.
- MPG scheduling mode, only allows coroutine execution, will not let the queue of other coroutine has been blocked, you can still concurrent / parallel.
Set the number of active CPU Golang
package main
import (
"fmt"
"runtime"
)
func main() {
num := runtime.NumCPU() //显示当前系统逻辑CPU个数
fmt.Println(num)
fmt.Println(runtime.GOMAXPROCS(num - 1)) //设置可同时执行的CPU最大个数
// 这个在Golang1.8之后不用在设置了
}
Several problems Golang corresponding coroutine concurrent / parallel occurring
- How to solve concurrency / parallelism while the write operation problems.
- How to avoid the main program not wait until the situation coroutine executing the exit to complete the execution .
- How the communication between different communication goroutine:
global mutex variable
using the pipe channel
Illustrates global mutex: 1-10 all factorial processing result with coroutine way, the output results are as follows: 1 is the results of the factorial factorial effect is 1,2 ... 2
package main
import (
"fmt"
"sync"
)
var FactMap = make(map[int]int,10)
var lock sync.Mutex //变量是一个结构体变量，有两个方法 lock() unlock()
func Factorial(n int) {
res := 1
for i := 1 ; i <= n ; i++ {
res *= i
}
lock.Lock()
FactMap[n] = res //防撞写，使用lock
lock.Unlock()
}
func main() {
/*
用协程方式处理1-200所有阶乘的结果，输出结果如下： 1的阶乘结果为1,2的阶乘效果为2...
把上述结果加入一个map中
思路分析：
类似排队上厕所，如果坑里有人，那么就直接加锁，外面的人找个队列候着
*/
for i := 1; i <= 10 ; i++ {
go Factorial(i)
}
fmt.Println(FactMap)
}
Why use a pipe channel
- Lock synchronization using global variables to solve communication problems goroutine not perfect.
- The main program is difficult to determine in wait for all goroutine completed in time, similar to defer as one keyword to define the completion time.
- Use sleep time control of the main thread do not fly, much less are inappropriate.
- Locking position to be analyzed, too complex, sometimes less plus, plus various problems will appear wrong.
- ★★★ essence of the code quality: Do not communicate through shared memory, and memory to be shared by the communication! ! ! ! !
The introduction pipe channel
- It is essentially a data structure - the queue , FIFO: FIFO, Stack: last out.
- Array, a subsequent slice can implement pipeline, in addition to implementation of the pipeline there queue linked list.
- Pipe is an upper limit of capacity , after the contents of all queues are out on the air duct.
- Thread-safe, do not need to lock multi goroutine visit, write to solve the problem hit .
- channel type itself is generally not recommended mix, if the mix, the value of the type required assertion .
Statement conduit channel:
var <conduit variable name> chan <data type>
var intChan Chan int
var stringChan Chan String
var infChan Chan interface {}
var mapChan Chan Map [int] String
pipe string stringChan = make (chan string, 10) // 10 of string
- channel is a reference type.
- channel must be initialized in order to make use of .
- There are types of pipe, if the pipe is defined as a mix put the empty interface mode.
- If the pipe is defined as empty interface, be sure to use the type of assertion when variable values, because the default pass in the data will be considered an interface type.
- Using the close () can turn off the pipeline, can only be read but not write after closing, which is equivalent to the closure of the inlet .
- Defined channel may be read only or write-only, corresponding to the individual transmission,
var intChan1 Chan <- // int write-only pipe
var intChan2 <int // read-only pipe -chan
Pipeline channel use cases by understanding the pipe case
package main
import "fmt"
func main() {
var stringChan chan string
fmt.Println(stringChan) //nil 符合引用类型不开辟内存栈就是nil
stringChan = make(chan string,3) //管道容量为3
fmt.Println(stringChan) //0xc000038060 发现是一个地址
fmt.Println(&stringChan) //0xc000082018 这个是管道变量的地址，也就等于说管道就是个指针变量，所以支持多个协程操作管道
//向管道写数据
stringChan<- "蔡徐坤" // \管道变量名称\<- string
str := "鸡你太美"
stringChan<- str //
//查看管道容量len() cap()
fmt.Printf("%v %v", len(stringChan),cap(stringChan)) //len是值目前管道中使用容量 2 ，cap是指管道总容量 3
//注意管道一定不能超，超了容量所有的goroutine都会deadlock，管道的价值在于边放边取
//在不使用goroutine的情况下，管道中的数据取完了，也会报deadlock，注意这些。
//从管道取出1个元素，管道的len()会减1
var _ string
_ = <-stringChan //取出一个元素
fmt.Printf("%v %v", len(stringChan),cap(stringChan)) //len是值目前管道中使用容量 1 ，cap是指管道总容量 3
//还剩1个元素，鬼畜一般再取2次
<-stringChan //还剩0个,直接扔 <-stringChan
//_ = <-stringChan //deadlock
fmt.Printf("%v %v", len(stringChan),cap(stringChan))
var infChan chan interface{}
infChan = make(chan interface{},3)
infChan <-"唱，跳，rap，篮球"
infChan<- 666
rev := <-infChan //唱，跳，rap，篮球
fmt.Println(rev)
close(infChan) //关闭信道，这样里面还剩下 666 int，不可以再接收了，只能再把666读出来了。
fmt.Println(len(infChan)) //1
infChan<- "shit!" // panic: send on closed channel
}
About pipeline shut down and traverse
- If the pipe is not closed, deadlock error occurs, because a traverse len conduit 1 will be reduced, so that traversal corresponds to n / 2 +1 when no data directly, deadlock, but all data are traversed .
- If traversing pipes, pipe must be closed, normally through the data, it will traverse the complete exit .
package main import "fmt" func main() { var intChan chan int intChan = make(chan int , 100) for i := 1 ; i <= 100 ; i++ { intChan<- i } //没有加close(intChan)就会报错 close(intChan) for v := range intChan { fmt.Println(v) } }
According to the previous coroutine knowledge and pipelines, processing integrated case
1. Learn to complete the pipeline in order to solve the main thread to complete before the end of the program directly, without waiting for coroutines.
package main
import "fmt"
func writeData(intChan chan int) {
for i:=1;i<=50;i++ {
intChan<- i
}
close(intChan) //后续就可以用for循环读了
}
func readData(intChan chan int,exitChan chan bool) {
for {
v, ok := <-intChan //如果管道中没有数据了，ok默认为true，会变为false
if !ok { //
break
}
fmt.Printf("读到数据为：%v",v)
}
exitChan<- true
close(exitChan)
}
func main() {
/*完成协同工作案例
1. writeData协程，向管道写如50个int
2. readData协程，负责从管道中读取writeData写入的数据
3. 上述两个协程操作同一个管道
4. 主线程要等待上述两个协程都完成后，才能结束
*/
intChan := make(chan int,50)
exitChan := make(chan bool,1)
go writeData(intChan)
go readData(intChan,exitChan)
//接下来保证协程结束后，才结束主线程
for {
<-exitChan
//_, ok := <-exitChan //反复读取，直到读空
//if !ok {
// break
//}
}
}
2. Use coroutine pipe and the completion of all prime numbers ranging between 1 and 100
package main
import "fmt"
func inNum(intChan chan int) {
for i := 3; i <= 100 ; i++ {
intChan<- i
}
defer close(intChan)
}
func outChan(intChan chan int,primeChan chan int,exitChan chan bool) {
for {
num, ok := <-intChan
if !ok{ //管道里面取不到东西了，凡是赋值变量取所有管道内数据都是用这种方法。
break
}
//判断num是不是素数
for i := 2; i <= num ;i++ {
if num % i == 0 && i < num {
//不是素数不要
break
}
if num == i {
primeChan<- i
}
}
}
exitChan<- true
}
func main() {
intChan := make(chan int,2000)
primeChan := make(chan int,2000)
exitChan := make(chan bool,4)
start := time.Now().Unix()
//协程 放入数据
go inNum(intChan) //defer 直接后续关闭intChan管道
//开启四个协程，从intChan里面取出数据，并判断是否为素数，是就放入primeChan中
for i := 0 ; i <= 3 ; i++ {
go outChan(intChan,primeChan,exitChan)
}
for i := 0; i < 4 ; i++ {
<-exitChan //当收到管道中有四个true时则说明协程已经取完了primeChan中所有的内容
}
close(primeChan) //所以可以关闭管道
end := time.Now().Unix()
fmt.Println("使用协程耗时为：",end - start) //查看协程节省多少时间，结果确实证实可以提高效率
for {
res, ok := <-primeChan //遍历primeChan中所有的素数
if !ok {
break
}
fmt.Println("素数为：",res)
}
}
3. do not know when the pipe is closed, you can use duct obstruction select solve the problem:
package main
import "fmt"
func main() {
/*
select语句，依次向下执行，解决阻塞问题，也不用考虑何时需要close(varchan)了
*/
var intChan = make(chan int, 10)
var strChan = make(chan string,10)
for i := 0 ; i < 5 ; i++ {
intChan<- i
}
for i := 0 ; i < 5 ; i++ {
strChan<- "shit" + fmt.Sprintf("%d",i)
}
//close(intChan) //其中传统管道关闭方法
//for v:= range intChan {
// fmt.Println(v)
//}
for {
select {
case v := <-intChan:
fmt.Println(v)
case s := <-strChan:
fmt.Println(s)
default:
fmt.Println("不玩了继续下面了")
return //跳出整个for循环而不是select，也可以使用label...break方式来结束for循环从而终止main()
}
}
}
4. If there is a function there was panic, how to avoid panic auxiliary function causes the main function can not continue to use defer + recover.
package main
import (
"fmt"
"time"
)
var strChan = make(chan string,100)
func test1() {
for i := 0 ; i < 10 ; i++ {
strChan<- "鸡你太美" + fmt.Sprintf("%d",i)
}
close(strChan)
for {
v, ok:= <-strChan
if !ok {
break
}
fmt.Println(v)
}
}
func test2() {
/*匿名函数捕获错误，不要影响其他协程和主线程运行*/
defer func() {
if err := recover();err != nil {
fmt.Println("test2()协程发生错误是：",err)
}
}()
var shitMap map[int]string //这里没有为shitMap分配一个栈，肯定会报错
shitMap[1] = "蔡徐坤" //而这个协程阻止了主线程和另外一个协程的运行如果避免则需要加入defer+recover捕获错误
}
func main() {
go test1()
go test2()
time.Sleep(10 * time.Second)
}
Run Results: test1 outputs the content, test2 error, and the main program to wait for 10 seconds off.
