background knowledge
Here to give some brief description of common knowledge in order to understand the content of the article that follows.
Process definition:
Process, the entity is a computer already running the program. Instruction program itself is just really running instance of the data and their organizations in the form of description, the process is the process.
Thread definition:
The operating system can schedule a minimum unit operation. It is included in the process, the actual operation of the unit process.
Processes and threads of relationship:
A thread refers to a single control flow of a process sequence, a process can be complicated by a plurality of threads, each thread in parallel to perform different tasks.
Minimum scheduling unit CPU thread is not the process, so a single multithreaded process can also use a multi-core CPU.
Coroutine definition:
By implementing coroutines in the thread scheduling, to avoid the performance penalty into the kernel level context switching caused, thus breaking the thread performance bottlenecks on IO's.
Relations Institute of processes and threads
Coroutines is to achieve the thread scheduling in the language level, to avoid the kernel level of consumption context.
python coroutine
Coroutine yield from Python commands. yield has two functions:
- yield item to output a value to the feedback next () caller.
- To make concessions to suspend the generator, so that the caller continues to work, then call the next until you need another value ().
import asyncio
async def compute(x, y):
print("Compute %s + %s ..." % (x, y))
await asyncio.sleep(x + y)
return x + y
async def print_sum(x, y):
result = await compute(x, y)
print("%s + %s = %s" % (x, y, result))
loop = asyncio.get_event_loop()
tasks = [print_sum(1, 2), print_sum(3, 4)]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()Coroutine is scheduled for the thread, yield similar manner lazy evaluation can be seen as a process control tool, implement cooperative multitasking, in Python3.5 formal introduction of the async/awaitexpression, making the coroutine is officially supported and optimized at the language level yield written before, greatly simplified.
Preemptive kernel thread is scheduled, thus ensuring that each thread has a chance to execute.
And coroutinerun in the same thread, by the runtime language is EventLoopto schedule (event loop).
As with most languages, in Python, coroutines are non preemptive scheduling, which means that a coroutine must take the initiative to give up the opportunity to perform, the other co-routines have a chance to run.
Let the keywords do is await. In other words, if a coroutine blocked, continue to keep the CPU, then the entire thread stuck, without any concurrency.
In short, any time there is only one coroutine is running.
PS: as a server, the event loop is the core IO multiplexing, all requests from clients by multiplexing function to handle IO; as a client, the core event loop delay is to utilize Future object execution, and use the send function to stimulate coroutine, hang, waiting for server processing continues with the following process invokes CallBack function returns after completion
Go coroutine
Go natural language support level, and Python are based on similar keywords, but in Go go using this keyword may want to indicate coroutines Go language is the most important characteristic.
Communication between the go coroutine, Go using the channel key.
Go implements two concurrent forms:
- Multi-threaded shared memory. When Java or C ++, etc. The shared data (e.g. an array, the Map, or a structure or object) in a multi-thread, accessed via the lock.
- Go language-specific, but also in Go Recommended: CSP (communicating sequential processes) concurrency model.
The CSP Go concurrency model implemented: M, P, G: [ HTTPS: //www.cnblogs.com/sunsk ... ]
package main
import (
"fmt"
)
//Go 协程（goroutines）和协程（coroutines）
//Go 协程意味着并行（或者可以以并行的方式部署），协程一般来说不是这样的
//Go 协程通过通道来通信；协程通过让出和恢复操作来通信
// 进程退出时不会等待并发任务结束，可用通道（channel）阻塞，然后发出退出信号
func main() {
jobs := make(chan int)
done := make(chan bool) // 结束标志
go func() {
for {
j, more := <-jobs // 利用more这个值来判断通道是否关闭，如果关闭了，那么more的值为false，并且通知给通道done
fmt.Println("----->:", j, more)
if more {
fmt.Println("received job", j)
} else {
fmt.Println("end received jobs")
done <- true
return
}
}
}()
go func() {
for j := 1; j <= 3; j++ {
jobs <- j
fmt.Println("sent job", j)
}
close(jobs) // 写完最后的数据,紧接着就close掉
fmt.Println("close(jobs)")
}()
fmt.Println("sent all jobs")
<-done // 让main等待全部协程完成工作
}By using the keyword before the function call go, we can make this function to goroutinefashion execute. goroutineIt is a more lightweight than threads, cheaper resources coroutine.
Go language to dispatch multiple threads of these functions through the implementation of the system, so that each function is performed using keywords can go run into a unit coroutine.
When a coroutine blocked, the scheduler will automatically arrange the other co-routines to perform additional thread, enabling the program operation without waiting for parallelization.
And scheduling overhead is very small, the size of a CPU scheduling of no less than one million times per second, which allows us to create a lot of goroutine, so it's easy to write highly concurrent program, we want to achieve the purpose. ---- a book
Four states coroutine
- Pending
- Running
- Done
- Cacelled
And mapping the relationship between the system thread
thread calls the coroutine nature go or system, and coroutines in Python is eventloop model implementation, so although coroutine call, but not a thing.
Python in coroutine is strictly 1: N relationship, It is a thread corresponding to a plurality of coroutine. Although asynchronous I / O, but can not effectively use the multi-core (GIL).
The Go is M: N relationship, that is, the N coroutine mapped assigned to M thread, this brings two advantages:
- Multiple threads can be assigned to different cores, CPU-intensive applications using goroutine will get accelerated.
- Even a small amount of blocking operation, it will only block a worker thread, the whole program will not clog.
PS: Go rarely mentioned thread or process, that is because of the above reasons.
Python and Golang coroutine comparison:
- Python is
asynca non-preemptive, once you start usingasyncthe function, then your whole program must beasync, otherwise there will always be blocking the place (a case of obstruction to the asynchronous nature of the library did not realize you can not take the initiative to let the dispatcher scheduling other coroutine a), that isasynccontagious. - Python entire asynchronous programming ecological problems, before the standard library and a variety of third-party libraries do not use the obstructive function, such as: requests, redis.py, open function and so on. So the biggest problem to join the Association after Python3.5 process is not difficult to use, but not good ecological environment, historical burden once again staged, together with task scheduling between the multi-core dynamic languages based on technology it should be difficult, sincerely hope that python4.0 can be optimized or abandon GIL lock, using multi-core performance improvements.
goroutineGo is an innate characteristic, so almost all libraries are used directly, avoiding the problem of Python libraries need to rewrite all of.goroutineDoes not require the explicit useawaitrelinquish control, but it will not Go in strict accordance with the time slice to schedulegoroutine, but scheduling will be inserted where possible obstruction.goroutineScheduling can be seen as semi-preemptive.
PS: python asynchronous library list [ HTTPS: //github.com/timofurrer ... ]
Do not communicate by sharing memory; instead, share memory by communicating (not shared memory approach to communication, on the contrary, to be shared memory via communication) - CSP concurrency model.
Expansion and summary
golang erlang and are based on the CSP (Communicating Sequential Processes) mode (Python in coroutine is eventloop model)
but erlang process based messaging, go goroutine and communication channel based.
Go have introduced Python and message scheduling system model, to avoid the impact of locks and process large / thread overhead problem.
Coroutine thread is essentially a user-state, the system does not need to perform preemptive scheduling, but to achieve the language level thread scheduling.
Because the shared memory is no longer used coroutine / data, but the use of shared memory communication to / locks, since a lock has numerous super system,
shared variables, etc. will make the entire system becomes very cumbersome, but through the message exchange mechanism, may be such that each cell concurrently become a separate entity,
with its own variables, which are not shared between the cells, only the output unit for the input message.
Developers need only concerned about the impact on a concurrent input and output unit, without the need to consider the similar impact modified shared memory / data to other programs.