Asynchronous IO is a difficult point in python development.It takes a long time to understand and learn, and it is boring, but his efficiency is undeniable.
If you want to learn asynchronous IO, you can't avoid the study of coroutines. My notes are written according to Liao Xuefeng, you can refer to Liao Xuefeng
Coroutines are also known as microthreads, English name: coroutine, the concept of coroutines was proposed very early, but in recent years, Cai has been widely used in some languages, lua,
Subroutines, or functions, are called hierarchically in all languages.For example, A calls B, B calls C during execution, C executes and returns, B executes and returns, and finally A executes and finishes.
Therefore, the subroutine is implemented through the stack, and a thread is to execute a subroutine.
Subroutine call is always an entry, once but returns, the calling sequence is also clear, the call of coroutine is different from the subroutine
Coroutines also seem to be subroutines, but during execution, they can be interrupted (paused) within the subroutine, and then execute other subroutines.When appropriate, return to execute again.
Note that interrupting in a subroutine to execute other subroutines is not a function call, which is somewhat similar to CPU interrupts, such as subroutines A and B
def A():
print('1')
print('2')
print('3')
def B():
print('x')
print('y')
print('z')Assuming that it is executed by a coroutine, during the execution of A, it can be interrupted at any time to execute B, and B may also be interrupted during execution and then execute A. The result may be:
1
2
x
y
3
z
But there is no call in A, so the call of coroutine is more difficult to understand than the call of function.
It seems that the execution of A and B is a bit like multithreading, but the characteristic of coroutines is that they are executed by one thread.What is the advantage of multithreading over coroutines?
1. Very high execution efficiency, because the switching of subroutines is not the switching of threads, but is controlled by the program itself. Therefore, the overhead of thread switching is eliminated.Compared with multi-threading, the more threads, the performance of coroutines The more obvious the advantage.
2. There is no need for a multi-threaded locking mechanism, because there is only one thread, and there is no conflict of simultaneous write variables. Controlling shared resources in a coroutine does not add a lock, just need to judge the state, so the execution efficiency is much higher than multi-thread .
Because coroutines are executed by one thread, the easiest way to use multi-core CPu is multi-threading + coroutines, make full use of multi-cores, give full play to the high efficiency of coroutines, and get extremely high performance.
Python's initial support for coroutines is achieved through generator generators
Starting from the learning generator to understand coroutines is an indispensable step, because it will let you understand coroutines faster.
In the generator, you can iterate through the for loop, and you can continue to call the next function to return the next value by yield
In Python, yield can not only generate a value, but also receive parameters issued by the caller.
example
The traditional producer-consumer model is a thread to write messages, a thread to take messages, control the queue and wait through the lock mechanism, but you may deadlock if you are not careful.
If the coroutine is used instead, the producer directly yieldexecutes the message by jumping to the consumer after producing the message. After the consumer completes the execution, it switches back to the producer to continue production, which is extremely efficient:
"""
1.回调模式编码复杂度高
2.同步编程的并发性不高
3.多线程编程需要线程间同步
1.采用同步的方式去编写异步代码
2.使用单线程去切换任务
1.线程是由操作系统切换的,单线程切换意味着我们需要程序员自己去调度任务
2.不再需要锁,并发性高,如果单线程内切换函数,性能远高于线程切换,并发性高
"""
# 生成器
def gen_func():
html = yield "http://wwww.baidu.com"
print(html) # 传入666之后再输出
yield 1
yield 2
return "1213"
"""
# 启动生成器有两种方式: next,send()
send可以传入值到生成器中,同时重启生成器执行到下一个yield
"""
if __name__ == '__main__':
gen = gen_func()
print(gen.send(None)) # 初次使用Send需要传入None,因为其根本未运行到第一行,所以需要先send到第一行
git = "666"
gen.send(git) # 传入一个数据,并且生成器内部可以自动接收值(变量名不需一致),它会返回下一个yild的值,也就是1,
print(gen.send(git)) # 此时传入git的值,但是yield已经执行到1,所以只能返回2的值
Results of the::
[PRODUCER] Producing 1...
[CONSUMER] Consuming 1...
[PRODUCER] Consumer return: 200 OK
[PRODUCER] Producing 2...
[CONSUMER] Consuming 2...
[PRODUCER] Consumer return: 200 OK
[PRODUCER] Producing 3...
[CONSUMER] Consuming 3...
[PRODUCER] Consumer return: 200 OK
[PRODUCER] Producing 4...
[CONSUMER] Consuming 4...
[PRODUCER] Consumer return: 200 OK
[PRODUCER] Producing 5...
[CONSUMER] Consuming 5...
[PRODUCER] Consumer return: 200 OKIt can be noticed that the consumer function is a generator.After passing a consumer to the producer:
1. First call c.send (None) to activate the generator
2. Once something is generated, switch to consumer execution via c.send (n)
3. The consumer gets the message through yield, processes it, and passes the result back through yield
4. The producer gets the result of consumer processing and continues to produce the next message.
5. The producer decided not to produce any more, and the process of closing the consumer through c.close () ended.
The entire process is lock-free, and there is only one thread. Produce and consumer cooperate to complete.The order is called a coroutine instead of preemptive multitasking of threads.
Finally, apply a sentence from Donald Knuth to summarize the characteristics of coroutines:
"Subroutines are a special case of coroutines."
The following code for your own practice can refer to:
"""
1.回调模式编码复杂度高
2.同步编程的并发性不高
3.多线程编程需要线程间同步
1.采用同步的方式去编写异步代码
2.使用单线程去切换任务
1.线程是由操作系统切换的,单线程切换意味着我们需要程序员自己去调度任务
2.不再需要锁,并发性高,如果单线程内切换函数,性能远高于线程切换,并发性高
"""
# 生成器
def gen_func():
html = yield "http://wwww.baidu.com"
print(html)
yield 1
yield 2
return "1213"
"""
# 启动生成器有两种方式: next,send()
send可以传入值到生成器中,同时重启生成器执行到下一个yield
"""
if __name__ == '__main__':
gen = gen_func()
print(gen.send(None)) # 初次使用Send需要传入None,因为其根本未运行到第一行,所以需要先send到第一行
git = "666"
print(gen.send(git)) # 传入一个数据,并且生成器内部可以自动接收值(变量名不需一致)
