Introduction to Python Coroutine [Book Donation Activity｜The fifth issue of "Python Programming Introduction and Practical Combat"]

Article Directory

1. Related concepts

1. Coroutine

Coroutines, also known as micro-threads, fibers. The English name is Coroutine.

The concept of coroutines has been around for a long time, but only in the last few years has it become widely used in some languages such as Lua.

2. Subroutine

Subroutines, or functions, are called hierarchically in all languages. For example, A calls B, B calls C during execution, C returns after execution, B returns after execution, and finally A completes execution.

So the subroutine call is implemented through the stack, and a thread executes a subroutine.

A subroutine call is always one entry and one return, and the calling sequence is clear. The call of a coroutine is different from a subroutine.

3. Difference

The coroutine also looks like a subroutine, but during the execution process, it can be interrupted inside the subroutine, and then turn to execute other subroutines, and then return to continue execution at an appropriate time.

Note that interrupting in a subroutine to execute other subroutines is not a function call, which is somewhat similar to CPU interrupts. For example, subroutines A and B:

def A():
    print '1'
    print '2'
    print '3'

def B():
    print 'x'
    print 'y'
    print 'z'

Assuming that it is executed by a coroutine, in the process of executing A, it can be interrupted at any time to execute B, and B may also be interrupted during the execution process and then execute A. The result may be:

1
2
x
y
3
z

But in A, B is not called, so the call of the coroutine is more difficult to understand than the function call.

It seems that the execution of A and B is a bit like multi-threading, but the characteristic of coroutine is that it is executed by one thread. Compared with multi-threading, what advantage does coroutine have?

4. Advantages of coroutines

The biggest advantage is the extremely high execution efficiency of the coroutine. Because subroutine switching is not thread switching, but controlled by the program itself, there is no overhead of thread switching. Compared with multi-threading, the more threads there are, the more obvious the performance advantage of coroutines will be.

The second big advantage is that there is no need for a multi-threaded locking mechanism, because there is only one thread, and there is no conflict of writing variables at the same time. In the coroutine, shared resources are controlled without locking, and only need to judge the state, so the execution efficiency is higher than many The thread is much taller.

Because the coroutine is executed by a thread, how to use multi-core CPU? The simplest method is multi-process + coroutine, which not only makes full use of multi-core, but also gives full play to the high efficiency of coroutine, and can obtain extremely high performance.

Python's support for coroutines is still very limited, and the yield used in the generator can realize coroutines to a certain extent. Although the support is not complete, it can already exert considerable power.

Two, examples

The traditional producer-consumer model is that one thread writes messages, one thread fetches messages, and controls the queue and waiting through the lock mechanism, but it may deadlock if you are not careful.

If you switch to a coroutine, after the producer produces a message, it directly jumps to the consumer through yield to start execution. After the consumer finishes executing, switch back to the producer to continue production, which is extremely efficient:

import time

def consumer():
    r = ''
    while True:
        n = yield r
        if not n:
            return
        print('[CONSUMER] Consuming %s...' % n)
        time.sleep(1)
        r = '200 OK'

def produce(c):
    c.next()
    n = 0
    while n < 5:
        n = n + 1
        print('[PRODUCER] Producing %s...' % n)
        r = c.send(n)
        print('[PRODUCER] Consumer return: %s' % r)
    c.close()

if __name__=='__main__':
    c = consumer()
    produce(c)

Results of the:

[PRODUCER] Producing 1...
[CONSUMER] Consuming 1...
[PRODUCER] Consumer return: 200 OK
[PRODUCER] Producing 2...
[CONSUMER] Consuming 2...
[PRODUCER] Consumer return: 200 OK
[PRODUCER] Producing 3...
[CONSUMER] Consuming 3...
[PRODUCER] Consumer return: 200 OK
[PRODUCER] Producing 4...
[CONSUMER] Consuming 4...
[PRODUCER] Consumer return: 200 OK
[PRODUCER] Producing 5...
[CONSUMER] Consuming 5...
[PRODUCER] Consumer return: 200 OK

Note that the consumer function is a generator (generator), after passing a consumer into produce:

First call c.next() to start the generator;
Then, once something is produced, switch to consumer execution via c.send(n);
The consumer gets the message through yield, processes it, and sends the result back through yield;
Produce gets the result processed by the consumer and continues to produce the next message;
Produce decides not to produce, and closes consumer through c.close(), and the whole process ends.

The entire process is lock-free and executed by one thread. Producer and consumer cooperate to complete the task, so it is called "coroutine" instead of thread preemptive multitasking.

Finally, apply a sentence from Donald Knuth to summarize the characteristics of coroutines:

"A subroutine is a special case of a coroutine."

book donation

insert image description here

Follow+Like+Favorite Articles
Leave a message in the comment area: Become an architect with He Chongtian (follow and leave a message to enter the prize pool, each person can leave a maximum of three messages)
Random draw on Sunday at 8pm
This time, 2~5 books will be given away [the more you read, the more you will give]
500-1000 2 books
1000-1500 3 books
1500-2000 4 books
2000+ 5 books