Production Process Python Association (rpm)

Original: https://www.liaoxuefeng.com/wiki/897692888725344/923057403198272

Coroutine, also called micro-threads, shred. English Coroutine.

The concept of coroutines brought up very early, but until recent years has been widely used in some languages ​​(eg Lua) in.

Subroutine or function is called, is a call hierarchy in all languages, such as A calls B, B in the implementation process and called C, C completes the return, return completed implementation of B, A is finally finished.

So subroutine call is achieved through the stack, a thread is to execute a subroutine.

Subroutine call is always an entrance, a return call to order is clear. Different coroutine calls and subroutines.

Coroutine also looks routine, but the implementation process can be interrupted within the subroutine, and then turn to perform another routine, at the appropriate time and then back again to execute.

Note that in a subroutine interrupt, to perform other routine, not a function call, somewhat similar to the CPU interrupt. For example subroutines A, B:

def A():
    print '1'
    print '2'
    print '3'

def B():
    print 'x'
    print 'y'
    print 'z'

Assumed by the coroutine execution, the execution of the A's may be interrupted at any time, to perform B, B may be interrupted again performs A during execution, the result may be:

1
2
x
y
3
z

But A is not called B, so coroutine calling than to understand the function call to be harder.

Implementation of A, B looks a bit like a multi-threaded, but coroutines characteristics so a thread of execution, and what are the advantages that multi-threading than coroutine?

The biggest advantage is the high efficiency coroutine. Because the switch is not thread-switching routines, but by the program itself, therefore, without the overhead of thread switching, and multi-threaded ratio, the greater the number of threads, the more obvious performance advantages coroutine.

The second big advantage is that do not need multi-threaded locking mechanisms, because only one thread, there is no conflict while writing variable, control of the shared resource is not locked in coroutine, just like a judge status, so the efficiency of libido many high thread.

Because coroutine is a thread of execution, how to take advantage of multi-core CPU it? The easiest way is multi-process + coroutine, both take full advantage of multi-core, high efficiency and give full play to the Association process, availability of high performance.

Python support coroutine is still very limited, with the yield in the generator can be achieved to some extent coroutine. Although support is not complete, but can already play a considerable power.

Look at an example:

Traditional producer - consumer model is to write a message thread, a thread take messages, through the lock mechanism to control the queue and wait, but believe it may deadlock.

If the switch to co-drive producers to produce messages directly to consumers through yield jump started, until the consumer is finished, switch back to the producers to continue production, high efficiency:

import time

def consumer():
    r = ''
    while True:
        n = yield r
        if not n:
            return
        print('[CONSUMER] Consuming %s...' % n)
        time.sleep(1)
        r = '200 OK'

def produce(c):
    c.next()
    n = 0
    while n < 5:
        n = n + 1
        print('[PRODUCER] Producing %s...' % n)
        r = c.send(n)
        print('[PRODUCER] Consumer return: %s' % r)
    c.close()

if __name__=='__main__':
    c = consumer()
    produce(c)

Results of the:

[PRODUCER] Producing 1...
[CONSUMER] Consuming 1...
[PRODUCER] Consumer return: 200 OK
[PRODUCER] Producing 2...
[CONSUMER] Consuming 2...
[PRODUCER] Consumer return: 200 OK
[PRODUCER] Producing 3...
[CONSUMER] Consuming 3...
[PRODUCER] Consumer return: 200 OK
[PRODUCER] Producing 4...
[CONSUMER] Consuming 4...
[PRODUCER] Consumer return: 200 OK
[PRODUCER] Producing 5...
[CONSUMER] Consuming 5...
[PRODUCER] Consumer return: 200 OK

Noting consumer is a function Generator (generator), to produce a consumer after passing:

1. First call c.next () start the generator;

2. Then, once the production of something, by switching c.send (n) to perform consumer;

3. consumer to get the message through yield, treatment, and the results returned by yield;

4. produce consumer process to get the result, we continue to produce the next message;

5. We decided not to produce the production, closed by consumer c.close (), the end of the whole process.

No lock entire process executed by a thread, produce and consumer collaboration to complete the task, so called "coroutine" preemptive multitasking rather than threads.

Finally, apply the characteristics of Donald Knuth's sentence summary coroutines:

"Subroutine is a kind of coroutine special case."