Python basis (XII) 02 concurrent programming

Coroutine

Coroutine, also called micro-threads, shred. English Coroutine.

Processes and threads are running preemptive

Coroutine is a collaborative (non-preemptive), run the program in the order that we have complete control

Only one thread on coroutine nature, mainly to solve the IO operation

advantage:

1 advantages: high efficiency coroutine. Because the switch is not thread-switching routines, but by the program itself, therefore, without the overhead of thread switching, and multi-threaded ratio, the greater the number of threads, the more obvious performance advantages coroutine.

Advantage 2: do not need multi-threaded locking mechanisms, because only one thread, there is no conflict while writing variable, control of the shared resource is not locked in coroutine, just like a judge status, so the efficiency is higher than multithreading a lot of.

Because coroutine is a thread of execution, how to take advantage of multi-core CPU it? The easiest way is multi-process + coroutine, both take full advantage of multi-core, high efficiency and give full play to the Association process, availability of high performance.

Simple implementation of yield

 1 import time
 2 import queue
 3 
 4 def consumer(name):
 5     print("--->ready to eat baozi...")
 6     while True:
 7         new_baozi = yield
 8         print("[%s] is eating baozi %s" % (name,new_baozi)) 9 #time.sleep(1) 10 11 def producer(): 12 13 r = con.__next__() 14 r = con2.__next__() 15 n = 0 16 while 1: 17 time.sleep(1) 18 print("\033[32;1m[producer]\033[0m is making baozi %s and %s" %(n,n+1) ) 19  con.send(n) 20 con2.send(n+1) 21 22 n +=2 23 24 25 if __name__ == '__main__': 26 con = consumer("c1") 27 con2 = consumer("c2") 28 p = producer()

Greenlet

greenlet implemented in C is a coroutine module, compared with the yield comes python, which allows you to freely switch between an arbitrary function, this function without the need for the generator to declare

 1 from greenlet import greenlet
 2  
 3  
 4 def test1():
 5     print(12)
 6     gr2.switch()
 7     print(34)
 8  gr2.switch() 9 10 11 def test2(): 12 print(56) 13  gr1.switch() 14 print(78) 15 16 17 gr1 = greenlet(test1) 18 gr2 = greenlet(test2) 19 gr1.switch()

peddled

Coroutine: meet the IO operation for automatically switching operation of a need cpu

 1 import gevent
 2 
 3 import requests,time
 4 
 5 
 6 start=time.time()
 7 
 8 def f(url):
 9     print('GET: %s' % url)
10     resp =requests.get(url) 11 data = resp.text 12 print('%d bytes received from %s.' % (len(data), url)) 13 14 gevent.joinall([ 15 16 gevent.spawn(f, 'https://www.python.org/'), 17 gevent.spawn(f, 'https://www.yahoo.com/'), 18 gevent.spawn(f, 'https://www.baidu.com/'), 19 gevent.spawn(f, 'https://www.sina.com.cn/'), 20 21 ]) 22 23 # f('https://www.python.org/') 24 # 25 # f('https://www.yahoo.com/') 26 # 27 # f('https://baidu.com/') 28 # 29 # f('https://www.sina.com.cn/') 30 31 print("cost time:",time.time()-start)

Context manager in Python ( contextlib Module )>

IO model

Event-driven model

Coroutine: meet IO operation will switch, then switch back when? How to determine the IO operation is over?

Linear programming mode

Start ---> block A ---> B block ---> code block C ---> code block D ---> ......---> End

Each code block is done in various things code, but the programmer knows the code block A, B, C, D ... execution order, the only way to change the flow of data. Different input data, determining according to the conditional statement, perhaps to process A ---> C ---> E ...---> end. Each program may have a different running order, but its control flow is determined by the input data and the programs you write. If you know the current state of the program (including the input data and the program itself), then you know the next even until the end of its operating procedures.

Event-driven model

Start ---> Initialize ---> waiting

And above traditional programming model is different from event-driven procedure after the startup, in that waiting, waiting for what is it? Waiting to be triggered event. Traditional programming when there are "wait", as in block D, you define an INPUT (), requires the user to enter data. But wait with the following different traditional programming "wait", such as input (), you, as the programmer knows or force the user to input something, perhaps a number, perhaps the file name, if the user input errors, you also need to remind him, and asked him to re-enter. Waiting for event-driven programs are completely unaware, or do not force the user to input. Whenever an event occurs, then the program will make the appropriate "response." These events include: information input, mouse, tap on the keyboard as well as a key trigger internal system timer.

If you create a thread to detect a user's mouse click events, Cons:

The CPU resources, mouse click frequency may be very small, but still would have been scanning the thread loop detection, it will cause a lot of CPU resources wasted; if the scan-click interface is blocking it?
If it is blocked, the following will be such a problem, if we want to scan only a mouse click, but also scan the keyboard is depressed, due to the scanning mouse is blocked, you may never go scan the keyboard;
If a scan cycle requires very large equipment, which in turn can lead to problems in response time;

Therefore, this method is very bad.

Therefore, the use of event-driven model to solve this problem, the general idea:

There is an event (message) queue;
When the mouse is pressed, the queue to add a click event (message);
There cycles continuously removed from the event queue, depending on the event, calls different functions, such as onClick (), onKeyDown () and the like;
Events (messages) are generally each storing respective processing function pointers, so that each has its own message handler;

Event-driven programming is a programming paradigm, where the program execution flow is determined by external events. It features include an event loop, when an external event occurs using a callback mechanism to trigger the appropriate treatment .

With examples to compare and contrast what single-threaded, multi-threaded and event-driven programming model. The following figure shows the work done over time these three modes program. This program has three tasks to complete each task is waiting for I / blocking their operation O. Time block on I / O operation has been marked spent in the gray boxes.

The initial problem: how to determine the IO operations cut back over it? Callback function (attention, event-driven monitor events by cpu operating system call to complete)