Introduction gevent (rpm)

Original: https://www.liaoxuefeng.com/wiki/897692888725344/966405998508320

Python by yieldproviding basic support for coroutines, but not entirely. The third gevent for Python provides a fairly complete coroutine support.

gevent third-party libraries, by greenlet achieve coroutine, the basic idea is:

When a greenlet encountered IO operations, such as access to the network, it automatically switches to the other greenlet, IO wait until the operation is complete, then switch back at the appropriate time to continue. Since the IO operation is very time-consuming, often make the program in a wait state, with gevent automatically switch coroutine for us to ensure that there is always greenlet running, instead of waiting for IO.

Since the switching is done automatically when operating in IO, so gevent need to modify some of the standard library that comes with Python, when you start this process completed by monkey patch:

from gevent import monkey; monkey.patch_socket()
import gevent

def f(n):
    for i in range(n):
        print gevent.getcurrent(), i

g1 = gevent.spawn(f, 5)
g2 = gevent.spawn(f, 5)
g3 = gevent.spawn(f, 5)
g1.join()
g2.join()
g3.join()

operation result:

<Greenlet at 0x10e49f550: f(5)> 0
<Greenlet at 0x10e49f550: f(5)> 1
<Greenlet at 0x10e49f550: f(5)> 2
<Greenlet at 0x10e49f550: f(5)> 3
<Greenlet at 0x10e49f550: f(5)> 4
<Greenlet at 0x10e49f910: f(5)> 0
<Greenlet at 0x10e49f910: f(5)> 1
<Greenlet at 0x10e49f910: f(5)> 2
<Greenlet at 0x10e49f910: f(5)> 3
<Greenlet at 0x10e49f910: f(5)> 4
<Greenlet at 0x10e49f4b0: f(5)> 0
<Greenlet at 0x10e49f4b0: f(5)> 1
<Greenlet at 0x10e49f4b0: f(5)> 2
<Greenlet at 0x10e49f4b0: f(5)> 3
<Greenlet at 0x10e49f4b0: f(5)> 4

You can see, three greenlet run sequentially rather than alternating operation.

Greenlet make alternate operation, you can gevent.sleep()hand over control:

def f(n):
    for i in range(n):
        print gevent.getcurrent(), i
        gevent.sleep(0)

Results of the:

<Greenlet at 0x10cd58550: f(5)> 0
<Greenlet at 0x10cd58910: f(5)> 0
<Greenlet at 0x10cd584b0: f(5)> 0
<Greenlet at 0x10cd58550: f(5)> 1
<Greenlet at 0x10cd584b0: f(5)> 1
<Greenlet at 0x10cd58910: f(5)> 1
<Greenlet at 0x10cd58550: f(5)> 2
<Greenlet at 0x10cd58910: f(5)> 2
<Greenlet at 0x10cd584b0: f(5)> 2
<Greenlet at 0x10cd58550: f(5)> 3
<Greenlet at 0x10cd584b0: f(5)> 3
<Greenlet at 0x10cd58910: f(5)> 3
<Greenlet at 0x10cd58550: f(5)> 4
<Greenlet at 0x10cd58910: f(5)> 4
<Greenlet at 0x10cd584b0: f(5)> 4

3 greenlet operated alternately,

The number of cycles was changed to 500,000, let them run a bit longer, and then in the operating system's process manager seen, only a number of threads.

Of course, the actual code, we do not use gevent.sleep()to switch coroutine, but when performing an IO operation, automatically switch GEVENT, as follows:

from gevent import monkey; monkey.patch_all()
import gevent
import urllib2

def f(url):
    print('GET: %s' % url)
    resp = urllib2.urlopen(url)
    data = resp.read()
    print('%d bytes received from %s.' % (len(data), url))

gevent.joinall([
        gevent.spawn(f, 'https://www.python.org/'),
        gevent.spawn(f, 'https://www.yahoo.com/' ), 
        Gevent.spawn (f, ' https://github.com/ ' ), 
])

operation result:

GET: https://www.python.org/
GET: https://www.yahoo.com/
GET: https://github.com/
45661 bytes received from https://www.python.org/.
14823 bytes received from https://github.com/.
304034 bytes received from https://www.yahoo.com/.

From the results, the three network operation is executed concurrently, and ending in a different order, but only one thread.

summary

Use gevent, you can get a high concurrent performance, but gevent can only run under Unix / Linux, Windows does not guarantee under normal installation and operation.

Since gevent IO is switched coroutine-based, so the most amazing is that we write the code for Web App does not require the introduction of gevent package, do not need to change any code, just in time to deploy, with a support of WSGI server gevent , immediately gained several times performance increase. Specific deployment can refer to the follow-up to "real" - "Deploying Web App" one.