Python learning--asyncio

asyncio is a standard library introduced in Python 3.4, which directly supports asynchronous IO.

The programming model of asyncio is a message loop. We directly obtain a reference to EventLoop from the asyncio module, and then throw the coroutine that needs to be executed into EventLoop for execution, and then asynchronous IO is realized.

Use asyncio to implement the Hello world code as follows:

import asyncio

@asyncio.coroutine
def hello():
	print('Hello world!')
	# 异步调用asyncio.sleep(1):
	r = yield from asyncio.sleep(1)
	print("Hello again!")

# 获取EventLoop:
loop = asyncio.get_event_loop()
# 执行coroutine
loop.run_until_complete(hello())
loop.close()

@asyncio.coroutine marks a generator as a coroutine type, and then we throw this coroutine into EventLoop for execution.

hello() will print Hu Hello world first! , Then, the yield from syntax allows us to conveniently call another generator. Since asyncio.sleep() is also a coroutine, the thread will not wait for asyncio.sleep(), but directly interrupt and execute the next message loop. When asyncio.sleep() returns, the thread can get the return value from yield from (here it is None), and then execute the next line of statements.

Consider asyncio.sleep(1) as an IO operation that takes 1 second. During this period, the main thread does not wait, but executes other executable coroutines in EventLoop, so concurrent execution can be achieved.

Let's try to encapsulate two coroutines with Task:

import threading
import asyncio

@asyncio.coroutine
def hello():
	print('Hello world! (%s)' % threading.currentThread())
	yield from asyncio.sleep(1)
	print('Hello again! (%s)' % threading.currentThread())

loop = asyncio.get_event_loop()
tasks = [hello(), hello()]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

Observe the execution process:

Hello world! (<_MainThread(MainThread, started 140735195337472)>)
Hello world! (<_MainThread(MainThread, started 140735195337472)>)
(暂停约1秒)
Hello again! (<_MainThread(MainThread, started 140735195337472)>)
Hello again! (<_MainThread(MainThread, started 140735195337472)>)

It can be seen from the printed current thread name that the two coroutines are executed concurrently by the same thread.

If you replace asyncio.sleep() with a real IO operation, multiple coroutines can be executed concurrently by one thread.

We use asyncio's asynchronous network connection to get the homepage of sina, sohu and 163:

import asyncio

@asyncio.coroutine
def wget(host):
	print('wget %s...' % host)
	connect = asyncio.open_connection(host, 80)
	reader, writer = yield from connect
	header = 'GET / HTTP/1.0\r\nHost: %s\r\n\r\n' % host
    writer.write(header.encode('utf-8'))
    yield from writer.drain()
    while True:
    	line = yield from reader.readline()
    	if line == b'\r\n':
            break
        print('%s header > %s' % (host, line.decode('utf-8').rstrip()))
    # Ignore the body, close the socket
    writer.close()

loop = asyncio.get_event_loop()
tasks = [wget(host) for host in ['www.sina.com.cn', 'www.sohu.com', 'www.163.com']]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
    	

The results are as follows:

wget www.sohu.com...
wget www.sina.com.cn...
wget www.163.com...
(等待一段时间)
(打印出sohu的header)
www.sohu.com header > HTTP/1.1 200 OK
www.sohu.com header > Content-Type: text/html
...
(打印出sina的header)
www.sina.com.cn header > HTTP/1.1 200 OK
www.sina.com.cn header > Date: Wed, 20 May 2015 04:56:33 GMT
...
(打印出163的header)
www.163.com header > HTTP/1.0 302 Moved Temporarily
www.163.com header > Server: Cdn Cache Server V2.0
...

It can be seen that 3 connections are completed concurrently by one thread through coroutine.

summary

asyncio provides complete asynchronous IO support;

Asynchronous operations need to be completed through yield from in coroutine;

Multiple coroutines can be encapsulated into a set of Tasks and then executed concurrently.
(Key: Concurrent execution after encapsulating a group of Tasks)