"Python Basics" Asynchronous I/O Programming

I/O-intensive applications greatly enhance system multitasking capabilities;

Asynchronous I/O Model

A message loop in which the main thread repeats 读取消息-处理消息;

# 获取线程池
loop = get_event_loop()
while True:
    # 接收事件消息
    event = loop.get_event()
    # 处理事件消息
    process_event(event)

When encountering I/O operations, the code will only issue I/O requests without waiting for I/O results. When the current round of messages ends and the next round of receiving messages receives I/O completion, the I/O results will be processed;

1. Coroutines

Scheduling strategies are written by programmers themselves, and are created, switched, and destroyed in user mode, and are invisible to the kernel through collaboration rather than preemption 用户空间线程;

The essence of the coroutine is the active yield (yield) and recovery (resume) mechanism of the control flow

• 子程序, also known as function, is a hierarchical call in all languages. It is implemented through the stack. A thread executes a subroutine. A subroutine call is always an entry and a return. The call sequence is clear;

• Coroutine, Python's support for coroutines is generatorimplemented , which can be interrupted internally during execution, and then execute other subroutines, and then return to continue execution in due course (similar to CPU interrupts);

The coroutine has no thread switching (preemptive) overhead, and there is no variable conflict, no thread lock is required, and the efficiency is higher than multi-threading;

1. Producer-consumer model (coroutine version)

def consumer():
    r = ''
    while True:
        # 2. 通过 yield 回传 r 给 send 调用
        # 4. 接收 send 的消息 n
        n = yield r
        if not n:
            return
        print(f'[CONSUMER] Consuming {
      
      n}...')
        r = '200 OK'


def produce(c):
    # 1. 启动生成器
    c.send(None)
    n = 0
    while n < 5:
        n += 1
        print(f'[PRODUCER] Producing {
      
      n}...')
        # 3. 发送消息 n 返回给 yield
        # 5. 接收 yield 的结果 r
        r = c.send(n)
        print(f'[PRODUCER] Consumer return: {
      
      r}')
    # 6. 关闭生成器
    c.close()


# 消费者 - 生成器对象
c = consumer()
produce(c)

2. asyncio

Python 3.4 introduces the standard library, which provides comprehensive asynchronous I/O support;

asyncioThe programming model of the program is a message loop. First, it needs to asyncioobtain a reference from a EventLoopmessage, and then throw the executed coroutine into EventLoopit for execution, so as to realize asynchronous I/O;

import asyncio


# @aysncio.coroutine 把 generator 标记成 coroutine
@asyncio.coroutine
def wget(host):
    print('wget %s...' % host)
    connect = asyncio.open_connection(host, 80)
    # yield from 调用 connect 生成器，并接受 connect 的调用结果
    # 主线程并未等待 connect 调用，而是执行 EventLoop 中其他 coroutine
    reader, writer = yield from connect
    header = 'GET / HTTP/1.0\r\nHost: %s\r\n\r\n' % host
    writer.write(header.encode('utf-8'))
    yield from writer.drain()
    while True:
        line = yield from reader.readline()
        if line == b'\r\n':
            break
        print('%s header > %s' % (host, line.decode('utf-8').rstrip()))
    # Ignore the body, close the socket
    writer.close()


loop = asyncio.get_event_loop()
tasks = [
    wget(host) for host in ['www.sina.com.cn', 'www.sohu.com', 'www.163.com']
]
# 把 coroutine 扔到 EventLoop 中执行
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

Asynchronous operations are done in coroutinevia yield from;

3. async/await

The new syntax introduced in Python 3.5 coroutinefor ;

• async, replace @asyncio.coroutine;
• await, replace yield from;

4. aiohttp

• asyncio, to achieve the agreement of TCP, UDP, SSLand so on;
• aiohttp, based on the asyncioimplemented HTTPframework;

1. Installation

$ pip install aiohttp

2. Examples

import asyncio
from aiohttp import web


async def index(request):
    await asyncio.sleep(1)
    return web.Response(body=b'<h1>Index</h1>')


async def hello(request):
    await asyncio.sleep(1)
    text = '<h1>hello, %s!</h1>' % request.match_info['name']
    return web.Response(body=text.encode('utf-8'))


async def init(loop):
    app = web.Application(loop=loop)
    app.router.add_route('GET', '/', index)
    app.router.add_route('GET', '/hello/{name}', hello)
    # 利用 asyncio 创建 TCP 服务
    srv = await loop.create_server(app.make_handler(), '', 8000)
    print('server started at http://localhost:8000...')
    return srv


loop = asyncio.get_event_loop()
loop.run_until_complete(init(loop))
loop.run_forever()

---

• Previous: "Python Basics" Web Application Programming
• Column: "Python Basics"

PS: Welcome friends from all walks of life 阅读, 评论thank you friends 点赞, 关注, 收藏!