The asynchronous context manager asyncio

Asynchronous context manager

The previous article we mentioned the context manager , but this only applies to the context manager synchronization code, the code can not be used for asynchronous (async def form), but do not worry Today we'll discuss how to use context managers in asynchronous.
Reminded Python version used in this tutorial is Python3.7.

async with

Asynchronous context manager. Similar to the synchronization context manager, we know that can be achieved with the use of a context management, while asynchronous context manager for the fundamental form of expression is async with, the following piece of code to tell you how async with works.

import asyncio
class AContext:
    def __init__(self):
        print("in init")

    async def __aenter__(self):
        print("in aenter")
    async def __aexit__(self, exc_type, exc_val, exc_tb):
        print("in aexit")

async def main():
    async with AContext() as ac:
        print("in with", ac)

if __name__ == '__main__':
    print("start")
    asyncio.run(main())

Output content

start
in init
in aenter
in with None
in aexit

Below that at different places and with async with the
grammar, with two methods to achieve the enter and exit, async with to achieve a similar method
aenter and aexit add a synchronization on the basis, in fact, it is to represent asynchronous.
The implementation, use common functions can be achieved with, but async with the need to achieve in the form of asynchronous functions, like the example above.

asynccontextmanager

From Python 3.7, there are two ways to write asynchronous context manager. One is to achieve the aforementioned magic function, another is contextlib another module asynccontextmanager. A context manager implemented through asynchronous manner decorator

import asyncio
from contextlib import asynccontextmanager
from concurrent.futures.thread import ThreadPoolExecutor


class AsyncFile(object):
    def __init__(self, file, loop=None, executor=None):
        if not loop:
            loop = asyncio.get_running_loop()  # 获取当前运行事件循环
        if not executor:
            executor = ThreadPoolExecutor(10)  # 线程池数量10
        self.file = file
        self.loop = loop
        self.executor = executor
        self.pending = []
        self.result = []

    def write(self, string):
        """
        实现异步写操作
        :param string: 要写的内容
        :return:
        """
        self.pending.append(
            self.loop.run_in_executor(
                self.executor, self.file.write, string,
            )
        )

    def read(self, i):
        """
        实现异步读操作
        :param i:
        :return:
        """
        self.pending.append(
            self.loop.run_in_executor(self.executor, self.file.read, i,)
        )

    def readlines(self):
        self.pending.append(
            self.loop.run_in_executor(self.executor, self.file.readlines, )
        )

@asynccontextmanager
async def async_open(path, mode="w"):
    with open(path, mode=mode) as f:
        loop = asyncio.get_running_loop()
        file = AsyncFile(f, loop=loop)
        try:
            yield file
        finally:
            file.result = await asyncio.gather(*file.pending, loop=loop)

By running the above code using a thread in asyncio run_in_executor, so that the blocking operation to the operation becomes non-blocking, non-blocking asynchronous to achieve the purpose.
AsyncFile class provides methods that will be used to add write, read and call readlines to the pending list. These tasks are scheduled by the event loop ThreadPoolExecutor finally block.
It represents the first half to yield __aenter __ ()
half of the yield used to indicate __aexit __ ()
for later use after the link can guarantee finally finished using the resources can be closed.

Run asynchronous context manager

If you call the previous example asynchronous context manager, async with the keywords you need to use to make the call. Also with async with the statement can only be used in asynchronous function.

from asynciodemo.asyncwith import async_open
import asyncio
import tempfile
import os


async def main():
    tempdir = tempfile.gettempdir()
    path = os.path.join(tempdir, "run.txt")
    print(f"临时文件目录:{path}")

    async with async_open(path, mode='w') as f:
        f.write("公众号: ")
        f.write("Python")
        f.write("学习")
        f.write("开发")


if __name__ == '__main__':
    asyncio.run(main())

Using similar methods and with by using as, then use its handle, only caveat is to use asynchronous function.

Synchronization task

In some previous asynchronous tutorials and everyone said on several synchronization method of coordination process, asyncio.wait and asyncio.gather, where we can complement these methods to synchronize asynchronous tasks by context manager, look at the following code

import asyncio


# 同步挂起协程

class Sync():
    def __init__(self):
        self.pending = []
        self.finished = None

    def schedule_coro(self, coro, shield=True):
       #如果去掉asyncio.shield，在取消fut函数的时候，就会导致coro协程也出错。
        fut = asyncio.shield(coro) if shield else asyncio.ensure_future(coro)
        self.pending.append(fut)
        return fut

    async def __aenter__(self):
        return self

    async def __aexit__(self, exc_type, exc_val, exc_tb):
        # 退出async with的时候，任务列表中的任务进行并发操作。
        self.finished = await asyncio.gather(*self.pending, return_exceptions=True)


async def workload1():
    await asyncio.sleep(2)
    print("These coroutines will be executed return 41")
    return 41


async def workload2():
    await asyncio.sleep(2)
    print("These coroutines will be executed return 42")
    return 42


async def workload3():
    await asyncio.sleep(2)
    print("These coroutines will be executed return 43")
    return 43


async def main():
    async with Sync() as sync:
        # 使用异步上下文可以创建同步协程程序
        sync.schedule_coro(workload1())
        sync.schedule_coro(workload2())
        sync.schedule_coro(workload3())
    print("All scheduled corotines have retuned or throw:", sync.finished)


if __name__ == '__main__':
    asyncio.run(main())

Export

These coroutines will be executed return 41
These coroutines will be executed return 42
These coroutines will be executed return 43
All scheduled corotines have retuned or throw: [41, 42, 43]

1. The form of the program is synchronized concurrent output.

1. Schedule_coro role is to co-drive workload1, workload2, workload3 added to the list of tasks pending, exit async with time, task list task concurrent operation.