Use asyncio to implement multi-threaded asynchronous multi-tasking interface operations

Preface

Recently, there is a project that requires asynchronous multi-tasking requests initiated from the window client. In order to prevent long-term background operations from blocking the foreground operations, multi-threads need to be initiated to isolate the UI main interface event loop and the asyncio background asynchronous task event loop.

Introduction to support libraries

1. Asynchronous coroutine library asyncio

asyncioUse coroutines to implement event loops to achieve the effect of asynchronous tasks.

pip install asyncio

Coroutines use @asyncio.coroutinedecorators to mark asynchronous tasks and wait yield fromfor the asynchronous tasks to complete. in

@asyncio.coroutine === async
yield from === await

Since the keyword was introduced async/await, the code implementation has become much more elegant.

2. Interface library wxPython

wxPython is a cross-platform GUI toolkit for the Python programming language. It allows Python programmers to simply and easily create programs with powerful and powerful graphical user interfaces. It is implemented as a set of Python extension modules that wrap the GUI components of the popular wxWidgets cross-platform library, which is written in C++.

pip install wxpython

Especially in windowsthe following , many unique and cool window effects can be achieved. For details, you can run wxDemothe example for reference.

# 运行 python目录下/Scripts/wxdemo.exe
C:\ProgramData\Anaconda3\envs\ctrip_code36\Scripts>wxdemo.exe

Multitasking coroutine

1. Create an asynchronous task

We first simply create an asynchronous task, randomize the task duration, and record the task start time and end time.

async def AsyncTask(self, name):
    t = random.randint(1, 5)
    self.log('Start %s: %.8f, duration %ds' % (name, time.time() - self.start, t))
    await asyncio.sleep(t)  # 模拟长时间后台操作
    self.log('End %s: %.8f' % (name, time.time() - self.start))
    return name

async keyword to define an asynchronous function, awaitwait for the operation to return. Used here asyncio.sleep()to simulate real IOoperations.

Set a callback function to be triggered after the asynchronous task ends.

def task_callback(self, task):
    time.sleep(1)
    self.log('Callback %s: %.8f' % (task.result(), time.time() - self.start))

2. Build window interface

Draw a simple window that contains a button to trigger an asynchronous task and a static text to display the current task status.

class MyFrame(wx.Frame):
    def __init__(
            self, parent, ID, title, pos=wx.DefaultPosition,
            size=wx.DefaultSize, style=wx.DEFAULT_FRAME_STYLE
    ):
        wx.Frame.__init__(self, parent, ID, title, pos, size, style)
        panel = wx.Panel(self, -1)

        self.start = time.time()  # 记录初始时间戳

        btn = wx.Button(panel, -1, "启动异步任务", size=(100, 30))
        btn.SetPosition((20, 20))

        txt = wx.StaticText(panel, -1, "任务未运行", (200, 40))
        txt.SetPosition((30, 70))
        self.txt = txt

        self.Bind(wx.EVT_BUTTON, self.OnButton, btn)
        self.Bind(wx.EVT_CLOSE, self.OnCloseWindow)

3. Create thread

Obtain asynciothe event loop, initiate an independent thread, and isolate the main interface events

def OnButton(self, evt):
    async_loop = asyncio.get_event_loop()
    if not async_loop.is_running():
        threading.Thread(target=self._asyncio_thread, args=(async_loop,)).start()
    return

4. Create the main task

Set up a main task, which contains an independent subtask and a set of task1concurrent subtasks :task2task3

First the main task Master taskstarts;
A 5-second subtask is started task1;
task2After completion, the 5-second subtask and the 4-second subtask were started at the same time task3, and the callback function was set;
Then task3the task time is short, it is completed first, and the task callback is triggered;
Then task2it is completed and the corresponding callback is executed;
Finally Master taskexit.

def _asyncio_thread(self, async_loop):
    self.log('Start %s: %.8f' % ('Master task', time.time() - self.start))
    asyncio.set_event_loop(async_loop)

    print('异步单任务', 'task1')
    task = asyncio.ensure_future(self.AsyncTask('task1'))
    async_loop.run_until_complete(task)

    print('异步多任务并发', 'task2', 'task3')
    tasks = []
    task = asyncio.ensure_future(self.AsyncTask('task2'))
    task.add_done_callback(self.task_callback)
    tasks.append(task)  # 将多个任务对象装在到一个任务列表中

    task = asyncio.ensure_future(self.AsyncTask('task3'))
    task.add_done_callback(self.task_callback)
    tasks.append(task)

    async_loop.run_until_complete(asyncio.wait(tasks))

    self.log('End %s: %.8f' % ('Master task', time.time() - self.start))
    return

To summarize the process:

Get asynciothe event loop async_loop = asyncio.get_event_loop();
Launch a thread and async_loopinject it into the child thread;
Set up an event loop in the sub-thread and isolate the main interface event loopasyncio.set_event_loop(async_loop);
Use to asyncio.ensure_futureconfigure asynchronous tasks;
Use add_done_callbackset callback function;
Use to async_loop.run_until_completeexecute asynchronous functions.

Concurrent multi-tasking coroutines

If there are no dependencies between subtasks, concurrent execution can be used to greatly improve the running efficiency of the program.

1. Create multiple asynchronous tasks

def OnButton(self, evt):
    async_loop = asyncio.get_event_loop()
    if not async_loop.is_running():
        t = threading.Thread(target=self._asyncio_thread, args=(async_loop,))
        t.start()

    for i in range(5):
        asyncio.run_coroutine_threadsafe(self.AsyncTask('task%d' % i), async_loop)

2. Create thread

def _asyncio_thread(self, async_loop):
    asyncio.set_event_loop(async_loop)
    async_loop.run_forever()

3. Exit the event loop

It is worth noting that since the thread in the thread async_loopis run_forever()started, when closing the program, call_soon_threadsafe(async_loop.stop)corresponding cleaning operations need to be performed to ensure that the program exits smoothly.

def OnCloseWindow(self, evt):
    async_loop = asyncio.get_event_loop()
    async_loop.call_soon_threadsafe(async_loop.stop)
    self.Destroy()

Source code download

The complete source code of this issue can be found on the public account "Deep Awakening" and reply: "asyncio" in the background to get the download link.

PS: When coding late at night, a coder is essentially a creature that converts coffee into code. However, my body cannot handle drinking too much coffee, so I recently gave up coffee and became obsessed with grapefruit tea.

I recommend the honey grapefruit tea from Hengshoutang. Buy one and get one free. It tastes better than the Korean one. By the way, I support domestic products.