python coroutine learning

1. What is coroutine and how to implement it

1.1 Coroutine

Also known as micro-threads, fibers, also known as user-level threads, they complete multiple tasks without opening up threads, that is, they complete multiple tasks in a single thread, and multiple tasks are executed alternately in a certain order.

1.2 Implementation method

greenlet, early module;
yield keyword;
asyncio decorator (py3.4);
async, await keywords (py3.5) (recommended);

2. Examples of implementation methods

2.1 greenlet

from greenlet import greenlet
def func1():
    print(1)        # 第1步: 输出 1
    gr2.switch()    # 第3步: 切换到 func2 函数
    print(2)        # 第6步: 输出 2
    gr2.switch()    # 第7步: 切换到 func2 函数，从上一次执行的位置继续向后执行

def func2() :
    print(3)        # 第4步: 输出 3
    grl.switch()    # 第5步: 切换到 func1 函数，从上一次执行的位置继续向后执行
    print(4)        # 第8步: 输出 4

gr1 = greenTet(func1)
gr2 = greenTet(func2)

grl.switch()        # 第1步: 去执行 func1 函数

2.2 yield keyword

import time

def work1():
    while True:
        print("----work1---")
        yield
        time.sleep(0.5)

def work2():
    while True:
        print("----work2---")
        yield
        time.sleep(0.5)

def main():
    w1 = work1()
    w2 = work2()
    while True:
        next(w1)
        next(w2)

if __name__ == "__main__":
    main()

2.3 asyncio decorator (py3.4)

import asyncio
@asyncio.coroutine
def func1():
    print(1)
    yield from asyncio.sleep(2)  # 遇到IO耗时操作，自动化切换到tasks中的其他任务
    print(2)

@asyncio.coroutine
def func2():
    print(3)
    yield from asyncio.sleep(2)  # 遇到IO耗时操作，自动化切换到tasks中的其他任务
    print(4)

tasks = [
    asyncio.ensure_future( func1()),
    asyncio.ensure_future( func2())
]
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait(tasks))

2.4 async, await keywords (py3.5)

import asyncio

async def func1():
    print(1)
    await asyncio.sleep(2)  # 遇到IO耗时操作，自动化切换到tasks中的其他任务
    print(2)

async def func2():
    print(3)
    await asyncio.sleep(2)  # 遇到IO耗时操作，自动化切换到tasks中的其他任务
    print(4)

tasks = [
    asyncio.ensure_future( func1()),
    asyncio.ensure_future( func2())
]
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait(tasks))

3. The meaning of coroutine

If a thread encounters IO waiting time (such as local disk data reading and network data request), the thread will not wait and use the idle time to do other things.

Case: Download three pictures (network IO)

Ordinary way (that is, synchronous programming):

import requests

def download_image(url):
    print("开始下载:", url)  #发送网络请求，下载图片
    response = requests.get(url)
    print("下载完成")
    # 图片保存到本地文件
    file_name = url.rsplit('_')[-1]
    with open(file_name, mode='wb') as file_object:
        file_object.write(response.content)


if __name__ == "_main__":
    url_list = [
    "https://www3.autoimg.cn/newsdfs/g26/M02/35/A9/120x90_0_autohomecar_ChsEe12AX06A0OH_AAFocMs8nzu621.jpg",
    "https://www2.autoimg.cn/newsdfs/g30/M01/3C/E2/120x90_0_autohomecar_chcCSV2BBICAUntfAADjJFd6800429.jpg",
    "https://www3.autoimg.cn/newsdfs/g26/M0B/3C/65/120x90_0_autohomecar_ChcCP12BFCmAIO83AAGq7vK0sGY193.jpg"
    ]
    for item in url_list:
        download_image(item)

Coroutine solution (that is, asynchronous programming):

import aiohttp
import asyncio

async def fetch(session, url):
    print("发送请求:", url)
    async with session.get(url, verify_ssl=False) as response:
        content = await response.content.read()
        file_name = url.rsplit('_')[-1]
        with open(file_name, mode="wb") as file_object:
            file_object.write(content)

async def main():
    async with aiohttp.ClientSession() as session:
        url_list =["https://ww3.autoimg.cn/newsdfs/g26/M02/35/A9/120x90_0_autohomecar_ChsEe2AX06A00H_AAFOCHs8nzU621.jpg",
                   "https://ww2.autoimg.cn/newsdfs/g30/M01/3C/E2/120x90_0_autohomecar_ChcCSV288ICAUntfAADjJFd6800429.jpg",
                   "https://wm3.automg,cn/newsdfs/g26/M0B/3C/65/120x90_0_autohomecar_ChcCP12BFCmA083AAGq7VK0sGY193.jpg"
        ]
        tasks = [asyncio.create_task(fetch(session, url)) for url in url_list]
        await asyncio.wait(tasks)

if __name__ == "__main__":
    asyncio.run(main())

4. Get started quickly

4.1 Basic syntax

Coroutine function, modified with async when defining the function, as follows

async def func():
    pass
result = func()  # 获取协程对象

Note that when executing a coroutine function to create a coroutine object, the code inside the function will not be executed; if you want to run the code inside the coroutine function, you must hand over the coroutine object to the event loop for processing. as follows

import asyncio

async def func():
    print("快来搞我吧!")

result = func()

# ss1oop = asyncio.get_event_loop()
# ss1oop.run_until_complete( result )

asyncio.run( result ) # python3.7

4.2 await

Example one:

await + awaitable object (coroutine object, Future, Task object)

import asyncio

async def func():
    print("快来搞我吧!")
    response = await asyncio.sleep(2)
    print("结束", response)

result = func()

ss1oop = asyncio.get_event_loop()
ss1oop.run_until_complete( result )

#asyncio.run( result ) # python3.7

4.3 Task object

Add multiple tasks in the event loop.

Tasks are used to concurrently schedule coroutines. Task objects are created through asyncio.create_task (coroutine object), which allows coroutines to join the event loop and wait to be scheduled for execution. In addition to using the asyncio.create_task() function, you can also use the low-level loop.create_task0 or ensure_future() functions. It is not recommended to instantiate Task objects manually. Note: The
asyncio.create_task() function was added in Python 3.7. In Python 3.7 Previously, you could use the low-level asyncio.ensure_future() function instead.

Example one: