async achieve coroutine, asynchronous programming
We all know that the server development priority for control of the IO scheduling no longer rely on the system, and hope that the use of coroutines efficient way of concurrent tasks, such as js, lua, etc. in an asynchronous coroutine They have made very strong .
python version 3.4 also joined in the concept of coroutines, and 3.5 establishes the basic syntax and perfect implementation. 3.6 was carried out at the same time also await and optimizing the yield is released as a function of the body in the same restrictions.
asyncio python3.4 version is introduced into the standard library, python2x not add this library, after all, is the future python3x ah, ha ha! python3.5 joined the async / await characteristics.
Before learning asyncio, must first figure out the concept of synchronous / asynchronous :
- event_loop event loop: start a program infinite loop, the programmer will register some functions to the event loop. When satisfied incident, call the appropriate coroutine function.
- coroutine coroutine: coroutine object refers to a function defined using async keyword, it will not execute the function call immediately, but will return a coroutine object. Coroutine objects need to be registered to the event loop, called by the event loop.
- Task task: a coroutine object is a native function may be suspended, the task is further encapsulated to coroutine, various states of the tasks contained therein.
- future: a guide to future results or perform tasks not performed. There is no essential difference between it and the task
- async / await keywords: python3.5-defined keywords for coroutines, async definition of a coroutine, await for suspending blocking asynchronous call interface.
We also mentioned above, Task, which is further encapsulated to coroutine object, objects inside it coroutine more compared operating state, such as running, finished, and we can use these states to obtain coroutine implementation object.
1, create a coroutine
First define a coroutine, DEF added async declared before, can define a coroutine function.
A coroutine function can not call directly run, only the coroutine added to the loop in the event loop. asyncio.get_event_loop method can create an event loop, and then use the coroutine run_until_complete registered to the event loop, and start the event loop.
E.g:
import asyncio
async def func(a):
print('leiting':a)
corouine = func(1)
loop = asyncio.get_event_loop()
loop.run_until_complete(func())
In the above example, when we coroutine object to run_until_complete () method, in fact it is to perform an operation coroutine packaged in a task object, we can explicitly declared as follows:
import asyncio
async def execute(x):
print('Number:', x)
return x
coroutine = execute(1)
print('Coroutine:', coroutine)
print('After calling execute')
loop = asyncio.get_event_loop()
task = loop.create_task(coroutine)
print('Task1:', task)
#当我们将 coroutine 对象传递给 run_until_complete() 方法的时候，实际上它进行了一个操作就是将 coroutine 封装成了 task 对象，我们也可以显式地进行声明
loop.run_until_complete(task)
print('Task:', task)
print('After calling loop')
结果
#Coroutine: <coroutine object execute at 0x0000017A398CB3C8>
#After calling execute
#Task1: <Task pending coro=<execute() running at D:/Python/项目位置/test.py:17>>
#Number: 1
#Task: <Task finished coro=<execute() done, defined at D:/Python/项目位置/test.py:17> result=1>
#After calling loopHere we define a loop after an object, then calls its create_task () method to task coroutine object into an object, and then we print out and found it was pending state. Then we will add a task object to the event loop to be implemented, then we'll look at the printout task object, find its state becomes a finished, but also can see the result becomes 1, that is, execute our definition () method returns the result.
In addition there is a way to define the task object, that is, directly through the asyncio 的 ensure_future()method, the result is returned task object, so we can not be defined by means of the loop, even though we have not declared loop can be well defined task objects in advance, worded as follows:
import asyncio
async def execute(x):
print('Number:', x)
return x
coroutine = execute(1)
print('Coroutine:', coroutine)
print('After calling execute')
task = asyncio.ensure_future(coroutine)
print('Task1:', task)
loop = asyncio.get_event_loop()
loop.run_until_complete(task)
结果
#Coroutine: <coroutine object execute at 0x0000016A2C34B3C8>
#After calling execute
#Task1: <Task pending coro=<execute() running at D:/Python/项目位置/test.py:17>>
#Number: 1
#Task: <Task finished coro=<execute() done, defined at D:/Python/项目位置test.py:17> result=1>
#After calling loopIt found that the effect is the same.
Bind callbacks
(1) calls add_done_callback () method to bind a callback method for a task. We will callback () method to pass the task encapsulated objects, so when the task is finished you can call the callback () method, and while the task object also passed as a parameter to the callback () method, call the result task object ( ) method can get the results returned
import asyncio
import requests
async def request():
url = 'https://www.baidu.com'
status = requests.get(url)
status = status.text
return status
def callback(task):
print('Status:', task.result())
coroutine = request()
task = asyncio.ensure_future(coroutine)
task.add_done_callback(callback)
print('Task：', task)
loop = asyncio.get_event_loop()
loop.run_until_complete(task)
print('Task:', task)Call the result directly after (2) direct call task has finished running () method to get results
import asyncio
import requests
async def request():
url = 'https://www.baidu.com'
status = requests.get(url)
return status
coroutine = request()
task = asyncio.ensure_future(coroutine)
print('Task:', task)
loop = asyncio.get_event_loop()
loop.run_until_complete(task)
print('Task:', task)
print('Task Result:', task.result())
#Task: <Task pending coro=<request() running at D:/Python/项目位置/test.py:53>>
#Task: <Task finished coro=<request() done, defined at D:/Python/项目位置/test.py:53> result=<Response [200]>>
#Task Result: <Response [200]>3, multi-task coroutine
The definition of a task list, and then use asyncio the wait () method to perform; we use a for loop created five task, make up a list, then the list is first transferred to the asyncio the wait () method, and then to register the time cycle can be initiated five task. Finally, we run out and then output the results of the task
import asyncio
import requests
async def request():
url = 'https://www.baidu.com'
status = requests.get(url)
return status
tasks = [asyncio.ensure_future(request()) for _ in range(5)]
print('Tasks:', tasks)
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait(tasks))
for task in tasks:
print('Task Result:', task.result())
Task Result: <Response [200]>
Task Result: <Response [200]>
Task Result: <Response [200]>
Task Result: <Response [200]>
Task Result: <Response [200]>4, to achieve coroutine
(1) can be time-consuming to use await pending operations suspended to give up control. When coroutine execution encounters await, the cycle time will be this coroutine suspend, turn to the implementation of other coroutine until the other coroutine suspends or executed.
import asyncio
import requests
import time
start = time.time()
async def get(url):
return requests.get(url)
async def request():
url = 'https://www.baidu.com'
print('Waiting for', url)
response = await get(url)
print('Get response from', url, 'Result', response.status_code)
tasks = [asyncio.ensure_future(request()) for _ in range(5)]
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait(tasks))
end = time.time()
print('Cost time:', end - start)
Waiting for https://www.baidu.com
Get response from https://www.baidu.com Result 200
Waiting for https://www.baidu.com
Get response from https://www.baidu.com Result 200
Waiting for https://www.baidu.com
Get response from https://www.baidu.com Result 200
Waiting for https://www.baidu.com
Get response from https://www.baidu.com Result 200
Waiting for https://www.baidu.com
Get response from http
5, using aiohttp
aiohttp is a support library for asynchronous requests, and use it with our asyncio can very easily implement asynchronous request operation.
Here we will request library consists of requests changed aiohttp, get ClientSession method request by the class of aiohttp ()
import asyncio
import aiohttp
import time
start = time.time()
async def get(url):
session = aiohttp.ClientSession()
response = await session.get(url)
result = await response.text()
await session.close()
return result
async def request():
url = 'http://www.newsmth.net/nForum/#!mainpage'
print('Waiting for', url)
result = await get(url)
print('Get response from', url, 'Result:', result)
tasks = [asyncio.ensure_future(request()) for _ in range(5)]
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait(tasks))
end = time.time()
print('Cost time:', end - start)
We use the code inside await, followed by a get () method, in the implementation of the five co-process, if you encounter await, it will suspend the current coroutine instead to perform other coroutine until other coroutine or suspend or finished, and then execute the next coroutine.
Starts running, the cycle time will run the first task, for the first task, when executed to first await follow the get () method, which is pending, but the get () method first step the execution is non-blocking, suspended immediately after being awakened, so they immediately entered the implementation, ClientSession created objects, then met with the second await, called session.get () request method, and then was suspended, because requests require time-consuming for a long time, it has not been awakened, a good first task is suspended, next, how to do it? Event loop will look not currently suspended coroutine continue, so he turned to the implementation of the second task, the process is the same operation until after the execution of the session.get fifth task () method, all task have been hung up. All task have been in a suspended state, it is supposed to? I had to wait. After three seconds, several requests almost simultaneously have a response, and then wake up several task is also performed, the output result of the request, and finally takes 3 seconds!
how about it? This is the convenience offered by the asynchronous operation, when it comes to blocking operation, the task is suspended, then program to perform other tasks, rather than innocently waiting for, so you can make full use of CPU time, without having to waste time waiting on IO.