Python asynchronous async / await concept, usage (1)

• IO programming and synchronous and asynchronous

  Synchronous and asynchronous, blocking and non-blocking are all concepts around IO.

  Although the multi-thread and multi-process models solve the concurrency problem, the overhead of switching threads in the system is also very large.

  Another solution to the IO problem is asynchronous IO. When the code needs to perform a time-consuming IO operation, it only issues IO instructions, does not wait for the IO result, and then executes other code. After a period of time, when the IO returns the result, it notifies the CPU to process again.

  During the period from "Issuing an IO request" to receiving "IO completion", under the synchronous IO model, the main thread can only be suspended (giving up CPU time slices), but under the asynchronous IO model, the main thread does not rest Instead, continue to process other messages in the message loop. In this way, under the asynchronous IO model, one thread can process multiple IO requests at the same time, and there is no operation to switch threads .

• IO programming

  The core of the computer is the CPU. The programs and data calculated by the CPU can only be in the memory. Therefore, the elements required at the time of the CPU operation must already be in the memory. For those that are not in the memory, you need to load the input memory. , Data that is not used after calculation needs to be output to disk.

  Because the CPU calculation speed is much higher than the I / O speed, there is a subdivision area of ​​I / O programming.

  The CPU can only run one process at a time.

  Under these conditions, there are several programs for I / O programming:

  1. Let the CPU wait for the I / O to complete before continuing to execute subsequent code: called synchronization
  2. CPU does not wait for I / O, to do other things: called asynchronous

  The difference between synchronous and asynchronous is whether the CPU waits for the result of IO execution.

  The ability to operate IO is ultimately realized by the operating system, and various languages ​​encapsulate the interface provided by the operating system.

  The process of programming variables that can be stored or transferred from memory is called serialization, and it is called pickling in Python , which is what the module does.pickle

• Subroutine (also called function)

  It is a function in the traditional sense:

  • Called hierarchically in all languages;
  • The subroutine call is realized through the stack;
  • A thread is to execute a subroutine;
  • The call of the subroutine is always one entry and one return, and the calling sequence is clear

• Coroutine

  Also known as microthreads, fibers.

  The coroutine looks like a subroutine, but during the execution, it can be interrupted inside the subroutine and turn to execute another subroutine.

  The interrupt and other subroutines mentioned here are not function calls .

  def A():
      print('1')
      print('2')
      print('3')
  
  def B():
      print('x')
      print('y')
      print('z')
      
  # 在协程下输出结果可能是
  1
  2
  x
  y
  3
  z
  # 关键点在于，这里并不是A调用B
  

  Not a function call, the result looks like multiple threads, but the coroutine is executed in one thread.

  Subroutines are a special case of coroutines.

  Compared with multithreading:

  • Coroutine execution efficiency is high, because subprogram switching is not thread switching, there is no thread switching overhead
  • No need for multi-threaded locking mechanism

• Coroutines utilize multi-core CPUs

  Multiple sub-functions are executed in the coroutine and are executed in the same thread.

  Want to make full use of multi-core CPU, the method is multi-process + coroutine.

• Python and coroutines

  Python 's support for coroutines is achieved through generator.

  Involving generator, next, yeildseveral concepts.

• The difference between asyncio and async / await

  asyncio ( official document ) is a standard library introduced in Python 3.4 that supports asynchronous IO , using syntax;yeild

  async / await is Python3.5 simplify the code introduced new syntax is Python3.3 in yield fromand Python3.4 are asynciocombined to produce the new syntax. It is used to solve the problem that the relationship between the yield generator and the asynchronous can not be effectively distinguished in the asynchronous programming of Python .

  Need to add preliminary knowledge:

  " Yield Concept and Usage Understanding in Python "

• Reference

1. Liao Xuefeng's official website
2. How does async and await work in Python 3.5?