Reptile Day - concept

• Thread Pool

  • 导包：from multiprocessing.dummy import Pool
  • pool.map(callback,alist)
    • Can make asynchronous callback list alist elements in some form of operation
    • Note: callback must have a parameter,
  • Mainly be used in time-consuming operation

• + Single-threaded multi-tasking asynchronous coroutine

  • Special Functions

    • If the definition of a function to be modified async keyword, the function is a special function.
    • Special features:
      • After the internal function is called to realize the function statement is not executed immediately.
      • This function returns a coroutine objects

  • Coroutine:

    • Object. When the special function is called, the function returns a coroutine object.
    • Coroutine objects == Special Functions

  • The task object

    • Coroutine is further encapsulated object (coroutine is a high-level object)
    • The task object == == coroutine objects Special Functions
    • Bind callbacks:
      • task.add_done_callback(funcName)
      • funName The callback function must take a parameter, this parameter indicates the current task object is
        • Parameter .result (): is the current task special functions corresponding to the object represented by the return value

  • Object event loop:

    • Objects created event loop
      • asyncio.get_event_loop()
    • Task object needs to be registered with the event loop object and start the event loop
      • loop.run_until_complete(task)

  • Wait (await): After the end of the blocking operation so that loop back to the code after the execution blocked

  • Suspend (wait ()): the current task object to hand over the right to use the cpu

  • [Note] focus on:

    • In the special functions not appear to achieve internal module code does not support asynchronous, otherwise it will break the whole effect of asynchronous

  • aiohttp：

    • does not support asynchronous requests, it can not appear inside a special function.

    • aiohttp: supports asynchronous web request module

      • pip install aiohttp

    • Preparation of the code:

      • Write basic architecture
        with aiohttp.ClientSession () AS sess:
        #with sess.get / POST (url = url, headers = headers, the Data / params, Proxy = " HTTP: // ip: Port ") AS the Response:
        with sess .get (URL = URL) AS response:
        #text (): Get response data string
        #read (): Get the type of response data bytes
        page_text = response.text ()

                return page_text

      • Additional details

        • Before each with plus async
        • Before each blocking operation coupled with await keyword
          • Code referring to the complete code

    • The complete code:
      the async with aiohttp.ClientSession () AS sess:
      #with sess.get / POST (url = url, headers = headers, the Data / params, Proxy = " HTTP: // ip: Port ") AS the Response:
      the async with sess.get the await (URL = URL) AS response:
      #text (): Get response data string
      #read (): Get the type of response data bytes
      page_text = await response.text ()

       return page_text

• To parse text with html tags
  • bs4