Use concurrent modules

concurrent module

Process pool and thread pool

Pool of functional limitations number of processes or threads.

When the maximum number of concurrent processes or threads need much, much larger than the operation can bear,

Using a thread pool or the pool control program processes the generated number of processes or threads to prevent the operating system from too much pressure resulting in server downtime

futures following two modules are in the file folder concurrent folder under the file py

from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor

Process pool

from concurrent.futures import ProcessPoolExecutor
import time

def run():
    print(123)
    time.sleep(0.5)
    print(234)
    return "我是最终结果"
def call_back(future):
    print(future.result())

if __name__ == '__main__':
    pool = ProcessPoolExecutor(3)
    for i in range(10):
        future = pool.submit(run)
        future.add_done_callback(call_back)

    pool.shutdown()
    print("用了shutdown以后我最后在打印")

ProcessPoolExecutor

pool = ProcessPoolExecutor(n)

• The number of open process pool, pool process set up process is n

future = pool.submit(task,i)

• The task into the process pool
• task-- task name
• i-- task parameters
• future-- generated task objects

pool.shutdown()

• Wait for the process in the pool all tasks are completed in the code execution

future.result()

• Get the task return value
• Note that using this method blocks the thread pool tasks performed live, the task becomes serial
• solution
  • Save each task using the list of objects together,
    and finally unified remove the return value from the list
  • future.add_done_callback(parse)
    • Callback
    • When the end of the thread when the object will automatically parse the callback function to execute

Thread Pool

from concurrent.futures import ThreadPoolExecutor
import time

def run():
    print(123)
    time.sleep(4)
    print(234)
def call_back(future):
    print(future.result())

if __name__ == '__main__':
    pool = ThreadPoolExecutor(3)
    for i in range(10):

        future = pool.submit(run)
        future.add_done_callback(call_back)

    pool.shutdown()
    print("用了shutdown以后我最后在打印")

ThreadPoolExecutor

pool = ThreadPoolExecutor(n)

• Open thread pool, provided the number of threads the thread pool is n

future = pool.submit(task,i)

• The tasks into the thread pool
• task-- task name
• i-- task parameters
• future-- generated task objects

pool.shutdown()

• Wait for the process in the pool all tasks are complete, then execution of the code

future.result()

• Get the task return value
• Note that using this method blocks the thread pool tasks performed live, the task becomes serial
• solution
  • Save each task using the list of objects together,
    and finally unified remove the return value from the list
  • future.add_done_callback(parse)
    • Callback
    • When the end of the thread when the object will automatically parse the callback function to execute

Synchronous, asynchronous, blocking and non-blocking

Synchronize

• That is, when issuing a function call, until no results, the call will not return

asynchronous

• When an asynchronous procedure call is issued, the caller can not immediately get results.
• The actual processing of the call after completion member, to inform the caller via a state, and a notification callback.

Clog

• Blocking call refers to the results before the call returns, the current thread is suspended. Function will return only after getting results.

Non-blocking

• Prior to refer not get the results immediately, the function does not block the current thread, and will return immediately.

XMind: ZEN - Trial Version