python3 multi-threaded application

Python3 thread

Big pile of data to be processed, accelerate the efficient use of multi-threading can save computation time.

Multithreading Basics

• threading.active_count () the number of currently active threads

• threading.enumerate () to enumerate the currently running multi-threaded

• threading.current_thread () the current process is running which thread

• Basic Usage

  target  ： 指定 这个线程去哪个函数里面去执行代码

  args：     指定将来调用 函数的时候   传递什么数据过去

                  args参数指定的一定是一个元组类型
 name属性是命名多线程的名字 
 start()是开始执行多线程
 join()是线程结束后再执行后来的语句才会用到

Sample code:

thread = threading.Thread(target=thread_job,)
thread.start()

thread_job output is a function name of the currently running thread, complete code.

import threading

#def main():
#    print(threading.active_count())
#    print(threading.enumerate()) # see the thread list
#    print(threading.current_thread())

def thread_job():
    print('This is a thread of %s' % threading.current_thread())

def main():
    thread = threading.Thread(target=thread_job,)
    thread.start()
if __name__ == '__main__':
    main()

join application

Want to wait for all multi-threaded operation is completed and then showed all done\nonly need to use the join statement.

import threading
import time
def thread_job():
    print('T1 start\n')
    for i in range(10):
        time.sleep(0.1)
    print('T1 finish\n')

def T2_job(temp):
    print('T2 start\n')
    print("-----in test2 temp=%s-----"% str(temp))
    print('T2 finish\n')
    
def main():
    added_thread = threading.Thread(target=thread_job, name='T1')
    thread2 = threading.Thread(target=T2_job, args=(g_nums,), name='T2')
    added_thread.start()
    thread2.start()
    thread2.join()
    added_thread.join()

    print('all done\n')

if __name__ == '__main__':
    main()

Queue application

Multithreading is no return value of the result, all the threads can be placed in a long queue, the computation results and then taken out.

import threading
import time
from queue import Queue

def job(l,q):                # 执行函数
    for i in range(len(l)):
        l[i] = l[i]**2
    q.put(l)                 # 把结果加入到队列中

def multithreading():
    q = Queue()
    threads = []
    data = [[1,2,3],[3,4,5],[4,4,4],[5,5,5]]
    for i in range(4):       # 定义4个线程
        t = threading.Thread(target=job, args=(data[i], q))
        t.start()
        threads.append(t)
    for thread in threads:
        thread.join()        # 结束后再执行接下来的内容
    results = []
    for _ in range(4):       # 从4个线程里面得到结果，按顺序拿出结果
          results.append(q.get())
    print(results)

if __name__ == '__main__':
    multithreading()

GIL

Multithreading on Python but not really a quick switch between multiple threads.

import threading
from queue import Queue
import copy
import time

def job(l, q):
    res = sum(l)
    q.put(res)

def multithreading(l):
    q = Queue()
    threads = []
    for i in range(4):
        t = threading.Thread(target=job, args=(copy.copy(l), q), name='T%i' % i)
        t.start()
        threads.append(t)
    [t.join() for t in threads]
    total = 0
    for _ in range(4):
        total += q.get()
    print(total)

def normal(l):
    total = sum(l)
    print(total)

if __name__ == '__main__':
    l = list(range(1000000))
    s_t = time.time()
    normal(l*4)
    print('normal: ',time.time()-s_t)
    s_t = time.time()
    multithreading(l)
    print('multithreading: ', time.time()-s_t)

lock

The first thread is finished with the result, and then a second time with a thread. Lock will be used, when the first thread, the second thread is temporarily unable to operate. After ranking one thread is finished before they can do next.

A is a global variable, when the two threads of the same variable operation. Then locked by another thread at the time of the operation A. When the operation is completed and then let the second thread operations.

import threading

def job1():
    global A, lock     # 开始锁线程
    lock.acquire()
    for i in range(10):
        A += 1
        print('job1', A)
    lock.release()     # 释放锁

def job2():
    global A, lock
    lock.acquire()
    for i in range(10):
        A += 10
        print('job2', A)
    lock.release()

if __name__ == '__main__':
    lock = threading.Lock()
    A = 0
    t1 = threading.Thread(target=job1)
    t2 = threading.Thread(target=job2)
    t1.start()
    t2.start()
    t1.join()
    t2.join()

reference

https://www.bilibili.com/video/av16944429/?p=5

https://github.com/MorvanZhou/tutorials/tree/master/threadingTUT