Python multithreading

Multithreading is similar to executing multiple different programs at the same time. Multithreading has the following advantages:

Using threads can put tasks in the program that take up a long time to the background for processing.
The user interface can be more attractive, so for example, the user clicks a button to trigger the processing of certain events, and a progress bar can pop up to show the progress of the processing
Programs may run faster
In the realization of some waiting tasks, such as user input, file reading and writing, and network sending and receiving data, threads are more useful. In this case we can release some precious resources such as memory usage and so on.

Threads are still different from processes during execution. Each independent thread has an entry for program execution, a sequential execution sequence, and an exit for the program. However, threads cannot be executed independently, and must depend on the application program, and the application program provides multiple thread execution control.

Each thread has its own set of CPU registers, called the thread's context, which reflects the state of the CPU registers the last time the thread ran the thread.

The instruction pointer and stack pointer registers are the two most important registers in the thread context. The thread always runs in the context of the process, and these addresses are used to mark the memory in the address space of the process that owns the thread.

Threads can be preempted (interrupted).
Threads can be put on hold (also known as sleeping) for a while while other threads are running -- this is the thread's yield.

Start learning Python threads

There are two ways to use threads in Python: functions or classes that wrap thread objects.

Functional: call the start_new_thread() function in the thread module to generate a new thread. The syntax is as follows:

thread.start_new_thread ( function, args[, kwargs] )

Parameter Description:

function - the thread function.
args - Arguments passed to the thread function, it must be a tuple type.
kwargs - optional arguments.

Example (Python 2.0+)

#!/usr/bin/python # -*- coding: UTF-8 -*- import thread import time # 为线程定义一个函数 def print_time( threadName, delay): count = 0 while count < 5: time.sleep(delay) count += 1 print "%s: %s" % ( threadName, time.ctime(time.time()) ) # 创建两个线程 try: thread.start_new_thread( print_time, ("Thread-1", 2, ) ) thread.start_new_thread( print_time, ("Thread-2", 4, ) ) except: print "Error: unable to start thread" while 1: pass

The output of executing the above program is as follows:

Thread-1: Thu Jan 22 15:42:17 2009
Thread-1: Thu Jan 22 15:42:19 2009
Thread-2: Thu Jan 22 15:42:19 2009
Thread-1: Thu Jan 22 15:42:21 2009
Thread-2: Thu Jan 22 15:42:23 2009
Thread-1: Thu Jan 22 15:42:23 2009
Thread-1: Thu Jan 22 15:42:25 2009
Thread-2: Thu Jan 22 15:42:27 2009
Thread-2: Thu Jan 22 15:42:31 2009
Thread-2: Thu Jan 22 15:42:35 2009

The end of the thread generally depends on the natural end of the thread function; you can also call thread.exit() in the thread function, which throws a SystemExit exception to achieve the purpose of exiting the thread.

thread module

Python provides support for threads through two standard libraries, thread and threading. thread provides low-level, primitive threads and a simple lock.

Other methods provided by the threading module:

threading.currentThread(): Returns the current thread variable.
threading.enumerate(): Returns a list of running threads. Running refers to after the thread is started and before the end, excluding the thread before the start and after the end.
threading.activeCount(): Returns the number of running threads, has the same result as len(threading.enumerate()).

In addition to using methods, the thread module also provides the Thread class to handle threads. The Thread class provides the following methods:

run(): A method used to represent thread activity.
start(): Start thread activity.
join([time]): Wait until the thread terminates. This blocks the calling thread until the thread's join() method is called to abort - exit normally or throw an unhandled exception - or an optional timeout occurs.
isAlive(): Returns whether the thread is alive.
getName(): Returns the thread name.
setName(): Set the thread name.

Create threads using the Threading module

Use the Threading module to create a thread, inherit directly from threading.Thread, and then override the __init__ method and the run method:

Example (Python 2.0+)

#!/usr/bin/python # -*- coding: UTF-8 -*- import threading import time exitFlag = 0 class myThread (threading.Thread): #Inherit the parent class threading.Thread def __init__(self, threadID, name , counter): threading.Thread.__init__(self) self.threadID = threadID self.name = name self.counter = counter def run(self): #Write the code to be executed into the run function and the thread will directly Run the run function print "Starting " + self.name print_time(self.name, self.counter, 5) print "Exiting " + self.name def print_time(threadName, delay, counter): while counter: if exitFlag: (threading. Thread).exit() time.sleep(delay) print "%s: %s" % (threadName, time.ctime(time.time())) counter -= 1 # create a new thread thread1 = myThread(1, " Thread-1", 1) thread2 = myThread(2, "Thread-2", 2) # Start thread thread1.start() thread2.start() print "Exiting Main Thread"

The execution result of the above program is as follows;

Starting Thread-1
Starting Thread-2
Exiting Main Thread
Thread-1: Thu Mar 21 09:10:03 2013
Thread-1: Thu Mar 21 09:10:04 2013
Thread-2: Thu Mar 21 09:10:04 2013
Thread-1: Thu Mar 21 09:10:05 2013
Thread-1: Thu Mar 21 09:10:06 2013
Thread-2: Thu Mar 21 09:10:06 2013
Thread-1: Thu Mar 21 09:10:07 2013
Exiting Thread-1
Thread-2: Thu Mar 21 09:10:08 2013
Thread-2: Thu Mar 21 09:10:10 2013
Thread-2: Thu Mar 21 09:10:12 2013
Exiting Thread-2

thread synchronization

If multiple threads jointly modify a certain data, unpredictable results may occur. In order to ensure the correctness of the data, multiple threads need to be synchronized.

Simple thread synchronization can be achieved by using Lock and Rlock of the Thread object. Both objects have acquire and release methods. For data that needs to be operated by only one thread at a time, its operation can be placed between the acquire and release methods. between. as follows:

The advantage of multithreading is that you can run multiple tasks at the same time (at least it feels like that). But when threads need to share data, there may be a problem of data asynchrony.

Consider a situation where all elements in a list are 0, thread "set" changes all elements to 1 from back to front, and thread "print" is responsible for reading and printing the list from front to back.

Then, maybe when the thread "set" starts to change, the thread "print" will print the list, and the output will become half 0 and half 1, which is data asynchrony. To avoid this, the concept of locks is introduced.

A lock has two states - locked and unlocked. Whenever a thread such as "set" wants to access shared data, it must first obtain a lock; if another thread such as "print" has acquired the lock, then let the thread "set" pause, that is, block synchronously; wait until the thread" Print "After the access is completed, after releasing the lock, let the thread "set" continue.

After such processing, when printing the list, either all 0s or all 1s are output, and the embarrassing scene of half 0 and half 1 will no longer appear.

Example (Python 2.0+)

#!/usr/bin/python # -*- coding: UTF-8 -*- import threading import time class myThread (threading.Thread): def __init__(self, threadID, name, counter): threading.Thread.__init__( self) self.threadID = threadID self.name = name self.counter = counter def run(self): print "Starting " + self.name # Get the lock, return True after the lock is successfully acquired # When the optional timeout parameter is not filled in Will block until the lock is acquired # otherwise it will return False after timeout threadLock.acquire() print_time(self.name, self.counter, 3) # release the lock threadLock.release() def print_time(threadName, delay, counter): while counter : time.sleep(delay) print "%s: %s" % (threadName, time.ctime(time.time())) counter -= 1 threadLock = threading.Lock() threads = [] # Create a new thread thread1 = myThread(1, "Thread-1", 1) thread2 = myThread(2, "Thread-2", 2) # start new thread thread1.start() thread2.start() # add thread to thread list threads.append (thread1) threads.append(thread2) # Wait for all threads to finish for t in threads: t.join() print "Exiting Main Thread"

Thread priority queue (Queue)

Python's Queue module provides synchronous, thread-safe queue classes, including FIFO (first in first out) queue Queue, LIFO (last in first out) queue LifoQueue, and priority queue PriorityQueue. These queues all implement locking primitives and can be used directly in multiple threads. Queues can be used to achieve synchronization between threads.

Common methods in the Queue module:

Queue.qsize() returns the size of the queue
Queue.empty() returns True if the queue is empty, otherwise False
Queue.full() returns True if the queue is full, otherwise False
Queue.full corresponds to maxsize size
Queue.get([block[, timeout]]) to get the queue, timeout waiting time
Queue.get_nowait() 相当Queue.get(False)
Queue.put(item) write to the queue, timeout waiting time
Queue.put_nowait(item) 相当Queue.put(item, False)
Queue.task_done() After completing a job, the Queue.task_done() function sends a signal to the queue that the task has completed
Queue.join() actually means wait until the queue is empty before doing anything else

Example (Python 2.0+)

#!/usr/bin/python # -*- coding: UTF-8 -*- import Queue import threading import time exitFlag = 0 class myThread (threading.Thread): def __init__(self, threadID, name, q): threading.Thread.__init__(self) self.threadID = threadID self.name = name self.q = q def run(self): print "Starting " + self.name process_data(self.name, self.q) print "Exiting " + self.name def process_data(threadName, q): while not exitFlag: queueLock.acquire() if not workQueue.empty(): data = q.get() queueLock.release() print "%s processing %s" % (threadName, data) else: queueLock.release() time.sleep(1) threadList = ["Thread-1", "Thread-2", "Thread-3"] nameList = ["One", "Two", "Three", "Four", "Five"] queueLock = threading.Lock() workQueue = Queue.Queue(10) threads = [] threadID = 1 # Create a new thread for tName in threadList: thread = myThread(threadID, tName, workQueue) thread.start() threads.append(thread) threadID += 1 # Fill the queue queueLock. acquire() for word in nameList: workQueue.put(word) queueLock.release() # Wait for the queue to be emptied while not workQueue.empty(): pass # Notify the thread that it is time to exit exitFlag = 1 # Wait for all threads to finish for t in threads : t.join() print "Exiting Main Thread"

The result of executing the above program:

Starting Thread-1
Starting Thread-2
Starting Thread-3
Thread-1 processing One
Thread-2 processing Two
Thread-3 processing Three
Thread-1 processing Four
Thread-2 processing Five
Exiting Thread-3
Exiting Thread-1
Exiting Thread-2
Exiting Main Thread