Multithreading
What is multi-threading is actually equivalent to the Naruto avatar, which is equivalent to the five-dimensional space where you have multiple mirror avatars that can do one thing at the same time. It is also equivalent to drinking coffee while taking a bath. I wrote a blog about the previous words; I can
get python multi-threading in a few minutes, but looking at the title, I know that the writing is not comprehensive, and there are many things to add. So here to make a complete summary, on the one hand, it is convenient for oneself and others on the other.
Use multi-threading (threading)
To use multithreading in python, you can use the module threading.
The previous example of Xiaoming and Xiaohong running is no longer used here, because that example is still relatively abstract and difficult to understand.
First import the threading
step;
1 create thread ming=threading.Thread(target=target function, args=[value])
2 declare thread ming.setDaemon(True)
3 run thread ming.start()
4 wait thread ming.join()
Here we assume that Xiao Hong and Xiao Ming want to take the express.
There are two ways to take the courier, 1, one person takes it, and 2, two people take it together.
Obviously it is much better for two people to get it together.
Now let's simulate the code; (now we only use steps 1 and 2) and see what happens.
import threading
import time
time0=time.time()
def print_ming():
time.sleep(4)
print('我拿到了（小明）')
def print_hong():
time.sleep(3)
print('我拿到了（小红）')
ming=threading.Thread(target=print_ming)
hong=threading.Thread(target=print_hong)
ming.start()
hong.start()
print('执行完了')
time1=time.time()
print(time1-time0)
We found that Xiao Ming Xiaohong finished running after the main program had finished running. So here we introduce the role of courier brother, the main process is courier brother, and Xiaoming and Xiaohong are two sub-processes respectively. If our code has only these two or more child processes, it does not matter, but if the main process is also very important and has some functions to be implemented, it will be very troublesome. Calling it is like taking a courier. Brother drives a courier car faster, so we have to let him (her) wait for Xiao Ming and Xiao Hong.
So if the little brother runs along, Xiao Ming and Xiaohong can't get the express, so we have to wait (let the little brother, that is, the main thread) to
use the detailed steps;
import threading
import time
time0=time.time()
def print_ming():
time.sleep(4)
print('我拿到了（小明）')
def print_hong():
time.sleep(3)
print('我拿到了（小红）')
ming=threading.Thread(target=print_ming)
hong=threading.Thread(target=print_hong)
ming.setDaemon(True)
ming.start()
hong.setDaemon(True)
hong.start()
ming.join()#让小哥即等等小明也等等小红
hong.join()
print('执行完了')
time1=time.time()
print(time1-time0)
The effect is as follows;
Lock problem (lock) (factory problem (my analogy))
Let’s not talk about anything else, let’s talk about my needs. Now I hope that my program can achieve 1000 times, starting from 0 and adding 1 to get 1000, such as the following code
a = 0
def go():
global a
for i in range(1000):
a+=1
print(a)
go()
Now I want to make it execute twice at the same time, which is to get 2000
code as follows;
import threading
import time
time0=time.time()
a = 0
def go():
global a
time.sleep(1)#让程序睡眠一秒便于对比
for i in range(1000):
a+=1
print(a)
def main():
tx = threading.Thread(target=go)
tx.setDaemon(True)
tx.start()
t1 = threading.Thread(target=go)
t1.setDaemon(True)
t1.start()
t1.join()
tx.join()
main()
time1=time.time()
print(time1-time0)
The results are as follows;
it seems that I have reached the goal, so now let's change the value of 1000 to 1000000.
The relationship between consumers and producers (don’t understand, it’s okay to remember the example I gave, after all, coding is not a technical term)
At this time, we found that something was wrong. According to reason, we should get twice, but now we don’t know what.
Now make an assumption. Now there are two employees in the factory producing parts. They need to place one part in the machine and the machine has only one place. When the worker puts the part into the placement port, the machine will pack one out. After packing, you will get a production ticket to prove that the worker has produced the parts and packed them in the machine. But this machine has a problem when two parts are put together, only one package will be packed.
The current situation is similar. The execution time of the threads is random. It may happen that two threads use a variable at the same time and the result is the same. For example, in the example of the factory, A and B had to process 200 parts with 100 parts each. When the boss checked, only 150 packages came out. The boss thought someone was lazy, but A and B had 200 bills to prove the same. When A and B were working, they put the parts in the machine at the same time and only packed one package.
So as an example to avoid this situation, the machine needs to be locked. When there is a part in it, it is locked and the other part is waiting.
Use
the'lock ' threading.Lock()
to change the code in the box.
It should be noted here that if two workers A and B correspond to machine No. 1, then B and C correspond to machine 2. So a lock means a machine corresponds to a global variable. That is to say, the variable is a and the lock is lock,
but when the variable is b, another lock, such as lock2, needs to be set. In other words, if one machine corresponds to two workers, and the other two workers correspond to another machine, the corresponding locks are also different.
Locking and blocking problem (condition) (factory problem (my analogy))
The relationship between consumers and producers (don’t understand, remember the example I gave, after all, typing code is not a professional term)
condition inherits lock,
but why use condition like this, mainly because locking and unlocking are more expensive , Sometimes there is no need to continue to execute some programs, and there is no need to continue unlocking with the lock. You can change the way to let it wait.
Continue to explain with the example of the factory, it is still the problem with this machine. Now this machine will overheat at every turn, so you can't put parts at this time. (Fortunately, this machine has an alarm to know if there is a problem) so workers have to wait at this time. First, you can hold the parts and wait for the machine to be okay.
Now introduce its method (threading.Condition())
acquire() lock
release() unlock
wait() waiting for
notify() alarm (only tell workers close to him)
notify_all() alarm (tell all workers)
Now change the code; also change the function.
import threading
import time
time0=time.time()
lock=threading.Condition()
a = 0
def go():
global a
time.sleep(1)#让程序睡眠一秒便于对比
lock.acquire()#上锁
for i in range(100):
#执行a加到100
a+=1
lock.notify_all()#发警报
lock.release()#开锁
print(a)
print(threading.current_thread())#查看当前线程
def stop():
global a
lock.acquire()
for i in range(100):
while a<=-20:
lock.wait()
a-=1
#a减1减100次如果减了一百次中有减到-20（线程是随机的不一定会减到-20）
# 那么就等等，所以如果减到了-20那么最后得到0，或者80（没有到-20），或者0
#有20次减到了-20其中加了20次还要加减80次。
lock.release()
print(a)
print(threading.current_thread())#查看当前线程
def main():
tx = threading.Thread(target=go)
tx.setDaemon(True)
tx.start()
t1 = threading.Thread(target=stop)
t1.setDaemon(True)
t1.start()
t1.join()
tx.join()
main()
time1=time.time()
print(time1-time0)
The results are as follows;
Queue
Mainly for the safety of some variables and the problem of locking. When our queue elements are not enough or full, it will be automatically locked, just like the global variable we mentioned earlier, except that this variable can be written into Multiple values are equivalent to a global list variable. But this guy can control our thread to pause and continue. At the same time, when several child threads call a global variable at the same time, the amount of change will be unstable. For example, the value of 2 is assigned by child process 1 and the value of 3 by child process 2. Then At this time, whether the value of the variable is 2 or 3. Since the thread is random and does not know its state, it is a quantum state.
So we use this thing to ensure the stability of this variable in this queue. Every time you modify it, I will write it down and give it to another person and this person has the right to choose the value of the first change. This is more commonly used in multi-threaded crawlers (read more than 500, write a multi-threaded crawler example, or wait until I have time to write, anyway, there is no time recently.)
1. Import this stuff is from queue import Queue
2 built in python This thing is a queue type (elements first come first go), and correspondingly it also needs a LifoQueue (elements first come first go) stack type.
3. Basic usage
q=Queue(4) 设置队列（最大允许几个值）
q.qsize() 查看最大允许几个
q.full() 看一看有没有满
q.empty()看看有没有空，空True，满False
q.put 加元素
q.get() 得到最先加入的元素并删去
Example;
from queue import Queue
q=Queue(4)
for i in range(1,5):
q.put(i)
for i in range(4):
print(q.get())
print(q.empty())
That's basically it.