(Turn) Python multi-threaded task of

https://www.cnblogs.com/yifchan/p/python-1-41.html

 

Multitasking Introduction

We first look at the program does not have multi-tasking

import time


def sing():
    for i in range(5):
        print("我喜欢唱")
        time.sleep(1)


def dance():
    for i in range(5):
        print("我喜欢跳")
        time.sleep(1)


def main():
    sing()
    dance()
    pass


if __name__ == "__main__":
    main()

The result: It took more than ten seconds, can only be performed in sequence, not together / synchronous execution

I like to sing 
I like to sing 
I like to sing 
I like to sing 
I sing like 
I like dancing 
I like dancing 
I like dancing 
I like dancing 
I like dancing

 

Let us look at the use of multi-threaded programs

import time
import threading


def sing():
    for i in range(5):
        print("我喜欢唱歌")
        time.sleep(1)


def dance():
    for i in range(5):
        print("我喜欢跳舞")
        time.sleep(1)


def main():
    t1 = threading.Thread(target=sing)
    t2 = threading.Thread(target=dance)
    t1.start()
    t2.start()


if __name__ == '__main__':
    main()

The result: spend a little more than five seconds, execute code synchronization

I like to sing and 
I like dancing 
I like dancing I like to sing 

I like dancing 
I like to sing and 
I love to dance 
I like to sing and 
I love to dance 
I like to sing

 

Multitasking

Here we can extend some knowledge of how to run a computer program consists of additional multi-tasking?

The operating principle of a single core cpu: round-robin

Single core cpu only run one program at the same time, but you can see a lot of programs running because fast switching single-core cpu, that is to take over a program running very short period of time such as .00001 seconds, it runs a program change , and so forth, that is, at the same time you see the execution of multiple programs. This is one way to achieve multi-tasking operating system, but in fact is a pseudo multi-tasking.

Round-robin philosophy is that as long as I switched fast enough, you see what I do more than one thing at the same time, this is the scheduling algorithms operating system. Operating system as well as priority scheduling, such as listening to music to be continued.

• If multicore cpu to run multiple tasks at the same time, we call parallel, true multi-tasking; the number of tasks is less than the number of cpu;
• If the switch is a single core cpu to run multiple tasks, we call it concurrent, multi-tasking is false. The number of tasks than the number of CPU;
• But because the daily, the number of tasks usually more than the number of cpu core, so we are talking about multi-tasking are generally concurrent, or false multi-task;

 

Thread Multithreading

We have already seen the thread multitasking, then we learn to use threads;

Creating multiple threads by Thread (target = xxx)

Thread using the following steps:

1. Introducing threading module;
2. Writing functions required of multi-tasking;
3. Threading.Thread create instances of the class object and pass a function reference;
4. Call start method instance of an object, create a child thread.

If you do not know how to use multithreading? It does not matter, look at the following chart to know

code show as below:

  Using multiple threads

note:

• () Function call represents the function name
• Function name indicates the use of a reference function, tells the function in which;

 

Code Reading

def main():
    t1 = threading.Thread(target=sing)
    t2 = threading.Thread(target=dance)
    t1.start()
    t2.start()

Each function will be performed when there is a thread, the main thread we call;
when we perform a

t1 = threading.Thread(target=sing)

When, you create an instance of an object represents a Thread class t1, and pass references to a Thread class sing function of t1.
Similarly, t2 is also true;
when we execute to

t1.start()

When, in this instance of the object creates a child thread, to call sing function; then go down the main thread, the child thread to call sing function.
When the main thread went t2.start (), re-create a child thread, the child thread to call the dance function, because there is no code behind, and then the main thread will wait for the completion of all sub-threads, and then end the program / the main thread; can understand the main thread to give the child thread corpses after death, then the main thread and then die.

The main thread to wait for the end of the sub-thread execution reason: child thread in the implementation process will call resources and produce some variables, when the child thread executed,
the main thread you want these useless resources and garbage cleanup.

 

Create a multi-threaded execution understanding

We can use the following code to get all threads in the current program;

threading.enumerate()

On the use of enumerate, you can view a blog on my  python functions of the built-in function enumerate  , but represented here is to get all threads in the current program, you can not see;

 

Let some threads to run

Because after the thread creation, thread execution order is uncertain, if we want the method to perform a thread, you can use the time.sleep. code show as below

  Let some threads to run

operation result

We can see, sing threads have to run, but only one thread at this time to see the main thread, this is because when the child thread execution finished before viewing thread to execute the code.

 

To cycle through the currently running thread count

We can many times, the main thread to see infinite loop by letting the child thread delay execution of the current number of threads (appropriate delay), you can see the number of the currently running thread, when the thread number 1 or less, use the break ends the main thread.

code show as below

  To cycle through the currently running thread count

operation result

You can see, the beginning of time is only one main thread, when the child thread start, there are three threads, sing at the end of sub-thread, leaving only two threads, and after the end of the dance, there is only one main thread.

 

Verify the execution time of the child thread

In order to verify the execution time of the child thread, we can run the code in interactive python, a function called in when the child thread execution which represents a sub-thread execution and when;

Verification results are as follows

After the execution of this, we can determine the child thread execution start is to call the object in the example of the thread () method.

Authentication Code

import threading
def sing():
    print("-----sing-----")
t1 = threading.Thread(target=sing)
t1.start()
-----sing-----

 

Verify the creation time of the child thread

Verification principle: we can number each time period to determine the child thread was created by computing the number of threads

Authentication Code

  Verify the creation time of the child thread

Validation results

The number of threads can be observed before calling the start method has always been a main thread, so we can determine the time for a thread is created in the instance of an object method calls start after;

Combined with the front, we can conclude that create time and execution time the child thread is created out of the Thread instance object after calling the start method, and the end of time the child thread is executed after the function call is completed.

 

To create a process through inheritance Thread class

We are in front of the child thread by calling a function, then when the function too, want those functions packaged as a class, we can not call a class by sub-thread of it?

The second method steps of creating a thread

1. Introducing threading module;
2. Define a class, the class which inherits threading.Thread class, which defines a run method;
3. Then create an instance of an object class;
4. Call an instance object's start method, you create a thread.

If you create a thread through the class inheritance is when a Thread class to create, you must define the run method in which, when you start method is called, it will automatically call the run method is to run the next method of executing code inside thread .

The sample code to create a process through inheritance Thread class

  To create a process through inheritance Thread class

operation result

--- run --- 
I Thread-1, i ---> 0 
I Thread-1, i ---> 1 
I Thread-1, i ---> 2

Knowledge Point

• This method is applicable to a thread inside to do more complicated, to be packaged into several functions to do, then we will package it into a class.
• Several other functions defined in the class, you can call these functions run inside.
• Which method is better when you create a thread? Which is simple to use which.

note:

An instance of an object can only create a thread;
time to create a process through inheritance Thread class, the class does not automatically call a function in addition to other functions run, if you want to call other numbers, you can use self.xxx in the run method () to call.

Multi-threaded shared variables

Modify global variables in a function, if it is digital and other immutable types, then use the global statement can be modified, such as if it is a list of variable types, you can not declare, and other direct append to the list of content to be modified, but, if not to modify the contents of the list, but point to a new list, you need to use the global statement;

In a global variable, if the data is referenced to modify, you need to use global, if it is modified (for a direct reference address) references to global variables, then you need to use global, at the same time, we should also pay attention to the global variable or variables are immutable type, such as a digital, immutable, can only be modified by modifying the variable global variable references, it is necessary to global;

 

Verify multiple threads share global variables

Verification principle:

Define a global variable, see the function 1 plus 1 in 2 functions, so that the thread control function 1 before execution, if the thread function to view the results and view the results of a function of the same 2, then that proves shared between multiple threads global variables.

Code verification

  Sharing global variables between multiple threads verification

operation result

--- sing in g_num: 101 --- 
--- Dance in g_num: 101 --- 
--- main thread of g_num: 101 ---

As we can see the code, share global variables between multiple threads.

 

We can share global variables between multiple threads to be understood as:
a house, there are several people, a person is a thread, everyone has things their own private resources, but in the big house there, there are also some things, such as said the only fountain of water, have a drink half, scored a man to pick water, only half of this water dispenser inside who is the global variable.

 

Multi-threaded pass parameters to the child thread

Pass parameters to the child thread syntax is as follows

g_nums = [11, 22]

t1 = threading.Thread(target=sing, args=(g_num,))

Parameter passing to the sub-thread Sample Code

  Pass parameters to the child thread

operation result

--- g_nums sing in: [. 11, 22 is, 33 is] --- 
--- Dance in g_nums: [11, 22, 33 is] --- 
--- main thread g_nums: [11, 22, 33] ---

 

Sharing between multiple threads: competition for resources

Shared global variables exist resource contention problems, two threads will use or modify the existing problems, a modification that uses not exist;
pass parameters when 100 might not be a problem, because the number is small, the probability of dots; However, parameter passing 1000000 when digital becomes large, the probability increases;

num + = 1 can be divided into three, the value of the num acquired, a value plus 1, for reassignment num; 1 is possible when performing the thread 12, it is intended to implement the three, cpu will resources to thread 2 , and the thread 2 the same way, then the implementation of the first three threads 1, so the thread 1 +1, global variables to store 1; 2 + 1 thread turn, storing global variables 1; the question arises, would add two should be 2, but still a global variable.

Competition for resources code examples

  Shared variable resource competition

operation result

the g_num of add1: 1096322
the g_num of add2: 1294601
the g_num of main: 1294601

 

Mutex to solve the problem of resource contention

Atomic operations: either do or done;

Mutex: a person to do something, other people are not allowed to do this, you must wait until the person in front to be done about it, then to do the examples attractions toilet.

Mutex grammar

# Create a lock: 
mutex = threading.Lock () 
# locked: 
mutex.acquire () 
# to unlock: 
mutex.release ()

Use a mutex to solve the problem of resource contention

  Use a mutex to solve the problem of resource contention

operation result

the g_num of add2: 1901141
the g_num of add1: 2000000
the g_num of main: 2000000

As can be seen, the use of mutex can resolve resource contention problems.

 

Deadlock

In particular, the use of multiple mutex mutex time, especially prone to deadlock, what you waiting for my resources, I'm waiting for your resources;

 

This chapter summarizes

Thread of the life cycle

• From the beginning to the end of the execution, there has been a main thread
• If the main thread to die first, then the child thread running will die.
• Child thread is created at the beginning when calling t.start (), instead of creating an instance of the Thread object.
• Child thread is started when t.start () call;
• Time of death is in the child thread after thread of the call function execution is completed;
• After the thread is created, a thread of execution order is uncertain;
• If you want to make a thread to execute, time.sleep method can be used.

 

Two ways to create multi-threaded

Creating multiple threads by Thread (target = xxx)

1. Introducing threading module;
2. Writing functions required of multi-tasking;
3. Threading.Thread create instances of the class object and pass a function reference;
4. Call start method instance of an object, create a child thread.

To create a process through inheritance Thread class

1. Introducing threading module;
2. Define a class, the class which inherits threading.Thread class, which defines a run method;
3. Then create an instance of an object class;
4. Call an instance object's start method, you create a thread.

 

Multithreading understand

• Creating multiple threads can be understood in preparation for the creation of threads;
• start () is directly create and run the thread when you're ready;
• After the main thread to wait for the child thread in the end is to clean up garbage in the child thread that may arise;

 

Multi-threaded shared global variables

• Global variables shared between threads and sub-sub-thread;
• Reference to the sub-thread can pass threading.Thread (target = sing, args = (g_num,)) for parameter passing;
• There may be resource contention issues between multiple threads;
• You can use a mutex to solve the problem of resource competition;

Multitasking Introduction

We first look at the program does not have multi-tasking

import time


def sing():
    for i in range(5):
        print("我喜欢唱")
        time.sleep(1)


def dance():
    for i in range(5):
        print("我喜欢跳")
        time.sleep(1)


def main():
    sing()
    dance()
    pass


if __name__ == "__main__":
    main()

The result: It took more than ten seconds, can only be performed in sequence, not together / synchronous execution

I like to sing 
I like to sing 
I like to sing 
I like to sing 
I sing like 
I like dancing 
I like dancing 
I like dancing 
I like dancing 
I like dancing

 

Let us look at the use of multi-threaded programs

import time
import threading


def sing():
    for i in range(5):
        print("我喜欢唱歌")
        time.sleep(1)


def dance():
    for i in range(5):
        print("我喜欢跳舞")
        time.sleep(1)


def main():
    t1 = threading.Thread(target=sing)
    t2 = threading.Thread(target=dance)
    t1.start()
    t2.start()


if __name__ == '__main__':
    main()

运行结果：花了五秒多一点，代码同步执行

我喜欢唱歌
我喜欢跳舞
我喜欢跳舞我喜欢唱歌

我喜欢跳舞
我喜欢唱歌
我喜欢跳舞
我喜欢唱歌
我喜欢跳舞
我喜欢唱歌

 

多任务

在这里我们可以由多任务额外扩展一些知识，电脑是怎么运行程序的？

单核cpu的运行原理：时间片轮转

单核cpu同一时间只能运行一个程序，但你看到的能运行很多程序是因为单核cpu的快速切换，即把一个程序拿过来运行极短的时间比如0.00001秒，就换运行下一个程序，如此往复，就是你看到的同一时间执行多个程序。这是操作系统实现多任务的一种方式，但其实是伪多任务。

时间片轮转的理念是，只要我切换的够快，你看到的就是我同时做多件事情，这是操作系统的调度算法。操作系统还有优先级调度，比如听歌要一直持续。

• 如果是多核cpu同时运行多个任务，我们就称之为并行，是真的多任务；任务数少于cpu数量；
• 如果是单核cpu切换着运行多个任务，我们就称之为并发，是假的多任务。任务数多于cpu数量；
• 但因为日常中，任务数一般多于cpu核数，所以我们说的多任务一般都是并发，即假的多任务；

 

Thread多线程

在前面我们已经看过了线程实现多任务，接下来我们学习线程的使用方法；

通过Thread(target=xxx)创建多线程

线程的使用步骤如下：

1. 导入threading模块；
2. 编写多任务所需要的的函数；
3. 创建threading.Thread类的实例对象并传入函数引用；
4. 调用实例对象的start方法，创建子线程。

如果你还不懂怎么使用多线程？没关系，看下面这个图就知道了

代码如下：

  多线程的使用

注意：

• 函数名() 表示函数的调用
• 函数名 表示使用对函数的引用，告诉函数在哪；

 

代码解读

def main():
    t1 = threading.Thread(target=sing)
    t2 = threading.Thread(target=dance)
    t1.start()
    t2.start()

每个函数在执行时都会有一个线程，我们称之为主线程；
当我们执行到

t1 = threading.Thread(target=sing)

时，表示创建了一个Thread类的实例对象t1，并且给t1的Thread类中传入了sing函数的引用。
同理，t2也是如此；
当我们执行到

t1.start()

时，这个实例对象就会创建一个子线程，去调用sing函数；然后主线程往下走，子线程去调用sing函数。
当主线程走到t2.start()时，再次创建一个子线程，子线程去调用dance函数，因为后面没有代码了，然后主线程就会等待所有子线程的完成，再结束程序/主线程；可以理解为主线程要给子线程死了之后收尸，然后主线程再去死。

主线程要等子线程执行结束的原因：子线程在执行过程中会调用资源以及产生一些变量等，当子线程执行完之后，
主线程要将这些无用的资源及垃圾进行清理工作。

 

多线程创建执行理解

我们可以使用如下代码获取当前程序中的所有线程；

threading.enumerate()

关于enumerate的使用，可以查看我的上一篇博客 python内置函数之enumerate函数 ，但这里表示的是获取当前程序中的所有线程，可以不必看；

 

让某些线程先运行

因为 线程创建完后，线程的执行顺序是不确定的，如果我们想要让某个线程先执行，可以采用time.sleep的方法。代码如下

  让某些线程先运行

运行结果

我们可以看到，sing线程已经先运行了，但是此时查看的线程只有一个主线程，这是因为当子线程执行完了才执行到查看线程的代码。

 

循环查看当前运行的线程数

我们可以通过让子线程延时执行多次，主线程死循环查看当前线程数(适当延时)，即可看到当前运行的线程数量，当线程数量小于等于1时，使用break结束主线程。

代码如下

  循环查看当前运行的线程数

运行结果

可以看到，刚开始的时候只有一个是主线程，当子线程开始后，有三个线程，在sing子线程结束后，只剩两个线程了，dance结束后，只有一个主线程。

 

验证子线程的执行时间

为了验证子线程的执行时间，我们可以在交互式python下运行代码，子线程调用的函数在何时执行即代表子线程在何时执行；

验证结果如下

据此，我们可以判断子线程的执行是在线程的示例对象调用start()方法之后执行的。

验证代码

import threading
def sing():
    print("-----sing-----")
t1 = threading.Thread(target=sing)
t1.start()
-----sing-----

 

验证子线程的创建时间

验证原理：我们可以通过计算线程数在各个时间段的数量来判断子线程的创建时间

验证代码

  验证子线程的创建时间

验证结果

可以观察到在调用start方法之前线程数一直都是1个主线程，由此我们可以判断线程的创建时间是在调用了实例对象的start方法之后；

结合前面，我们可以得出结论，子线程的创建时间和执行时间是在Thread创建出来的实例对象调用了start方法之后，而子线程的结束时间是在调用的函数执行完成后。

 

通过继承Thread类来创建进程

前面我们是通过子线程调用一个函数，那么当函数过多时，想将那些函数封装成一个类，我们可以不可以通过子线程调用一个类呢？

创建线程的第二种方法步骤

1. 导入threading模块；
2. 定义一个类，类里面继承threading.Thread类，里面定义一个run方法；
3. 然后创建这个类的实例对象；
4. 调用实例对象的start方法，就创建了一个线程。

如果你创建一个线程的时候是通过 类继承一个Thread类来创建的，必须在里面定义run方法，当你调用start方法的时候，会自动调用run方法，接下来线程执行的就是run方法里面的代码。

通过继承Thread类来创建进程示例代码

  通过继承Thread类来创建进程

运行结果

---run---
我是Thread-1,i--->0
我是Thread-1,i--->1
我是Thread-1,i--->2

知识点

• 这种方法适用于一个线程里面要做的事情比较复杂，要封装成几个函数来做，那么我们就将它封装成一个类。
• 在类中定义其他的几个函数，可以在run里面进行调用这几个函数。
• 创建线程时使用哪种方法比较好？哪个简单使用哪个。

注意：

一个实例对象只能创建一个线程；
通过继承Thread类来创建进程时，不会自动调用类中除run函数的其他函数，如果想要调用其他可数，可以在run方法中使用self.xxx()来调用。

多线程共享变量

在函数中修改全局变量，如果是数字等不可变类型，要用global声明之后才能修改，如果是列表等可变类型，就可以不用声明，直接append等对列表内容进行修改，但，如果不是对列表内容进行修改，而是指向一个新的列表，就需要使用global声明；

在全局变量中，如果是对引用的数据进行修改，那么不需要使用global，如果是对全局变量的引用进行修改(直接换一个引用地址)，那么就需要使用global，同时，我们也应注意全局变量是可变类型还是不可变类型，比如数字，不可变，就只能通过修改变量的引用来进行修改全局变量了，所以需要global；

 

验证多线程中共享全局变量

验证原理：

定义一个全局变量，在函数1中加1，在函数2中查看，让线程控制的函数1先执行，如果线程函数2的查看结果和函数1的查看结果一样，那么就证明多线程之间共享全局变量。

代码验证

  多线程之间共享全局变量验证

运行结果

---sing中的g_num: 101---
---dance中的g_num: 101---
---主线程中的g_num: 101---

如上代码我们可知，多线程之间共享全局变量。

 

我们可以将多线程之间共享去全局变量理解为：
一个房子里面有几个人，一个人就是一个线程，每个人有自己私有的东西资源，但在这个大房子里面，也有些共有的东西，比如说唯一一台饮水机的水，有一个人喝了一半，拿下一个人来接水，也只剩下一半了，这个饮水机里面的谁就是全局变量。

 

多线程给子线程传参

给子线程传参数语法如下

g_nums = [11, 22]

t1 = threading.Thread(target=sing, args=(g_num,))

给子线程传参示例代码

  给子线程传参数

运行结果

---sing中的g_nums: [11, 22, 33]---
---dance中的g_nums: [11, 22, 33]---
---主线程中的g_nums: [11, 22, 33]---

 

多线程之间共享问题：资源竞争

共享全局变量存在资源竞争的问题，两个线程同时使用或者修改就会存在问题，一个修改一个使用不会存在；
传参100的时候可能不会出现问题，因为数字较小，概率也小点；但传参1000000的时候，数字变大，概率也变大；

num += 1可以分解为三句，获取num的值，给值加1，给num重赋值；有可能当线程1执行12句，正打算执行3句的时候，cpu就将资源给了线程2，而线程2同理，然后又执行线程1的第3句，因此线程1 +1，存储全局变量为1；轮到线程2 +1，存储全局变量也为1；问题就出现了，本来加两次应该是2的，但全局变量还是1。

资源竞争代码示例

  共享变量的资源竞争问题

运行结果

the g_num of add1: 1096322
the g_num of add2: 1294601
the g_num of main: 1294601

 

互斥锁解决资源竞争问题

原子性操作：要么不做，要么做完；

互斥锁：一个人做某事的时候，别人不允许做这件事，必须需得等到前面的人做完了这件事，才能接着做，例子景点上厕所。

互斥锁语法

# 创建锁：
mutex = threading.Lock()
# 上锁：
mutex.acquire()
# 解锁：
mutex.release()

使用互斥锁解决资源竞争问题

  使用互斥锁解决资源竞争的问题

运行结果

the g_num of add2: 1901141
the g_num of add1: 2000000
the g_num of main: 2000000

可以看出，使用互斥锁可以解决资源竞争的问题。

 

死锁问题

使用互斥锁特别是多个互斥锁的时候，特别容易产生死锁，就是你在等我的资源，我在等你的资源；

 

本章内容总结

线程的生命周期

• 从程序开始执行到结束，一直都有一条主线程
• 如果主线程先死了，那么正在运行的子线程也会死。
• 子线程开始创建是在调用t.start()时，而不是创建Thread的实例化对象时。
• 子线程的开始执行是在调用t.start()时；
• 子线程的死亡时间是在子线程调用的函数执行完成后；
• 线程创建完后，线程的执行顺序是不确定的；
• 如果想要让某个线程先执行，可以采用time.sleep的方法。

 

创建多线程的两种方式

通过Thread(target=xxx)创建多线程

1. 导入threading模块；
2. 编写多任务所需要的的函数；
3. 创建threading.Thread类的实例对象并传入函数引用；
4. 调用实例对象的start方法，创建子线程。

通过继承Thread类来创建进程

1. 导入threading模块；
2. 定义一个类，类里面继承threading.Thread类，里面定义一个run方法；
3. 然后创建这个类的实例对象；
4. 调用实例对象的start方法，就创建了一个线程。

 

多线程理解

• 创建多线程可以理解为创建线程做准备；
• start () is directly create and run the thread when you're ready;
• After the main thread to wait for the child thread in the end is to clean up garbage in the child thread that may arise;

 

Multi-threaded shared global variables

• Global variables shared between threads and sub-sub-thread;
• Reference to the sub-thread can pass threading.Thread (target = sing, args = (g_num,)) for parameter passing;
• There may be resource contention issues between multiple threads;
• You can use a mutex to solve the problem of resource competition;