One of the multithreading articles of concurrent programming

1. Process related theories

　　1. What is a thread?

　　　　Process: an ongoing process or a task (such as running an application QQ). The cpu is responsible for executing the task.

 

　　2. What is the difference between a process and a program?

　　　　A program is just a string of code, and a process refers to the running process of a program.

　　Note: If the same program is executed twice, it is also two processes, such as opening Baofengyingyin, although it is the same software, but one can play "Da Qin Empire",

　　　　One can play "The Beautiful Legend of Sicily".

 

　　3. Concurrency and Parallelism

　　　　　　3.1 Whether it is parallel or concurrent, in the eyes of the user, it runs 'simultaneously'. Whether it is a process or a thread, it is just a task. The real work is the CPU.

　　　　The cpu does these tasks, and a cpu can only perform one task at a time.

　　　　　　3.2 Concurrency: It is pseudo-parallel, that is, it appears to be running at the same time. A single cpu + multi-channel technology can achieve concurrency (single core + multi-channel, to achieve concurrent execution of multiple processes).

　　　　　　　　For example: you want to do your homework, you want to play games, you also want to take a walk and chat on QQ. But you can only do one thing at the same time, how can you play the effect of concurrent execution of multiple tasks?

　　　　　　　　It can be like this: you do your homework for a while, play games for a while, then go for a walk, and then chat with others on QQ. That way, you can do all kinds of things for a while (feeling pointless...)

　　　　　　3.3 Parallelism: Running at the same time, parallelism can only be achieved if there are multiple CPUs. Under a single core, multi-channel technology, multiple cores,

　　　　　　　　Each core can also take advantage of multi-pass technology (multi-pass technology is for a single core).

　　　　　　　　For example: there are four cores and six tasks, so four tasks are executed at the same time, assuming they are assigned to cpu1, cpu2, cpu3, ​​cpu4 respectively.

　　4, the creation of the process

　　　　For general-purpose systems (running many applications, such as our commonly used Windows systems), the ability to create or cancel processes during system operation is required, which is mainly divided into 4 forms to create new processes　　　

1. System initialization (use the ps command to view the process in linux, use the task manager in windows, the foreground process is responsible for interacting with the user, the process running in the background has nothing to do with the user, the process that runs in the background and only wakes up when needed is called a daemon process , such as email, web pages, news, print)

2. A process opens a subprocess during the running process (such as subprocess.Popen, etc.)

3. User's interactive request, and create a new process (such as user double-click Baofengyingyin)

4. Initialization of a batch job (applies only to mainframe batch systems)　　　

　　Either way, a new process is created by an existing process executing a system call to create a process:

1. In UNIX the system call is: fork, fork creates an exact copy of the parent process, with the same storage image, the same environment strings, and the same open files (in the shell interpreter process, executing a command will create a child process)

2. In windows, the system call is: CreateProcess, CreateProcess not only handles the creation of the process, but also is responsible for loading the correct program into the new process.

   5. About the child process created, UNIX and Windows

   1. The same thing is: after the process is created, the parent process and the child process have their own different address spaces (multi-channel technology requires the physical level to achieve memory isolation between processes), any process's modification in its address space will not be affect another process.

   2. The difference is: in UNIX, the initial address space of the child process is a copy of the parent process, prompt: the child process and the parent process can have a read-only shared memory area. But for the windows system, the address space of the parent process and the child process is different from the beginning.

　　6. Termination of the process

　　　

1. Normal exit (voluntary, such as when the user clicks the cross on the interactive page, or when the program is executed, a system call is invoked to initiate a normal exit, using exit in linux and ExitProcess in windows)

2. Exit with error (voluntary, a.py does not exist in python a.py)

3. Fatal errors (involuntary, executing illegal instructions, such as referencing non-existing memory, 1/0, etc., can catch exceptions, try...except...)

4. Killed by another process (involuntary, like kill -9)

 　　7, the process hierarchy

　　　　Regardless of UNIX or Windows, a process has only one parent process, the difference is:　

1. All processes in UNIX are rooted in the init process and form a tree structure. The parent and child processes together form a process group, so that when a signal is sent from the keyboard, the signal is sent to all members of the current keyboard-related process group.

2. In windows, there is no concept of process level, all processes have the same status, the only hint similar to process level is that when a process is created, the parent process gets a special token (called a handle), the handle can be Used to control the child process, but the parent process has the right to pass the handle to other child processes, so there is no hierarchy.

　　8. The state of the process　　　　

　　　　tail -f access.log |grep '404'

　　　　Execute the program tail, start a child process, execute the program grep, start another child process, communicate between the two processes based on the pipe '|', and use the result of tail as the input of grep.

　　　　The state of the process grep waiting for input (i.e. I/O) is called blocking, and the grep command cannot run at this time

　　　　There are two situations in which a process can logically fail to run:

　　　　　　1. The process hang is its own reason. When I/O is blocked, the CPU must be given up for other processes to execute, so as to ensure that the CPU is always working.

　　　　　　2. It has nothing to do with the process. It is the operating system level. It may call other processes to use the CPU due to a process taking too much time or priority.

　　　　Therefore, a process has the following three states:

　　　　　　

2. How to start the process

　　1. Introduction to the multiprocessing module

　　　　Multi-threading in python cannot take advantage of multi-core. If you want to fully use the resources of multi-core CPU (os.cpu\_count\(\) view), in most cases in python, you need to use multi-process.

　　　　Python provides multiprocessing. The multiprocessing module is used to start child processes and execute our custom tasks (such as functions) in the child processes,

　　This module has a similar programming interface to the multithreading module threading. The multiprocessing module has many functions: support for subprocesses, communication and sharing of data,

　　Perform different forms of synchronization, > provides components such as Process, Queue, Pipe, Lock, etc.

　　2. Introduction to the Process class

　　　　创建进程的类：　　　

　　　　Process([group [, target [, name [, args [, kwargs]]]]])，由该类实例化得到的对象，可用来开启一个子进程

　　　　强调：

　　　　　　1. 需要使用关键字的方式来指定参数
　　　　　　2. args指定的为传给target函数的位置参数，是一个元组形式，必须有逗号

　　　　参数说明：

　　group参数未使用，值始终为None

　　target表示调用对象，即子进程要执行的任务

　　args表示调用对象的位置参数元组，args=(1,2,'egon',)

　　kwargs表示调用对象的字典,kwargs={'name':'egon','age':18}

　　name为子进程的名称

　　　　方法介绍：　

p.start()：启动进程，并调用该子进程中的p.run() 
p.run():进程启动时运行的方法，正是它去调用target指定的函数，我们自定义类的类中一定要实现该方法  

p.terminate():强制终止进程p，不会进行任何清理操作，如果p创建了子进程，该子进程就成了僵尸进程，使用该方法需要特别小心这种情况。如果p还保存了一个锁那么也将不会被释放，进而导致死锁
p.is_alive():如果p仍然运行，返回True

p.join([timeout]):主线程等待p终止（强调：是主线程处于等的状态，而p是处于运行的状态）。timeout是可选的超时时间。

　　　　属性介绍：

p.daemon：默认值为False，如果设为True，代表p为后台运行的守护进程，当p的父进程终止时，p也随之终止，并且设定为True后，p不能创建自己的新进程，必须在p.start()之前设置

p.name:进程的名称

p.pid：进程的pid

　　　3、Process类的使用

　　　　注意：在windows中Process()必须放到# if __name__ == '__main__':下。

　　　　方式一：　　

from multiprocessing import Process
import time
def task(name):
    print('%s is sleeping'%name)
    time.sleep(3)
    print('%s is awaked'%name)

if __name__ == '__main__':
    p = Process(target=task,args=('子进程1',)) #括号里的逗号不可少
    p.start()  #仅仅是给操作系统发送了一个信号

    print('主进程')
    '''
'''运行结果
主进程
子进程1 is sleeping
子进程1 is awaked
'''

View Code

　　　　方式二：

from multiprocessing import Process
import time
class MyProcess(Process):
    def __init__(self,name):
        super().__init__()
        self.name = name
    def run(self):
        '''默认就是run方法'''
        print('%s is runing'%self.name)
        time.sleep(3)
        print('%s is walking'%self.name)
if __name__ == '__main__':
    p2 = MyProcess('进程1')
    p2.start()
    print('主进程')
'''运行结果
主进程
进程1 is runing
进程1 is walking
'''

View Code