Parallel Programming - Introduction Part
Design Parallel Programming 1
Task Decomposition : Decomposition task program, executed to implement the parallelization on different processors. (The following two methods may be used)
- Field Decomposition : The
问题数据decomposition (when the amount of data processed is large, separately)- Functional decomposition : the
问题decomposition of the task (the big tasks into multiple small-tasking)
Task assignment : assign tasks to each processor (object 负载均衡)
Gathering : to merge with the large task into small tasks together to improve the performance of the process
if the number of tasks far exceeds the number of processors available, due to other factors such as thread switching can result in reduced efficiency
if the computer has a large number of small tasks for special design such as the use of GPU computing, it would benefit from running a large number of small tasks in
Mapping : specify which tasks will be executed (the goal is to minimize the total execution time) (2 can use the following strategies in between these two strategies due to the conflict, the need to make trade-offs)
- Frequent communication tasks on the same processor to increase
局部性- Tasks can execute concurrently on different processors increase
并发性
Note! Mapping problem, namely NP complete 2 . For the same size of the task, it is easy to direct mapping
if the workload of each task is vastly different, you need 负载均衡算法to determine the aggregation and mapped during policy runs.
If the situation changes during program execution or the traffic assignments may be used动态的负载均衡算法
(A variety of different problems load balancing algorithm, global or local. Global algorithms need to grasp the overall situation, the cost will increase a lot. Local algorithm only depend on the information for a particular task in itself, but it is difficult to find the optimal)
(presence manager / executive (manager schedule), hierarchical Managerial / executive (grouping, each has a scheduling manager), decentralized model (each processor maintains its own task pool))
Compared with the process thread 3
Performance Comparison
Multi-threaded multi-process lower than the cost, but lower performance
Comparative advantages and disadvantages
Multi-process advantages
- Each process mutually independent , abnormal child process does not affect the parent process
- By increasing the CPU, capacity can easily expand performance
- No lock effect
Each child has a separate address space and resources
Multi-process shortcomings
- Logic control complex , the need and the main process interaction
Needs across process boundaries , if large amounts of data to be transmitted is not very good for a small amount of data transmission, intensive operations
Multithreading advantage
- Logic control simple
- No need to cross process boundaries
- Direct share resources
Resource consumption than the process less
Multithreading shortcomings
- Where the collapse of a thread may affect the entire program
- And synchronization between threads lock control is too much trouble
Each thread and the main thread of the common address space
Note! No matter the process, thread or more after their dispatch will consume more CPU resources
python parallel programming problem
In general, a process can contain multiple parallel threads. However, Python interpreter is not fully thread-safe. In order to support multi-threaded Python program that uses a process called global interpreter lock GIL(Global Interpreter Lock) is a global lock . This means that only one thread can execute Python code at the same time. (If multiple threads try to access the same object data, then it may be in an inconsistent state)
Therefore, parallel programming in python you want to play the advantages of multi-core recommended to use multi-threaded multi-process instead of
Why not remove the GIL More details:? Python most difficult problems
The efficiency of processes and threads in python
GIL python due to problems of their own existence, can only run one thread at a time, in some experimental comparison, we find that the difference between processes and threads in python, in conclusion here given direct
comparison of the operations: CPU密集型, IO密集型,网络请求密集型
Multithreading
- CPU intensive operations, worse performance
- IO intensive operations, worse performance
- Network requests intensive operations, have obvious advantages
multi-Progress
- CPU-intensive operations, have obvious advantages
- IO-intensive operations, have obvious advantages
- Network request-intensive operations, and multi-threading is almost the same, but more of CPU resources
It is CPU-intensive, IO-intensive operations, the preferred multi-process, multi-threaded network request intensive selection
(PS: If the pursuit of efficiency, CPU-intensive operations should be the preferred C/C++language of
IO-intensive operations, CPU consumes very little and replace it with C / C ++ is also difficult to obtain improve efficiency should be used with high efficiency development language, 脚本语言is the first choice)
More details: efficiency comparative experiments Python in single-threaded, multi-threaded and multi-process
1. Reference books: "Python Parallel Programming Manual"
2.NP complete problem: NP-complete problem is one of the seven mathematical problems of the world. All non-deterministic polynomial complete problem can be converted to a class of problems called the logic operation to meet the problems. Since all of the possible answers to such questions, can be calculated in polynomial time, then people would suspect that there is a deterministic algorithm whether such problems can be directly calculated in polynomial time, or search for the correct answer? This is the famous NP = P? Guess (Baidu Encyclopedia)
https://zh.wikipedia.org/wiki/NP%E5%AE%8C%E5%85%A8(wiki Wikipedia )
3. the difference between multi-threaded and multi-process: Reference: the first one dating back to 2011 pages have been ineffective