Article directory
Fork/Join Introduction
The Fork/Join framework was introduced since JDK 7. It is a parallel computing framework, as the name suggests. It is based on the two operations of Fork and Join. Their function is to split a large task into multiple small tasks and divide these small tasks. Tasks are combined to get the result of a large task. Using the Fork/Join architecture can make full use of the performance advantages of multi-core CPUs and improve program execution efficiency.
Fork/Join component
The Fork/Join framework mainly contains three components:
- Thread pool: ForkJoinPool maintains a thread pool for executing tasks in parallel, and it can dynamically adjust the number of threads as needed.
- Task object: ForkJoinTask is an abstract class that represents a task that can be split. Subclasses of ForkJoinTask (RecursiveAction and RecursiveTask) must implement the fork() and compute() methods, where the fork() method is used to split the task and return the results of the subtasks, and the compute() method is used to perform the actual calculation of the task.
- Thread that executes tasks: ForkJoinWorkerThread is the worker thread in ForkJoinPool, which is responsible for executing tasks.
Fork/Join principle-divide and conquer method
ForkJoinPool is mainly used to solve problems using the Divide-and-Conquer Algorithm. Typical applications such as quick sort algorithm, ForkJoinPool need to use relatively few threads to handle a large number of tasks. For example, if you want to sort 10 million pieces of data, then this task will be divided into two sorting tasks of 5 million pieces and a merge task for these two sets of 5 million pieces of data. By analogy, the same segmentation process will be performed for 5 million data. In the end, a threshold will be set to specify when the data size reaches what size, such segmentation processing will be stopped. For example, when the number of elements is less than 10, the splitting will stop and insertion sort will be used to sort them. So in the end, all the tasks will add up to about 2,000,000+. The key to the problem is that for a task, it can only be executed after all its subtasks are completed.
Fork/Join principle-work stealing algorithm
The core of Fork/Join is the use of multi-core modern hardware devices. During an operation, there will be an idle CPU. So how to make good use of this idle CPU becomes the key to improving performance, and here we are going to mention work stealing. The (work-stealing) algorithm is the core concept of the entire Fork/Join framework. The Fork/Join work-stealing (work-stealing) algorithm refers to a thread stealing tasks from other queues for execution.
Why do we need to use work-stealing algorithms? If we need to do a relatively large task, we can divide the task into a number of subtasks that are independent of each other. In order to reduce the competition between threads, we put these subtasks into different queues and assign them to each queue. Create a separate thread to execute the tasks in the queue. Threads and queues correspond one to one. For example, thread A is responsible for processing tasks in queue A. However, some threads will finish the tasks in their own queue first, while there are still tasks waiting to be processed in the queues corresponding to other threads. Instead of waiting, the thread that has finished its work might as well help other threads to work, so it steals a task from the queue of other threads for execution. At this time, they will access the same queue, so in order to reduce the competition between the stealing task thread and the stolen task thread, a double-ended queue is usually used. The stolen task thread always takes the task from the head of the double-ended queue for execution. The thread that steals the task always takes the task from the tail of the double-ended queue for execution.
The advantage of the work-stealing algorithm is that it makes full use of threads for parallel computing and reduces competition between threads. Its disadvantage is that competition still exists in some cases. As mentioned above, the concept of automatic parallelization was introduced in Java 8. It allows a part of Java code to be automatically executed in parallel, that is, we use ForkJoinPool's ParallelStream.
For the number of threads in the ForkJoinPool general thread pool, it is usually sufficient to use the default value, which is the number of processors of the runtime computer. You can adjust the number of threads in ForkJoinPool by setting the system property: java.util.concurrent.ForkJoinPool.common.parallelism=N (N is the number of threads). You can try adjusting different parameters to observe the output results each time.
Fork/Join case
Use Fork/Join to calculate the sum of 1-10000. When the number of calculations of a task is greater than 3000, the task is split. When the number is less than 3000, it is calculated.
public class Test05 {
* @param args
*/
public static void main(String[] args) {
long start = System.currentTimeMillis();
ForkJoinPool pool = new ForkJoinPool();
SumRecursiveTask task = new SumRecursiveTask(1,10000l);
Long result = pool.invoke(task);
System.out.println("result="+result);
long end = System.currentTimeMillis();
System.out.println("总的耗时:" + (end-start));
}
}
class SumRecursiveTask extends RecursiveTask<Long>{
// 定义一个拆分的临界值
private static final long THRESHOLD = 3000l;
private final long start;
private final long end;
public SumRecursiveTask(long start, long end) {
this.start = start;
this.end = end;
}
@Override
protected Long compute() {
long length = end -start;
if(length <= THRESHOLD){
// 任务不用拆分，可以计算
long sum = 0;
for(long i=start ; i <= end ;i++){
sum += i;
}
System.out.println("计算："+ start+"-->" + end +",的结果为：" + sum);
return sum;
}else{
// 数量大于预定的数量，那说明任务还需要继续拆分
long middle = (start+end)/2;
System.out.println("拆分:左边 " + start+"-->" + middle+", 右边" + (middle+1) + "-->" + end);
SumRecursiveTask left = new SumRecursiveTask(start, middle);
left.fork();
SumRecursiveTask right = new SumRecursiveTask(middle + 1, end);
right.fork();
return left.join()+right.join();
}
}
}