(Horizontal screen phone to see the source code is more convenient)
Note: java source code analysis section if no special instructions are based on java8 version.
Note: This rule thread pool classes based on ForkJoinPool points.
Brief introduction
With the development and widespread use of multi-core processors on the hardware, concurrent programming a programmer must master a technique, in the interview the interviewer is often complicated by examining the relevant knowledge.
Today, we look at a face questions:
How to take full advantage of multi-core CPU, calculate large array of all integers?
Analyze
- Single-threaded sum?
We most likely to think that adding a single thread, a for loop to get.
- Adding thread pool?
If further optimization, we will naturally think of using a thread pool are added to the segment, and finally the result of the addition of each segment.
- other?
Yes, today is our protagonist --ForkJoinPool, but it is to how to achieve it? It seems to be how not used Ha ^ ^
Three implementation
OK, the analysis is over, we look at three implementation directly, not ink, directly serving.
/**
* 计算1亿个整数的和
*/
public class ForkJoinPoolTest01 {
public static void main(String[] args) throws ExecutionException, InterruptedException {
// 构造数据
int length = 100000000;
long[] arr = new long[length];
for (int i = 0; i < length; i++) {
arr[i] = ThreadLocalRandom.current().nextInt(Integer.MAX_VALUE);
}
// 单线程
singleThreadSum(arr);
// ThreadPoolExecutor线程池
multiThreadSum(arr);
// ForkJoinPool线程池
forkJoinSum(arr);
}
private static void singleThreadSum(long[] arr) {
long start = System.currentTimeMillis();
long sum = 0;
for (int i = 0; i < arr.length; i++) {
// 模拟耗时，本文由公从号“彤哥读源码”原创
sum += (arr[i]/3*3/3*3/3*3/3*3/3*3);
}
System.out.println("sum: " + sum);
System.out.println("single thread elapse: " + (System.currentTimeMillis() - start));
}
private static void multiThreadSum(long[] arr) throws ExecutionException, InterruptedException {
long start = System.currentTimeMillis();
int count = 8;
ExecutorService threadPool = Executors.newFixedThreadPool(count);
List<Future<Long>> list = new ArrayList<>();
for (int i = 0; i < count; i++) {
int num = i;
// 分段提交任务
Future<Long> future = threadPool.submit(() -> {
long sum = 0;
for (int j = arr.length / count * num; j < (arr.length / count * (num + 1)); j++) {
try {
// 模拟耗时
sum += (arr[j]/3*3/3*3/3*3/3*3/3*3);
} catch (Exception e) {
e.printStackTrace();
}
}
return sum;
});
list.add(future);
}
// 每个段结果相加
long sum = 0;
for (Future<Long> future : list) {
sum += future.get();
}
System.out.println("sum: " + sum);
System.out.println("multi thread elapse: " + (System.currentTimeMillis() - start));
}
private static void forkJoinSum(long[] arr) throws ExecutionException, InterruptedException {
long start = System.currentTimeMillis();
ForkJoinPool forkJoinPool = ForkJoinPool.commonPool();
// 提交任务
ForkJoinTask<Long> forkJoinTask = forkJoinPool.submit(new SumTask(arr, 0, arr.length));
// 获取结果
Long sum = forkJoinTask.get();
forkJoinPool.shutdown();
System.out.println("sum: " + sum);
System.out.println("fork join elapse: " + (System.currentTimeMillis() - start));
}
private static class SumTask extends RecursiveTask<Long> {
private long[] arr;
private int from;
private int to;
public SumTask(long[] arr, int from, int to) {
this.arr = arr;
this.from = from;
this.to = to;
}
@Override
protected Long compute() {
// 小于1000的时候直接相加，可灵活调整
if (to - from <= 1000) {
long sum = 0;
for (int i = from; i < to; i++) {
// 模拟耗时
sum += (arr[i]/3*3/3*3/3*3/3*3/3*3);
}
return sum;
}
// 分成两段任务，本文由公从号“彤哥读源码”原创
int middle = (from + to) / 2;
SumTask left = new SumTask(arr, from, middle);
SumTask right = new SumTask(arr, middle, to);
// 提交左边的任务
left.fork();
// 右边的任务直接利用当前线程计算，节约开销
Long rightResult = right.compute();
// 等待左边计算完毕
Long leftResult = left.join();
// 返回结果
return leftResult + rightResult;
}
}
}复制代码Tong Brother secretly tell you, in fact, calculate 100 million integer add, is the fastest single-threaded, my computer is probably around 100ms, use the thread pool but will slow down.
So, in order to demonstrate Niubi place ForkJoinPool, I put each number/3*3/3*3/3*3/3*3/3*3Meal operation, for simulation time consuming.
See the results:
sum: 107352457433800662
single thread elapse: 789
sum: 107352457433800662
multi thread elapse: 228
sum: 107352457433800662
fork join elapse: 189复制代码You can see, ForkJoinPool relatively common thread pool still has much room for improvement.
Question: common thread pool can be achieved ForkJoinPool this way it is calculated that the big task demolition tasks, tasks demolition of small tasks, and finally a summary?
You can try (-᷅_-᷄)
OK, let's formal entry into the analytical ForkJoinPool.
Minute root treatment
- The basic idea
To a large-scale problem is divided into smaller sub-problems, then divide and rule, de-merger last sub-problem to get solution of the original problem.
- step
(1) split the original question:
(2) sub-problem solving:
The combined solution of the problem of child (3) as a solution to the original problem.
In the divide and conquer, the sub-question is generally independent of each other, therefore, often sub-problems solved by calling a recursive algorithm.
- Typical application scenarios
(1) binary search
(2) large integer multiplication
(3) Strassen matrix multiplication
(4) covers the chessboard
(5) merge sort
(6) quicksort
(7) Linear timing
(8) Tower of Hanoi
ForkJoinPool inheritance system
ForkJoinPool is java 7 in the new thread pool class, its successor system are as follows:
ForkJoinPool and ThreadPoolExecutor AbstractExecutorService are inherited from the abstract class, so it ThreadPoolExecutor use hardly any difference, in addition to the task became outside ForkJoinTask.
Here again we applied to a very important design principle - the principle of opening and closing - closed for modification, open for extension.
Interface design visible throughout the system thread pool is a very good beginning, a new thread pool class, it will not interfere with the original code, but also to take advantage of existing features.
ForkJoinTask
The two main methods
- fork()
fork () method is similar to the thread Thread.start () method, but it is not really start a thread, but the work put into the task queue.
- join()
join () method is similar to the thread Thread.join () method, but it is not simply blocking the thread, but to use the worker to run other tasks. When a worker thread calls the join () method, which will handle other tasks, notes that until the target sub-task has been completed.
Three subclasses
- RecursiveAction
No return value tasks.
- RecursiveTask
It returns a value tasks.
- CountedCompleter
After no return value tasks, complete the task can trigger a callback.
ForkJoinPool internal principle
Internal ForkJoinPool using the "work-stealing" algorithm implementation.
(1) Each worker thread has its own job queue WorkQueue;
(2) which is a deque, which is private to the thread;
(3) ForkJoinTask fork in the sub-tasks will run into the task work queue head thread, the worker thread will LIFO order to process the task work queue;
(4) In order to maximize a CPU, a thread from the idle queue, other threads to "steal" tasks are performed;
(5) task steal from the end of the work queue, in order to reduce competition;
(6) Operation deque: push () / pop () at which the worker thread calls the owner only, poll () is called when the task stolen by other threads;
(7) When there are only a task, or there will be competition, is achieved by CAS;
ForkJoinPool Best Practices
(1) The most suitable are compute-intensive tasks, known from the article by the original number "Brother Tong read source";
(2) when the worker thread needs to block, you can use ManagedBlocker;
(3) should not be in RecursiveTask
to sum up
(1) ForkJoinPool particularly suitable for the realization of "divide and conquer" algorithm;
(2) ForkJoinPool and ThreadPoolExecutor are complementary, not who substitute who relationship between the two different scenarios applicable;
(3) ForkJoinTask method has two core --fork () and join (), there are three important subclasses --RecursiveAction, RecursiveTask and CountedCompleter;
(4) ForkjoinPool based on internal "work-stealing" algorithm;
(5) Each thread has its own work queue, which is a deque, their access to the head of the queue task, other tasks from the tail steal thread;
(6) ForkJoinPool best suited for computationally intensive tasks, it is also possible to use for blocking ManagedBlocker type task;
(7) RecursiveTask interior can be called one less fork (), using the current thread, which is a skill;
Egg
ManagedBlocker how to use?
A: ManagedBlocker equivalent framework to explicitly tell ForkJoinPool blocked, ForkJoinPool will start another thread to run tasks in order to maximize the use of the CPU.
Consider the following example, her own ha ^ ^.
/**
* 斐波那契数列
* 一个数是它前面两个数之和
* 1,1,2,3,5,8,13,21
*/
public class Fibonacci {
public static void main(String[] args) {
long time = System.currentTimeMillis();
Fibonacci fib = new Fibonacci();
int result = fib.f(1_000).bitCount();
time = System.currentTimeMillis() - time;
System.out.println("result，本文由公从号“彤哥读源码”原创 = " + result);
System.out.println("test1_000() time = " + time);
}
public BigInteger f(int n) {
Map<Integer, BigInteger> cache = new ConcurrentHashMap<>();
cache.put(0, BigInteger.ZERO);
cache.put(1, BigInteger.ONE);
return f(n, cache);
}
private final BigInteger RESERVED = BigInteger.valueOf(-1000);
public BigInteger f(int n, Map<Integer, BigInteger> cache) {
BigInteger result = cache.putIfAbsent(n, RESERVED);
if (result == null) {
int half = (n + 1) / 2;
RecursiveTask<BigInteger> f0_task = new RecursiveTask<BigInteger>() {
@Override
protected BigInteger compute() {
return f(half - 1, cache);
}
};
f0_task.fork();
BigInteger f1 = f(half, cache);
BigInteger f0 = f0_task.join();
long time = n > 10_000 ? System.currentTimeMillis() : 0;
try {
if (n % 2 == 1) {
result = f0.multiply(f0).add(f1.multiply(f1));
} else {
result = f0.shiftLeft(1).add(f1).multiply(f1);
}
synchronized (RESERVED) {
cache.put(n, result);
RESERVED.notifyAll();
}
} finally {
time = n > 10_000 ? System.currentTimeMillis() - time : 0;
if (time > 50)
System.out.printf("f(%d) took %d%n", n, time);
}
} else if (result == RESERVED) {
try {
ReservedFibonacciBlocker blocker = new ReservedFibonacciBlocker(n, cache);
ForkJoinPool.managedBlock(blocker);
result = blocker.result;
} catch (InterruptedException e) {
throw new CancellationException("interrupted");
}
}
return result;
// return f(n - 1).add(f(n - 2));
}
private class ReservedFibonacciBlocker implements ForkJoinPool.ManagedBlocker {
private BigInteger result;
private final int n;
private final Map<Integer, BigInteger> cache;
public ReservedFibonacciBlocker(int n, Map<Integer, BigInteger> cache) {
this.n = n;
this.cache = cache;
}
@Override
public boolean block() throws InterruptedException {
synchronized (RESERVED) {
while (!isReleasable()) {
RESERVED.wait();
}
}
return true;
}
@Override
public boolean isReleasable() {
return (result = cache.get(n)) != RESERVED;
}
}
}复制代码I welcome the attention of the public number "Tong brother read source" view source code more series, and brother Tong source of ocean swim together.