Fork/Join
Fork/Join框架是Java 7提供的一个用于并行执行任务的框架,是一个把大任务分割成若干个小任务,最终汇总每个小任务结果后得到大任务结果的框架。Fork/Join框架要完成两件事情:
- 任务分割:首先Fork/Join框架需要把大的任务分割成足够小的子任务,如果子任务比较大的话还要对子任务进行继续分割
- 执行任务并合并结果:分割的子任务分别放到双端队列里,然后几个启动线程分别从双端队列里获取任务执行。子任务执行完的结果都放在另外一个队列里,启动一个线程从队列里取数据,然后合并这些数据。
Fork/Join框架需要通过两个类来实现上述功能
ForkJoinTask:先创建一个ForkJoin任务,可以继承ForkJoinTask其中的两个子类
- RecursiveAction:用于没有返回结果的任务
- RecursiveTask:用于有返回结果的任务
ForkJoinPool:一个ForkJoin任务需要通过ForkJoinPool来执行
使用RecursiveTask的一个简单示例:使用Fork/Join框架计算10亿个2的和
/**
* @CalssName RecursiveTaskDemo
* @Description 使用Fork/Join框架计算10亿个2的和
* @since JDK 1.8
*/
public class RecursiveTaskDemo {
private static final int ARRAY_SIZE = 10_0000_0000;
public static int[] findArr() {
int[] array = new int[ARRAY_SIZE];
for (int i = 0; i < ARRAY_SIZE; i++) {
array[i] = 2;
}
return array;
}
//有返回值的task
public static class SumTask extends RecursiveTask<Integer> {
private static final int THRESHOLD = ARRAY_SIZE / 10;//阈值,拆分的最小单位
private int[] arr;
private int left;
private int right;
public SumTask(int[] arr, int left, int right) {
this.arr = arr;
this.left = left;
this.right = right;
}
@Override
protected Integer compute() {
if ((right - left) > THRESHOLD) {
//大于拆分的最小单位,继续拆分
int middle = (right + left) / 2;
SumTask leftTask = new SumTask(this.arr, left, middle);
SumTask rightTask = new SumTask(this.arr, middle + 1, right);
//拆分
leftTask.fork();
rightTask.fork();
//或者
// invokeAll(leftTask, rightTask);
//合并
return leftTask.join() + rightTask.join();
} else {
int count = 0;
for (int i = left; i <= right; i++) {
count += arr[i];
}
return count;
}
}
}
public static void main(String[] args) {
int[] arr = findArr();
ForkJoinPool forkJoinPool = new ForkJoinPool();
SumTask sumTask = new SumTask(arr, 0, arr.length - 1);
long start = System.currentTimeMillis();
forkJoinPool.invoke(sumTask);
System.out.println("sum:" + sumTask.join() + " 用时:" + (System.currentTimeMillis() - start) + "毫秒");
//sum:2000000000 用时:259毫秒
// start = System.currentTimeMillis();
// int count = 0;
// for (int i = 0; i < ARRAY_SIZE; i++) {
// count += arr[i];
// }
// System.out.println("sum:" + count + " 用时:" + (System.currentTimeMillis() - start) + "毫秒");
//sum:2000000000 用时:445毫秒
}
}
RecursiveAction示例:搜索文件
/**
* @CalssName RecursiveActionDemo
* @Description 搜索文件
* @since JDK 1.8
*/
public class RecursiveActionDemo {
public static void main(String[] args) {
ForkJoinPool pool = new ForkJoinPool();
SearchFileTask searchFileTask = new SearchFileTask(new File("D:/"), "spring-cloud");
pool.execute(searchFileTask);
searchFileTask.join();
}
}
class SearchFileTask extends RecursiveAction {
private File file;
private String searchKey;
public SearchFileTask(File file, String searchKey) {
this.file = file;
this.searchKey = searchKey;
}
@Override
protected void compute() {
List<SearchFileTask> searchFileTasks = new ArrayList<>();
File[] files = file.listFiles();
if (files != null) {
for (File file1 : files) {
if (file1.isDirectory()) {
searchFileTasks.add(new SearchFileTask(file1, searchKey));
} else {
//文件
if (file1.getAbsolutePath().indexOf(searchKey) != -1) {
System.out.println("文件:" + file1.getAbsolutePath());
}
}
}
if (!searchFileTasks.isEmpty()) {
for (SearchFileTask task : invokeAll(searchFileTasks)) {
task.join();
}
}
}
}
}
分而治之:分而治之就是将一个大任务层层拆分成一个个的小任务,直到不可拆分,拆分依据定义的阈值划分任务规模。
fork/join通过fork将大任务拆分成小任务,在将小任务的结果join汇总
CountDownLatch
CountDownLatch的作用是等待一组线程执行完后在执行等待的线程
示例
/**
* @CalssName CountDownLatchDemo
* @Description CountDownLatch示例
* @since JDK 1.8
*/
public class CountDownLatchDemo {
//计数5
static CountDownLatch latch = new CountDownLatch(5);
static class sumRunnable implements Runnable {
@Override
public void run() {
latch.countDown();//每执行一次,计数减1
System.out.println("求和计算。。。");
}
}
static class subRunnable implements Runnable {
@Override
public void run() {
latch.countDown();
System.out.println("减法计算。。。");
}
}
public static void main(String[] args) {
try {
new Thread(new sumRunnable()).start();
new Thread(new sumRunnable()).start();
new Thread(new subRunnable()).start();
new Thread(new subRunnable()).start();
new Thread(new sumRunnable()).start();
latch.await();//直到调用了countDown()方法的线程执行完,计数减到0为止,才会继续执行main方法
System.out.println("执行main方法。。。");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
CyclicBarrier
线程运行时约定在指定位置等待其他线程都运行到等待位置时一起继续执行
示例
/**
* @CalssName CyclicBarrierDemo
* @Description CyclicBarrier示例
* @since JDK 1.8
*/
public class CyclicBarrierDemo {
static CyclicBarrier barrier = new CyclicBarrier(3);
public static void main(String[] args) {
new Thread(new InitRunnable()).start();
new Thread(new InitRunnable()).start();
new Thread(new InitRunnable()).start();
}
//初始化线程
static class InitRunnable implements Runnable {
@Override
public void run() {
try {
System.out.println("初始化前....");
Random random = new Random();
if (random.nextBoolean()) {
Thread.sleep(3000);
}
barrier.await();
System.out.println("初始化后....");
} catch (BrokenBarrierException | InterruptedException e) {
e.printStackTrace();
}
}
}
}
运行结果:
初始化前....
初始化前....
初始化前....
初始化后....
初始化后....
初始化后....
运行数据收集汇总
/**
* @CalssName CyclicBarrierDemo
* @Description CyclicBarrier示例
* @since JDK 1.8
*/
public class CyclicBarrierDemo {
static CyclicBarrier barrier = new CyclicBarrier(3, new CollectRunnable());
static ConcurrentHashMap<Long, String> collocatMap = new ConcurrentHashMap<>();
public static void main(String[] args) {
new Thread(new InitRunnable(), "线程1").start();
new Thread(new InitRunnable()).start();
new Thread(new InitRunnable()).start();
}
//收集数据汇总
static class CollectRunnable implements Runnable {
@Override
public void run() {
System.out.println("在运行线程:" + collocatMap);
}
}
//初始化线程
static class InitRunnable implements Runnable {
@Override
public void run() {
long threadId = Thread.currentThread().getId();
collocatMap.put(threadId, Thread.currentThread().getName());
try {
System.out.println("初始化前....");
Random random = new Random();
if (random.nextBoolean()) {
Thread.sleep(3000);
}
barrier.await();
System.out.println("初始化后....");
} catch (BrokenBarrierException | InterruptedException e) {
e.printStackTrace();
}
}
}
}
Semaphore信号量
控制某个资源的同时访问线程数量,用于流量控制
连接池示例
/**
* @CalssName DBPool
* @Description 数据库链接池工具
* @since JDK 1.8
*/
public class DBPool {
private static LinkedList<Connection> pool = new LinkedList<>();
private static String url = "jdbc:mysql://192.168.86.101:3306/mysql";
private static String username = "root";
private static String password = "my-pwd";
private final Semaphore useful;
/**
* @return java.sql.Connection
* @Title getConnection
* @Description 从链接池中获取链接,如果没有链接了在指定有效时间内获取链接,不然返回null
* @Throws InterruptedException
*/
public Connection getConnection() throws InterruptedException {
useful.acquire();//获取,减1
Connection connection;
synchronized (pool) {
connection = pool.removeFirst();
System.out.println("连接池中还有"+useful.availablePermits()+"个链接,阻塞线程为"+useful.getQueueLength()+"个");
}
return connection;
}
/**
* @Title 回收链接
* @Param [connection]
* @return void
*/
public void recyclingConnection(Connection connection) {
useful.release();//释放,加1,acquire()方法往下执行
if (connection != null) {
synchronized (pool) {
pool.addLast(connection);
}
}
}
/**
* @CalssName DBPool
* @Description 初始化链接池
* @since JDK 1.8
*/
public DBPool(int initSize) {
this.useful = new Semaphore(initSize);
if (initSize > 0) {
try {
Class.forName("com.mysql.jdbc.Driver");
for (int i = 0; i < initSize; i++) {
Connection connection = DriverManager.getConnection(url, username, password);
pool.add(connection);
}
} catch (ClassNotFoundException | SQLException e) {
e.printStackTrace();
}
}
}
}
测试
/**
* @CalssName App
* @Description Semaphore信号量测试
* @since JDK 1.8
*/
public class App {
static DBPool dbPool = new DBPool(10);
private static class BusiThread implements Runnable {
@Override
public void run() {
Connection conn = null;
Random random = new Random();
try {
conn = dbPool.getConnection();
//模拟业务处理
Thread.sleep(10 * random.nextInt(500));
dbPool.recyclingConnection(conn);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public static void main(String[] args) {
DBPool pool = new DBPool(10);
for (int i = 0; i < 50; i++) {
new Thread(new BusiThread()).start();
}
}
}