Recently, due to work needs, it is necessary to develop an executor task based on the xxljob scheduler. The main business of the executor is to collect data regularly from the flow table, classify and count the flow data according to different dimensions, process them and store them in the oracle database. Since all data processing dimensions do not depend on each other, asynchronous multi-thread scheduling execution based on thread pool is adopted here.
1. Development environment
虚拟机vmware16
centos7
jdk8
springboot2.3
xxljob2.3.0
mybatis-plus
2. Specific implementation
This time we mainly used the Springboot2.3 thread pool non-scheduled task class ThreadPoolTaskExecutor. The main advantage of this class is that it realizes multithreading and thread pool at the same time. It is no longer necessary to create a Thread class or implement the Runnable interface when using multithreading in the past, and use the thread pool to create Executors. Now we only need to add @EnableAsync in the startup file to use multithreading, and add @Async to the thread business class file or thread business method to use the thread pool provided by ThreadPoolTaskExecutor. At the same time, the thread pool is called asynchronously, which is very practical .
2.1 Create a thread pool
Create a thread pool configuration file AsyncConfig. The content is as follows:
@Configuration
@EnableAsync
public class AsyncConfig implements AsyncConfigurer {
@Override
@Bean
public ThreadPoolTaskExecutor getAsyncExecutor() {
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
// 核心线程数：线程池创建的时候初始化的线程数
taskExecutor.setCorePoolSize(10);
// 最大线程数：线程池最大的线程数，只有缓冲队列满了之后才会申请超过核心线程数的线程
taskExecutor.setMaxPoolSize(100);
// 缓冲队列：用来缓冲执行任务的队列
taskExecutor.setQueueCapacity(50);
// 线程池关闭：等待所有任务都完成再关闭
taskExecutor.setWaitForTasksToCompleteOnShutdown(true);
// 等待时间：等待5秒后强制停止
taskExecutor.setAwaitTerminationSeconds(5);
// 允许空闲时间：超过核心线程之外的线程到达60秒后会被销毁
taskExecutor.setKeepAliveSeconds(60);
// 线程名称前缀
taskExecutor.setThreadNamePrefix("taskExecutor--");
taskExecutor.initialize();
return taskExecutor;
}
}
2.2 Enable multi-threaded tasks
There are two ways to enable multi-threaded tasks.
The first one can add the @EnableAsync annotation to the startup class
@Configuration
@EnableAsync
public class AsyncConfig implements AsyncConfigurer {
@Override
@Bean
public ThreadPoolTaskExecutor getAsyncExecutor() {
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
……
taskExecutor.initialize();
return taskExecutor;
}
}
The second can add @EnableAsync to the thread pool configuration file
@SpringBootApplication
@EnableAsync
public class ProcessFlowExecutorApplication {
public static void main(String[] args) {
SpringApplication.run(ProcessFlowExecutorApplication.class, args);
}
}
I added it to the thread pool configuration file this time.
2.3 The service layer uses the @Async annotation to implement asynchronous thread pool tasks
The service layer interface code is as follows:
public interface ChannelService extends IService<Channel> {
public void saveBatchChannel(int i);
}
The service layer implementation class code is as follows:
@Service
public class ChannelServiceImpl extends ServiceImpl<ChannelMapper, Channel> implements ChannelService {
@Async
public void saveBatchChannel(int i) {
System.out.println(" 执行异步任务：" + i);
}
}
2.4 The controller layer calls the business layer
@RestController
public class IndexController {
@Autowired
private ChannelService channelService;
@RequestMapping("/userlist")
public void getUserList() {
for (int i = 0; i < 10; i++) {
channelService.saveBatchChannel(i);
}
}
}
2.5 The printed result is
3 Conclusion
Well, that's all for today's article. If you think this article is useful, please like it and bookmark it. Your encouragement is my biggest motivation. If you think there is a problem, you can leave a message in the comment area, criticism and correction are welcome. Learning from each other.
