SpringBoot asynchronous task (2) | (thread pool use)

SpringBoot asynchronous task (2) | (thread pool use)

---

Chapter
Link to Chapter 1: SpringBoot Asynchronous Task (1) | (Asynchronous task execution and callback)

Preface

Thread pool enables the use of asynchronous tasks in springboot

1. Usage scenarios

There is a batch scheduling task in the project. The customer uploads batches of articles, and then these articles are processed for the task.

2. Add asynchronous tasks to springboot

1. Configure the thread pool

Configure the thread pool in the springboot container, and then directly inject the bean for subsequent use.

@Configuration
@EnableAsync
public class ExecutorEmbPoolConfig {
    
    

    private static final Logger logger = LoggerFactory.getLogger(ExecutorEmbPoolConfig.class);

    @Value("${embedding.pool.corePoolSize:20}")
    private int corePoolSize = 20;
    @Value("${embedding.pool.maxPoolSize:20}")
    private int maxPoolSize = 20;
    @Value("${embedding.pool.queueCapacity:100000}")
    private int queueCapacity = 100000;

    private String namePrefix = "embedding-service-";

    @Bean(name = "embeddingServiceExecutor")
    public ThreadPoolTaskExecutor asyncServiceExecutor() {
    
    
        logger.debug("start embedding embeddingServiceExecutor");
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        //配置核心线程数
        executor.setCorePoolSize(corePoolSize);
        //配置最大线程数
        executor.setMaxPoolSize(maxPoolSize);
        //配置队列大小
        executor.setQueueCapacity(queueCapacity);
        //配置线程池中的线程的名称前缀
        executor.setThreadNamePrefix(namePrefix);
        // 允许回收核心线程
        executor.setAllowCoreThreadTimeOut(true);
        // CALLER_RUNS: 不在新线程中执行任务，而是有调用者所在的线程来执行
        executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
        //执行初始化
        executor.initialize();
        return executor;
    }
}

2. Use of thread pool

  @Resource(name = "embeddingServiceExecutor")
  private ThreadPoolTaskExecutor executor;

  @Scheduled(cron = "0/30 * * * * ?")
  public void FileToMilvesJob() {
    
    
        //定义计数器
        List<DocumentMilvusRecord> documentMilvusRecords = recordService.findByStatus(RecordStatus.WAIT);
        if (CollectionUtils.isEmpty(documentMilvusRecords)) {
    
    
            return;
        }
        List<DocumentMilvusRecord> excuteList;
        if (documentMilvusRecords.size() > 50) {
    
    
            excuteList = documentMilvusRecords.subList(0, 50);
        } else {
    
    
            excuteList = documentMilvusRecords;
        }
        log.info("本次任务需要执行任务“{}条", excuteList.size());
        for (DocumentMilvusRecord record : excuteList) {
    
    
            recordService.updateRecordStatus(record);
            executor.execute(() -> {
    
    
                try {
    
    
                    docEmbeddingCreate(record); // 执行业务逻辑
                } catch (Exception e) {
    
    
                    log.error(e.getMessage());
                }
            });
        }


    }

Summarize

The above method implements a custom thread pool, and then obtains the thread pool and executes the task when executing the task.