Timing task scheduling scheme

With the development of the system scale, the organizational structure and structure of the project become more and more complex, the scope of business coverage is wider and wider, the number of scheduled tasks is increasing, and the tasks are becoming more and more complex, especially in order to meet the needs of users When the calendar increases, the system can run stably, and we often expand servers to form clusters, whether it is a traditional vertical project or a mainstream distributed one. Then the requirements for timing tasks are gradually becoming higher. Based on the current mainstream project architecture, timing tasks need to meet the following requirements:

1. Unified task management, providing a graphical interface to configure and schedule tasks.
1. Guarantee the idempotence of task scheduling (task concurrency control, the same task can only allow one execution at the same time)
1. Task elastic expansion, can dynamically increase or decrease the server to share the pressure according to the busy situation, and perform fragmentation processing for large tasks.
1. The problem of task dependence can handle the situation that a task contains subtasks, and the execution of subtasks is triggered after the previous one is completed.
1. Support multiple types of tasks, support Spring Bean, Shell, etc.
1. The task node is highly available, and the task node can be transferred to other nodes for execution when it is abnormal or busy.
1. The dispatch center is highly available and supports cluster deployment to avoid single point of failure.
1. Execution status monitoring, easy to view task execution status, abnormal situation alarm, support multi-channel notification.

#. Development history:
With the development of technology, scheduled tasks have changed from single-thread scheduling to multi-thread scheduling, from single-machine deployment to cluster deployment, and from independent execution to multi-task collaborative execution. Please add a picture description

1. Thread

Implemented through thread sleep, provided by JDK

private static int count = 0;

public static void main(String[] args) {
    
    
  Runnable runnable = new Runnable() {
    
    
    @Override
    public void run() {
    
    
      while (count < 8) {
    
    
        try {
    
    
          Thread.sleep(1000);
          System.out.printf("执行第：%d 次\n", ++count);
        } catch (InterruptedException e) {
    
    
          e.printStackTrace();
        }
      }
    }
  };
  new Thread(runnable).start();
}

Please add a picture description

2. Thread pool

There is a deferred execution implementation, provided by the JDK

private static int count = 0;

public static void main(String[] args) throws InterruptedException {
    
    
  for (int i = 0; i < 8; i++) {
    
    
    ExecutorService executorService = Executors.newSingleThreadScheduledExecutor();
    Runnable runnable = new Runnable() {
    
    
      @Override
      public void run() {
    
    
        ++count;
        System.out.printf("执行从 %d 次\n", count);
        if (count > 7) {
    
    
          System.exit(0);
        }
      }
    };
    // 也可以使用 ScheduledExecutorService.schedule
    executorService.awaitTermination(1L, TimeUnit.SECONDS);
    executorService.execute(runnable);
  }
}

Please add a picture description

3. Timer TimeTask

Provided by JDK, mostly used for mobile terminals (such as Android development)

private static int count = 0;

public static void main(String[] args) {
    
    
    TimerTask timerTask = new TimerTask() {
    
    
        @Override
        public void run() {
    
    
            System.out.printf("执行第 %d 次\n", ++count);
            if (count > 7) {
    
    
                System.exit(0);
            }
        }
    };
    Timer timer = new Timer();
    timer.schedule(timerTask, 500, 500);
}

Please add a picture description

4. Schedule

Provided by Spring, used with @EnableScheduling

@Component
public class SpringScheduleTest {
    
    

    private static int count;

    @Scheduled(cron = "0/1 * * * * ?")
    public void testSchedule() {
    
    
        System.out.printf("执行第 %d 次\n", ++count);
    }
}

cron expression: second-minute-hour-day-month-week-year

5.quartz

There are maven independent dependencies, and the package comes with SpringBoot2.0

Pure maven project dependencies

<!-- 核心包 -->
<dependency>
   <groupId>org.quartz-scheduler</groupId>
   <artifactId>quartz</artifactId>
   <version>2.3.0</version>
</dependency>

<!-- 工具包 -->
<dependency>
   <groupId>org.quartz-scheduler</groupId>
   <artifactId>quartz-jobs</artifactId>
   <version>2.3.0</version>
</dependency>

SpringBoot2.0 and above dependencies

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-quartz</artifactId>
</dependency>

Test class Test.java

public class Test {
      
      

    public static void main(String[] args) throws SchedulerException, ParseException {
      
      
        Scheduler factory = StdSchedulerFactory.getDefaultScheduler();
        JobDataMap jobDataMap = new JobDataMap();
        jobDataMap.put("tips", "去你大爷的");
        JobDetail jobDetail = JobBuilder.newJob(QuartzJob.class)
                .withIdentity("testJob", "testJobGroup")
                .setJobData(jobDataMap).build();
        Trigger trigger = TriggerBuilder.newTrigger()
                .withIdentity("testTrigger", "testTriggerGroup")
                .withSchedule(SimpleScheduleBuilder.repeatSecondlyForever(5))
                .startNow().build();
        factory.scheduleJob(jobDetail, trigger);
        factory.start();
    }
}

Create task class QuartzJob.java

public class QuartzJob implements Job {
      
      
    public static int count;
    @Override
    public void execute(JobExecutionContext jobExecutionContext) throws JobExecutionException {
      
      
        JobDataMap jobDataMap = jobExecutionContext.getMergedJobDataMap();
        String tips = jobDataMap.getString("tips");
        System.out.printf("获取到参数：%s\n", tips);
        System.out.printf("执行第：%d 次\n", ++count);
    }
}

6. Xxl-job

XXL-JOB is a lightweight distributed task scheduling platform. Its core design goals are rapid development, easy learning, lightweight, and easy expansion. The source code is now open and connected to the online product lines of many companies, out of the box.
Please add a picture description
For detailed content introduction, please check: document address

Create a library and create a table. The sql file is in the project directory: xxl-job-master/doc/db/tables_xxl_job.sql Note: Its
database name is xxl_job. If you don’t want to create a new library, but import it to an existing library , you need to enter the sql file to change the name.
Run the project: xxl-job-admin
1. The project structure is as follows:
2. Modify the mysql connection configuration as follows:
After running, visit in the browser: http://localhost:8080/xxl-job-admin

Default account: admin Password: 123456
Enter the dispatch center after successful login

To register a task executor, you can create an executor service separately or integrate it into an existing project service.
The following is an example of creating a new SpringBoot project

Create a SpringBoot project and add dependencies

<dependency>
  <groupId>com.xuxueli</groupId>
  <artifactId>xxl-job-core</artifactId>
  <version>2.3.1</version>
</dependency>

Create the application.properties file in the resources directory (the same is true for the yaml file, if there is, you don't need to create it)

# 执行器所在的服务的运行端口， 非执行器端口
server.port=8081


### xxl-job admin address list, such as "http://address" or "http://address01,http://address02"
# 注册地址， 这个地址其实就是部署或者运行完xxl-job-admin调度中心的浏览器访问地址
xxl.job.admin.addresses=http://127.0.0.1:8080/xxl-job-admin

### xxl-job, access token
xxl.job.accessToken=default_token

### xxl-job executor appname
### 执行器名称，需要配置到调度中心
xxl.job.executor.appname=xxl-job-executor
### xxl-job executor registry-address: default use address to registry , otherwise use ip:port if address is null
xxl.job.executor.address=
### xxl-job executor server-info
xxl.job.executor.ip=
# 这个端口是执行器端口， 不是这个项目server.port 端口，也不能一样，否则会出现端口占用等错误信息
# 也就是说，执行器项目部署或者运行起来会有两个端口，如果是用docker部署，注意两个端口都要映射到宿主机端口
xxl.job.executor.port=9999
### xxl-job executor log-path
xxl.job.executor.logpath=/data/applogs/xxl-job/jobhandler
### xxl-job executor log-retention-days
xxl.job.executor.logretentiondays=30

Create configuration class XxlJobConfig.java

@Configuration
public class XxlJobConfig {
        
        

    @Value("${xxl.job.admin.addresses}")
    private String adminAddresses;

    @Value("${xxl.job.accessToken}")
    private String accessToken;

    @Value("${xxl.job.executor.appname}")
    private String appname;

    @Value("${xxl.job.executor.address}")
    private String address;

    @Value("${xxl.job.executor.ip}")
    private String ip;

    @Value("${xxl.job.executor.port}")
    private int port;

    @Value("${xxl.job.executor.logpath}")
    private String logPath;

    @Value("${xxl.job.executor.logretentiondays}")
    private int logRetentionDays;

    @Bean
    public XxlJobSpringExecutor xxlJobExecutor() {
        
        
        XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
        xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
        xxlJobSpringExecutor.setAppname(appname);
        xxlJobSpringExecutor.setAddress(address);
        xxlJobSpringExecutor.setIp(ip);
        xxlJobSpringExecutor.setPort(port);
        xxlJobSpringExecutor.setAccessToken(accessToken);
        xxlJobSpringExecutor.setLogPath(logPath);
        xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);
        return xxlJobSpringExecutor;
    }
}

Create a task (Job), note that the wording of different versions will be slightly different

@Component
public class SampleXxlJob {
        
        

    /**
     * 1、简单任务示例（Bean模式）
     */
    @XxlJob("demoJobHandler")
    public void demoJobHandler() throws Exception {
        
        
        XxlJobHelper.log("XXL-JOB, Hello World.");
        for (int i = 0; i < 5; i++) {
        
        
            XxlJobHelper.log("beat at:" + i);
            TimeUnit.SECONDS.sleep(2);
        }
        XxlJobHelper.handleSuccess("XXL-JOB：执行成功");
    }
}

configure the executor
1. configuration
2. success
Configure tasks and execute
1. configuration tasks
2. execute once
3. view log