The originator of the distributed task scheduling framework: Quartz

1 Introduction

Recently, our company has launched a distributed task scheduling framework: XXL-JOB to facilitate the management and control of tasks. Originally, I wanted to talk about the framework, but in the process of learning and understanding, I found that the framework was Quartzdeveloped and implemented based on ideas. It is Quartza very popular open source task scheduling framework. JavaOr the reference standard, so let's talk about the framework first Quartz.

1.1. What is Quartz

Quartz is another open source project of the OpenSymphony open source organization in the field of Job scheduling. It is an open source task schedule management system completely developed by java. A system that executes (or notifies) other software components. Its functionality is similar to java.util.Timer. But compared to Timer, Quartz has added many functions. As an excellent open source scheduling framework, Quartz has the following characteristics:

Powerful scheduling functions, such as supporting a variety of scheduling methods, can meet various conventional and special needs
Flexible application methods, supporting multiple storage methods for scheduling data
Distributed and Cluster Capabilities

1.2. Storage method

The comparison between RAMJobStore and JDBCJobStore is as follows:

type	advantage	shortcoming
RAMJobStore	No external database, easy to configure, fast to run	Because the scheduler information is stored in memory allocated to the JVM, all scheduling information is lost when the application stops running. In addition, because it is stored in the JVM memory, the number of Jobs and Triggers that can be stored will be limited.
JDBCJobStore	Support clusters , because all task information will be saved in the database, you can control things, and if the application server is shut down or restarted, the task information will not be lost, and you can restore tasks that fail to execute due to server shutdown or restart	The speed of operation depends on the speed of connecting to the database

According to the above, if you want to support distributed clusters, you must belong to it JDBCJobStore. It needs to rely on the database MySQL, the database initialization table SQL download: tables , and the table description is as follows:

Table Name	illustrate
qrtz_blob_triggers	Trigger is stored as a Blob type (used when Quartz users use JDBC to create their own custom Trigger type, and the JobStore does not know how to store the instance)
qrtz_calendars	Quartz's Calendar calendar information is stored in Blob type. Quartz can configure a calendar to specify a time range
qrtz_cron_triggers	Stores Cron Triggers, including Cron expressions and time zone information
qrtz_fired_triggers	Store the status information related to the triggered Trigger, as well as the execution information of the associated Job
qrtz_job_details	Store the details of each configured Job
qrtz_locks	Stores information about non-pessimistic locks of the program (if pessimistic locks are used)
qrtz_paused_trigger_graps	Stores information about paused Trigger groups
qrtz_scheduler_state	Store a small amount of state information about the Scheduler, and other Scheduler instances (if used in a cluster)
qrtz_simple_triggers	Store a simple Trigger, including the number of repetitions, intervals, and the number of touches
qrtz_triggers	Store configured Trigger information

Project recommendation : Based on the encapsulation of the underlying framework of SpringBoot2.x, SpringCloud and SpringCloudAlibaba enterprise-level system architecture, it solves common non-functional requirements during business development, prevents repeated wheel creation, and facilitates rapid business development and unified management of enterprise technology stack frameworks. Introduce the idea of componentization to achieve high cohesion and low coupling and highly configurable, so as to be pluggable. Strictly control package dependencies and unified version management to minimize dependencies. Pay attention to code specifications and comments, very suitable for personal learning and enterprise use

Github address : https://github.com/plasticene/plasticene-boot-starter-parent

Gitee address : https://gitee.com/plasticene3/plasticene-boot-starter-parent

WeChat public account : Shepherd Advanced Notes

Exchange discussion group: Shepherd_126

2. springboot integration example

It is very simple to integrate quartz with springboot. Here we demonstrate the cluster mode, so using it JDBCJobStore, the related dependencies are as follows:

        <!-- 实现对 Quartz 的自动化配置 -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-quartz</artifactId>
        </dependency>

        <!-- 实现对数据库连接池的自动化配置 -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>

        <dependency> <!-- 本示例，我们使用 MySQL -->
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.48</version>
        </dependency>

Before creating a task, we need to download the above SQL statement and execute it. Here, the table can be built in the same database as the business database, or it can be placed in a separate database. If the database and table are built separately, then the business service is a multi-data source. The data source connection needs to be repackaged. The following multi-data source configuration:

@Configuration
public class DataSourceConfiguration {

    /**
     * 创建 user 数据源的配置对象
     */
    @Primary
    @Bean(name = "userDataSourceProperties")
    @ConfigurationProperties(prefix = "spring.datasource.user") // 读取 spring.datasource.user 配置到 DataSourceProperties 对象
    public DataSourceProperties userDataSourceProperties() {
        return new DataSourceProperties();
    }

    /**
     * 创建 user 数据源
     */
    @Primary
    @Bean(name = "userDataSource")
    @ConfigurationProperties(prefix = "spring.datasource.user.hikari") // 读取 spring.datasource.user 配置到 HikariDataSource 对象
    public DataSource userDataSource() {
        // 获得 DataSourceProperties 对象
        DataSourceProperties properties =  this.userDataSourceProperties();
        // 创建 HikariDataSource 对象
        return createHikariDataSource(properties);
    }

    /**
     * 创建 quartz 数据源的配置对象
     */
    @Bean(name = "quartzDataSourceProperties")
    @ConfigurationProperties(prefix = "spring.datasource.quartz") // 读取 spring.datasource.quartz 配置到 DataSourceProperties 对象
    public DataSourceProperties quartzDataSourceProperties() {
        return new DataSourceProperties();
    }

    /**
     * 创建 quartz 数据源
     */
    @Bean(name = "quartzDataSource")
    @ConfigurationProperties(prefix = "spring.datasource.quartz.hikari")
    @QuartzDataSource
    public DataSource quartzDataSource() {
        // 获得 DataSourceProperties 对象
        DataSourceProperties properties =  this.quartzDataSourceProperties();
        // 创建 HikariDataSource 对象
        return createHikariDataSource(properties);
    }

    private static HikariDataSource createHikariDataSource(DataSourceProperties properties) {
        // 创建 HikariDataSource 对象
        HikariDataSource dataSource = properties.initializeDataSourceBuilder().type(HikariDataSource.class).build();
        // 设置线程池名
        if (StringUtils.hasText(properties.getName())) {
            dataSource.setPoolName(properties.getName());
        }
        return dataSource;
    }

}

In order to test quickly and easily, we put Quartz's built table together with the business library, and then configure it as follows:

spring:
  datasource:
    url: jdbc:mysql://10.10.0.10:3306/ptc_job?useSSL=false&useUnicode=true&characterEncoding=UTF-8
    driver-class-name: com.mysql.jdbc.Driver
    username: root
    password: root
  # Quartz 的配置，对应 QuartzProperties 配置类
  quartz:
    scheduler-name: clusteredScheduler # Scheduler 名字。默认为 schedulerName
    job-store-type: jdbc # Job 存储器类型。默认为 memory 表示内存，可选 jdbc 使用数据库。
    auto-startup: true # Quartz 是否自动启动
    startup-delay: 0 # 延迟 N 秒启动
    wait-for-jobs-to-complete-on-shutdown: true # 应用关闭时，是否等待定时任务执行完成。默认为 false ，建议设置为 true
    overwrite-existing-jobs: true # 是否覆盖已有 Job 的配置，注意为false时，修改已存在的任务调度cron，周期不生效
    jdbc: # 使用 JDBC 的 JobStore 的时候，JDBC 的配置
      initialize-schema: never # 是否自动使用 SQL 初始化 Quartz 表结构。这里设置成 never ，我们手动创建表结构。
    properties: # 添加 Quartz Scheduler 附加属性，更多可以看 http://www.quartz-scheduler.org/documentation/2.4.0-SNAPSHOT/configuration.html 文档
      org:
        quartz:
          # JobStore 相关配置
          jobStore:
            # 数据源名称
            dataSource: quartzDataSource # 使用的数据源
            class: org.quartz.impl.jdbcjobstore.JobStoreTX # JobStore 实现类
            driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegate
            tablePrefix: QRTZ_ # Quartz 表前缀
            isClustered: true # 是集群模式
            clusterCheckinInterval: 1000
            useProperties: false
          # 线程池相关配置
          threadPool:
            threadCount: 25 # 线程池大小。默认为 10 。
            threadPriority: 5 # 线程优先级
            class: org.quartz.simpl.SimpleThreadPool # 线程池类型

create taskJob1

@DisallowConcurrentExecution
public class Job1 extends QuartzJobBean {

    private Logger logger = LoggerFactory.getLogger(getClass());
    private static SimpleDateFormat fullDateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
    private final AtomicInteger count = new AtomicInteger();

    @Autowired
    private DemoService demoService;

    private String k1;

    public void setK1(String k1) {
        this.k1 = k1;
    }
    
    @Override
    protected void executeInternal(JobExecutionContext context) {
        logger.info("[job1的执行了，时间: {}, k1={}, count={}, demoService={}]", fullDateFormat.format(new Date()), k1,
                count.incrementAndGet(), demoService);
    }

}

Inherit the QuartzJobBean abstract class, implement #executeInternal(JobExecutionContext context)the method, and execute the logic of the custom scheduled task.

QuartzJobBean implements org.quartz.Jobthe interface, which provides dependency property injection of the JobDataMap data into the Job Bean every time Quartz creates a Job to execute timing logic.

// QuartzJobBean.java

public final void execute(JobExecutionContext context) throws JobExecutionException {
    try {
        // 将当前对象，包装成 BeanWrapper 对象
        BeanWrapper bw = PropertyAccessorFactory.forBeanPropertyAccess(this);
        // 设置属性到 bw 中
        MutablePropertyValues pvs = new MutablePropertyValues();
        pvs.addPropertyValues(context.getScheduler().getContext());
        pvs.addPropertyValues(context.getMergedJobDataMap());
        bw.setPropertyValues(pvs, true);
 } catch (SchedulerException ex) {
  throw new JobExecutionException(ex);
 }

    // 执行提供给子类实现的抽象方法
    this.executeInternal(context);
}

protected abstract void executeInternal(JobExecutionContext context) throws JobExecutionException;

The injection job task configuration is as follows:

        @Bean
        public JobDetail job1() {
            return JobBuilder.newJob(Job1.class)
                    .withIdentity("job1")
                    .storeDurably() 
                    .usingJobData("k1", "v1")
                    .build();
        }

        @Bean
        public Trigger simpleJobTrigger() {
            // 简单的调度计划的构造器
            SimpleScheduleBuilder scheduleBuilder = SimpleScheduleBuilder.simpleSchedule()
                    .withIntervalInSeconds(30) // 频率  30s执行一次。
                    .repeatForever(); // 次数。
            // Trigger 构造器
            return TriggerBuilder.newTrigger()
                    .forJob(job1()) 
                    .withIdentity("job1Trigger") 
                    .withSchedule(scheduleBuilder) 
                    .build();
        }

At this time, start the project to view the log as follows:

2022-09-20 23:17:33.500  INFO 18982 --- [eduler_Worker-2]  : [job1的执行了，时间: 2022-09-20 23:17:33, k1=v1, count=1, demoService=DemoService@3258ebff]
2022-09-20 23:18:03.463  INFO 18982 --- [eduler_Worker-3]  : [job1的执行了，时间: 2022-09-20 23:18:03, k1=v1, count=1, demoService=DemoService@3258ebff]
2022-09-20 23:18:33.439  INFO 18982 --- [eduler_Worker-4]  : [job1的执行了，时间: 2022-09-20 23:18:33, k1=v1, count=1, demoService=DemoService@3258ebff]
2022-09-20 23:19:03.448  INFO 18982 --- [eduler_Worker-5]  : [job1的执行了，时间: 2022-09-20 23:19:03, k1=v1, count=1, demoService=DemoService@3258ebff]

countIt can be seen from the counter that each time Job0 will create a new Job object by Quartz to execute the task , but DemoServicethe attribute values are the same, it is a Spring singleton bean, and the data of JobData is automatically mapped and injected into the task bean attribute.

The above is to execute tasks at a specified frequency through a simple scheduler simpleSchedule. Of course, I can also use mainstream cron expressions to implement task cycle execution:

       @Bean
        public JobDetail job1() {
            return JobBuilder.newJob(Job1.class)
                    .withIdentity("job1")
                    .storeDurably() 
                    .usingJobData("k1", "v1")
                    .build();
        }

        @Bean
        public Trigger cronJobTrigger() {
            // 每隔1分钟执行一次
            CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule("0 0/1 * * * ? *");
            // Trigger 构造器
            return TriggerBuilder.newTrigger()
                    .forJob(job1()) 
                    .withIdentity("job1Trigger") 
                    .withSchedule(scheduleBuilder) 
                    .build();
        }

The task scheduling execution result will not be displayed here, it is the same as above.

3. Implementation principle

Quartz operates tasks through the Scheduler scheduler. It can add task JobDetail and trigger Trigger to the task pool, delete tasks, or stop tasks. The scheduler puts these tasks and triggers into a JobStore. Here jobStore has memory form and persistent form. Of course, it can also be customized and expanded into an independent service.

Quartz internally uses a scheduling thread QuartzSchedulerThread to continuously find tasks that need to be executed next time in the JobStore, and encapsulates these tasks into a thread pool ThreadPool to run. The component structure is as follows:

core class

QuartzSchedulerThread: The thread responsible for executing the work that triggers the Trigger registered with QuartzScheduler. ThreadPool：SchedulerUsing a thread pool as the infrastructure for task execution, tasks provide operational efficiency by sharing threads in the thread pool. QuartzSchedulerResources: Contains all the resources (JobStore, ThreadPool, etc.) needed to create a QuartzScheduler instance. SchedulerFactory: Provides a mechanism for obtaining a client-side usable handle to a scheduler instance. JobStore: Interfaces implemented by classes that provide an org.quartz.Job and org.quartz.Trigger storage mechanism for the use of org.quartz.core.QuartzScheduler. The storage of jobs and triggers should be unique by the combination of their name and group. QuartzScheduler: This is the core of Quartz, which is an indirect implementation of the org.quartz.Scheduler interface, including methods for scheduling org.quartz.Jobs, registering org.quartz.JobListener instances, etc. Scheduler: This is the main interface of Quartz Scheduler, which represents an independent running container. The scheduler maintains a registry of JobDetails and Triggers. Once registered, the scheduler is responsible for executing jobs when their associated triggers fire (when their scheduled time arrives). Trigger: A basic interface with common properties for all triggers, describing the time trigger rules for job execution. - Use TriggerBuilder to instantiate the actual trigger. JobDetail: Passes the details attribute for the given job instance. JobDetails will be created/defined using JobBuilder. Job: Interface to be implemented by classes representing "jobs" to be performed. There is only one method void execute(jobExecutionContext context) (jobExecutionContext provides various information of the scheduling context, and the runtime data is saved in the jobDataMap)

Job has a sub-interface StatefulJob, which represents a stateful task. Stateful tasks cannot be concurrent. The previous task has not been executed, and the subsequent task is blocked and waits. The following shows the native Quartz creating tasks, binding triggers, registering tasks and timers, and starting the scheduler.

/**
     * 原生创建任务流程示例，有助于分析quartz实现原理
     * @throws SchedulerException
     */
    public static void test() throws SchedulerException {
        //1.创建Scheduler的工厂
        SchedulerFactory sf = new StdSchedulerFactory();
        //2.从工厂中获取调度器实例
        Scheduler scheduler = sf.getScheduler();
        //3.创建JobDetail
        JobDetail jb = JobBuilder.newJob(Job1.class)
                .withDescription("this is a job") //job的描述
                .withIdentity("job1", "test-job") //job 的name和group
                .build();

        //任务运行的时间，SimpleSchedule类型触发器有效
        long time=  System.currentTimeMillis() + 3*1000L; //3秒后启动任务
        Date statTime = new Date(time);

        //4.创建Trigger
        //使用SimpleScheduleBuilder或者CronScheduleBuilder
        Trigger t = TriggerBuilder.newTrigger()
                .withDescription("")
                .withIdentity("job1Trigger", "job1TriggerGroup")
                //.withSchedule(SimpleScheduleBuilder.simpleSchedule())
                .startAt(statTime)  //默认当前时间启动
                .withSchedule(CronScheduleBuilder.cronSchedule("0/10 * * * * ?")) //10秒执行一次
                .build();

        //5.注册任务和定时器
        scheduler.scheduleJob(jb, t);//源码分析

        //6.启动 调度器
        scheduler.start();
    }

    public static void main(String[] args) throws SchedulerException {
        test();
    }

Next, analyze the main three steps: creating a scheduler, registering tasks and triggers, and starting the scheduler to execute tasks

Scheduler initialization

 SchedulerFactory sf = new StdSchedulerFactory();
 Scheduler scheduler = sf.getScheduler();

SchedulerFacotoryIt is a factory interface for creating a scheduler. It has two implementations. StdSchedulerFacotoryCreate a Scheduler according to the configuration file, DirectSchedulerFactorymainly through coding to control the Scheduler. Usually, in order to be less intrusive and more convenient to implement, we use StdSchedulerFacotorythe type to create the StdScheduler, in quartz.properties The configurations are all corresponding to this StdSchedulerFactory, so if you don't understand the default value of a certain configuration, you can look at StdSchedulerFactorythe code to obtain the configuration.

From sf.getScheduler()the start, StdSchedulerFacotoryyou can see the logic of the method by entering:

public Scheduler getScheduler() throws SchedulerException {
        // 第一步：加载配置文件，System的properties覆盖前面的配置
        if (cfg == null) {
            initialize();
        }
        SchedulerRepository schedRep = SchedulerRepository.getInstance();
        Scheduler sched = schedRep.lookup(getSchedulerName());
        if (sched != null) {
            if (sched.isShutdown()) {
                schedRep.remove(getSchedulerName());
            } else {
                return sched;
            }
        }
        // 第二步：初始化，生成scheduler
        sched = instantiate();
        return sched;
    }

A total of two logics are completed here: load configuration and generate scheduler, and then enter the core method. instantiate()There are many logics in it. The core operation is to initialize the objects required for various scheduling, such as thread pool, JobStore, etc., and finally create the above Put the object into QuartzSchedulerResources and put the thread pool up, which is equivalent to the resource storage place of QuartzScheduler. The relevant code of the method is as follows:

private Scheduler instantiate() throws SchedulerException{
        ......
        // 要初始化的对象
        JobStore js = null;
        ThreadPool tp = null;
        QuartzScheduler qs = null;
        DBConnectionManager dbMgr = null;
        String instanceIdGeneratorClass = null;
        Properties tProps = null;
        String userTXLocation = null;
        boolean wrapJobInTx = false;
        boolean autoId = false;
        long idleWaitTime = -1;
        long dbFailureRetry = 15000L; // 15 secs
        String classLoadHelperClass;
        String jobFactoryClass;
        ThreadExecutor threadExecutor;
  
        .....
          
        QuartzSchedulerResources rsrcs = new QuartzSchedulerResources();
        rsrcs.setName(schedName);
        rsrcs.setThreadName(threadName);
        rsrcs.setInstanceId(schedInstId);
        rsrcs.setJobRunShellFactory(jrsf);
        rsrcs.setMakeSchedulerThreadDaemon(makeSchedulerThreadDaemon);
        rsrcs.setThreadsInheritInitializersClassLoadContext(threadsInheritInitalizersClassLoader);
        rsrcs.setRunUpdateCheck(!skipUpdateCheck);
        rsrcs.setBatchTimeWindow(batchTimeWindow);
        rsrcs.setMaxBatchSize(maxBatchSize);
        rsrcs.setInterruptJobsOnShutdown(interruptJobsOnShutdown);
        rsrcs.setInterruptJobsOnShutdownWithWait(interruptJobsOnShutdownWithWait);
        rsrcs.setJMXExport(jmxExport);
        rsrcs.setJMXObjectName(jmxObjectName);

        //这个线程执行者用于后面启动调度线程
        rsrcs.setThreadExecutor(threadExecutor);
        threadExecutor.initialize();

        rsrcs.setThreadPool(tp);
        if (tp instanceof SimpleThreadPool) {
          if (threadsInheritInitalizersClassLoader)
            ((SimpleThreadPool) tp).setThreadsInheritContextClassLoaderOfInitializingThread(
            threadsInheritInitalizersClassLoader);
        }
        //执行线程池启动
        tp.initialize();
        tpInited = true;

        rsrcs.setJobStore(js);

        // add plugins
        for (int i = 0; i < plugins.length; i++) {
          rsrcs.addSchedulerPlugin(plugins[i]);
        }

        //调度线程在构造方法里面启动的
        qs = new QuartzScheduler(rsrcs, idleWaitTime, dbFailureRetry);
    }

The scheduler is initialized after the above scheduler, and then you can define the Job and Trigger, and then scheduler.scheduleJob(jb, t)register the tasks and triggers.

Register tasks and triggers

scheduler.scheduleJob(jb, t)

EnterStdScheduler#scheduleJob(JobDetail jobDetail, Trigger trigger)

   public Date scheduleJob(JobDetail jobDetail, Trigger trigger)
        throws SchedulerException {
        return sched.scheduleJob(jobDetail, trigger);
    }

The object here schedis QuartzSchedulerto enter sched.scheduleJob(jobDetail, trigger), here is the core logic of registration tasks and scheduled tasks.

 public Date scheduleJob(JobDetail jobDetail, Trigger trigger) throws SchedulerException {
        .....
		   //核心代码：存储给定的org.quartz.JobDetail和org.quartz.Trigger。
        resources.getJobStore().storeJobAndTrigger(jobDetail, trig);
        notifySchedulerListenersJobAdded(jobDetail);
        notifySchedulerThread(trigger.getNextFireTime().getTime());
        notifySchedulerListenersSchduled(trigger);

        return ft;
    }

Here resourcesis to initialize various objects when creating the scheduler above and then put them in the resource management office QuartzSchedulerResources, which contains the JobStore object, and then save tasks and triggers through this object. As for the details of the storage logic, I will not elaborate here, please do it yourself Check, anyway, the context of the core logic is right here.

Start the scheduler to execute tasks

Quartz uses a thread to continuously poll to find the task to be executed next time, and hands the task to the thread pool for execution. There are two roles involved here: scheduling thread and execution thread pool.

scheduler.start();

scheduler.start()Call QuartzScheduler.start(), the start of Quartz needs to call the start() method to start the thread. The start of the thread in the thread is to call the start() method, but the operation of actually executing the thread task is in run()

QuartzScheduler.start()code show as below:

public void start() throws SchedulerException {
    if (shuttingDown|| closed) {
        throw new SchedulerException(
            "The Scheduler cannot be restarted after shutdown() has been called.");
    }
    notifySchedulerListenersStarting();
    if (initialStart == null) {//初始化标识为null，进行初始化操作
        initialStart = new Date();
        this.resources.getJobStore().schedulerStarted();//1 主要分析的地方      
        startPlugins();
    } else {

        resources.getJobStore().schedulerResumed();//2 如果已经初始化过，则恢复jobStore
    }

    schedThread.togglePause(false);//3 唤醒所有等待的线程

    getLog().info(
        "Scheduler " + resources.getUniqueIdentifier() + " started.");

    notifySchedulerListenersStarted();
}

this.resources.getJobStore().schedulerStarted(); The main analysis is actually the start QuartzSchedulerResourcesof the call.JobStore

Finally, QuartzSchedulerThread.run() is mainly to obtain the Trigger that needs to be executed when there are available threads and trigger the scheduling of tasks!

Look at the run () method of the thread QuartzSchedulerThread in a while (true) manner, and continuously obtain the trigger set to be triggered next time from the jobStore, and put the task in the thread pool for execution. This is also the way Quartz implements the periodic execution of tasks The core, please see the specific analysis: https://my.oschina.net/chengxiaoyuan/blog/674603