Implementation Ideas of Task Scheduling Module in Large-Scale System

Product requirements: A task scheduling module is required. Users can create new tasks through the page. The main task is to send emails on a regular basis, or send emails on a weekly basis. Users can customize the sending rules, and users can suspend tasks, delete tasks, and Can know the implementation of the task.

one. The @Scheduled annotation of the spring framework

Traditional timed tasks can be implemented with the @Scheduled annotation of the spring framework

 

As shown above, we use the @Scheduled annotation with cron expressions in the project to achieve the effect of synchronizing data every minute. This also realizes the function of timing tasks.

Thinking: What are the disadvantages of implementing timing tasks in this way?

Disadvantage 1: If the single-point architecture needs to be load balanced, the task may be started multiple times, which may lead to data inconsistency and concurrency problems in severe cases. In a distributed architecture, if only one task module is deployed, although it can ensure that the task runs only once at a time, it cannot guarantee high availability.

Solution: introduce distributed locks

 

As shown in the figure above, a script for synchronizing all users of Zhejiang Zhengding is written in the user module, and it is started by a scheduled task. The redis distributed lock is introduced in the task module to ensure that only one scheduled task will be executed at the same time

Disadvantage 2: Once the task is started, supervision and control cannot be achieved. For example: If the task does not want to continue to execute, or the execution interval needs to be modified, it can only be achieved by modifying the code and re-publishing.

two. Jdk's built-in Timer and TimerTask implement timing tasks

Let's see the implementation:

 

First, the Timer scheduler is defined, and then TimerTask is defined to define tasks. In addition to the regular execution of scheduled tasks, the timer scheduler also provides a cancel method to cancel the execution, so to a certain extent, it can control the running tasks.

   

    What are the disadvantages of this approach?

Analysis source code:

The core of the Sched method is to add tasks to a queue, and this method adds a lock. Then, by analyzing this source code, it can be concluded that the timer is thread-safe and is a single-threaded operation.

 

Next, look at the core code: how to trigger a scheduled task

 

Through infinite polling (no thread sleep) to determine whether the task needs to be executed in real time. And only catch in exception handling

InterruptedException, which means that if a runtime exception is thrown in your task, the thread will be killed.

The disadvantages of timer can be easily obtained through source code analysis:

  1. general performance
  2. The timer has only one execution thread. If an exception is encountered, all tasks will be canceled (ScheduledThreadPoolExecutor implements multi-threading to solve this problem. If you are interested, you can learn more about it, and I won’t explain it in detail)
  3. All tasks are placed in the memory queue. Once the service restarts or the task fails, all data will be lost without persistence.

three. Quartz framework

The Quartz framework is an open source job scheduling framework

It implements task scheduling based on scheduler, job, jobdetail

 

 

The Scheduler interface provides API interfaces for adding jobs, deleting jobs, suspending jobs and resuming jobs, allowing developers to freely combine businesses to achieve various needs.

 

In addition, quartz also combines multi-thread optimization, persistence strategy (can be combined with jdbc), and cluster strategy.

Four. Xxl-job framework

   

 

It is an open source project based on springboot, which provides a visual interface and is easier to get started.

However, I personally feel that it is over-encapsulated. For development, only the underlying capabilities need to be provided. It is possible to add cron expressions through the visual interface to trigger scheduled tasks. If it is a purely technical scheduled task, this framework can be used. If it is customized development at the business level, the cost of modification will be relatively high.

Guess you like

Origin blog.csdn.net/babing18258840900/article/details/131961582