Spring Boot integrates timing tasks, which can be dynamically edited

Timing tasks, Song Ge has written many articles to introduce to you before. Last time, I DIYed a timing task that can be dynamically edited, and also recorded a supporting video:

Relevant information links click here:

1. Spring timed tasks play out!
2. Teach you how to customize editable timed tasks!
3. Develop configurable timing tasks ~ the second bullet
4. Atypical usage of Vue, a simple admin page

However, the one we wrote at the time did not support distributed environments, and it was not difficult to support it. We used a zookeeper or redis as a public information center, which recorded various running conditions of scheduled tasks. With this, we can support distributed environment.

Today, instead of writing it ourselves, let's take a look at a ready-made framework: ElasticJob. There is an xxljob with the same name as him, which we will introduce later.

1. ElasticJob

1.1 Introduction

ElasticJob is a distributed job scheduling solution, its official website is:

• http://shardingsphere.apache.org/elasticjob

The predecessor of Elastic Job was dd-job, a distributed task scheduling framework open sourced by Dangdang, but it joined the Apache Foundation on May 28, 2020 and became an open source project under Apache:

• https://shardingsphere.apache.org/

ElasticJob creates a distributed scheduling solution suitable for Internet scenarios through the functions of elastic scheduling, resource management and control, and job governance, and provides a diversified job ecosystem through an open architecture design.

Using ElasticJob can make development engineers no longer worry about non-functional requirements such as linear throughput improvement of tasks, so that they can focus more on business-oriented coding design; at the same time, it can also liberate operation and maintenance engineers, so that they no longer have to worry about task availability and For related management requirements, the purpose of automatic operation and maintenance can be achieved by simply adding service nodes.

ElasticJob is a distributed scheduling solution for the Internet ecosystem and massive tasks. It consists of two independent sub-projects ElasticJob-Liteand ElasticJob-Cloud.

Among them, it is ElasticJob-Litepositioned as a lightweight and decentralized solution, and provides coordination services for distributed tasks in the form of jars:

ElasticJob-CloudThe self-developed Mesos Framework solution is used to provide additional functions such as resource management, application distribution, and process isolation:

ElasticJob-Lite VS ElasticJob-Cloud：

	ElasticJob-Lite	ElasticJob-Cloud
Decentralization	Yes	no
Resource allocation	not support	support
work mode	permanent	Permanent + Momentary
Deployment dependencies	ZooKeeper	ZooKeeper + Mesos

Each of its products uses a unified job API, and developers only need to develop once and deploy at will (that is, ElasticJob-Liteand ElasticJob-Clouduse the same API, mainly in different deployment methods).

1.2 Function list

• Flexible scheduling

  • Support task sharding and high availability in distributed scenarios
  • Ability to horizontally scale the throughput and execution efficiency of tasks
  • The task processing capability can be elastically scaled with resource allocation

• Resource allocation

  • Allocate the right resources to tasks and make them effective at the right time
  • The same tasks are aggregated to the same executor for unified processing
  • Dynamically provision additional resources to newly assigned tasks

• job management

  • failover
  • Missed job re-execution
  • self-diagnostic repair

• Job Dependency (TODO)

  • Inter-job dependencies based on directed acyclic graph (DAG)
  • Dependency between job shards based on directed acyclic graph (DAG)

• Open work ecology

  • Extensible Job Type Unified Interface
  • Rich job type library, such as data flow, script, HTTP, file, big data, etc.
  • It is easy to connect business operations and can be seamlessly integrated with Spring Dependency Injection

• Visual operation and maintenance platform (https://github.com/apache/shardingsphere-elasticjob-ui)

  • Operation control terminal
  • Job execution history data tracking
  • Registry management

2. Practice

Having said that, let's experience an ElasticJob through a simple case. Having code after all, it feels more real.

First we create a Spring Boot project and introduce web dependencies:

Then manually join the ElasticJob starter:

<dependency>
    <groupId>org.apache.shardingsphere.elasticjob</groupId>
    <artifactId>elasticjob-lite-spring-boot-starter</artifactId>
    <version>3.0.1</version>
</dependency>

Next, we create a job. There are several different ways to create a job. Let's first look at a job created based on implementing the SimpleJob interface:

/**
 * @author 江南一点雨
 * @微信公众号 江南一点雨
 * @网站 http://www.itboyhub.com
 * @国际站 http://www.javaboy.org
 * @微信 a_java_boy
 * @GitHub https://github.com/lenve
 * @Gitee https://gitee.com/lenve
 */
@Component
public class MyFirstJob implements SimpleJob {
    
    
private static final Logger logger = LoggerFactory.getLogger(MyFirstJob.class);
    @Override
    public void execute(ShardingContext shardingContext) {
    
    
        logger.info("作业名称：{}；作业参数：{}；分片总数：{}；当前分片：{}；分片参数：{}；任务编号：{}",shardingContext.getJobName(),shardingContext.getJobParameter(),shardingContext.getShardingTotalCount(),shardingContext.getShardingItem(),shardingContext.getShardingParameter(),shardingContext.getTaskId());
    }
}

When the scheduled task is executed, the execute method will be triggered. The parameters related to the scheduled task are stored in the parameter ShardingContext. These parameters are all configured in application.properties. Let's continue to see:

elasticjob.reg-center.server-lists=zoo1:2181,zoo2:2182,zoo3:2183
elasticjob.reg-center.namespace=javaboy

elasticjob.jobs.my-first-job.elastic-job-class=org.javaboy.elasticjob.job.MyFirstJob
elasticjob.jobs.my-first-job.cron=0/3 * * * * ?
elasticjob.jobs.my-first-job.sharding-total-count=1
elasticjob.jobs.my-first-job.overwrite=true
elasticjob.jobs.my-first-job.job-parameter=hello javaboy!
elasticjob.jobs.my-first-job.sharding-item-parameters=0=A,1=B,2=C

The configuration here is divided into two categories:

• Registry Configuration
• Scheduled task configuration

Using ElasticJob requires the registration center zookeeper, which is easy to understand, because ElasticJob supports task fragmentation and high availability in distributed scenarios, so a scheduling center is necessary, and this zk is the scheduling center. I have opened a zk cluster here, there are three instances in it, and the three zk addresses are ,separated by . At the same time, we also need to configure a namespace. The role of this namespace is to prevent conflicts between scheduled tasks of different applications. We give each application a namespace that is different from other applications, so that there is no need to worry about conflicts.

Next is the configuration job.

The prefix of the configuration job is unified elasticjob.jobs, followed by the name of the job. The job name can be configured at will, but it is best to see which job it is at a glance . The value obtainedMyFirstJob#execute in the method is this value.shardingContext.getJobName()

We have configured a total of six properties here, let me explain them one by one:

• elastic-job-class: The full path to the job.
• cron: cron expression.
• sharding-total-count: The total number of shards, that is, how many instances perform the current scheduled task, and the value obtainedMyFirstJob#execute in the method is this value.shardingContext.getShardingTotalCount()
• overwrite: Whether to overwrite the previous configuration each time it is started, if it is set to false, if the cron expression is modified, it will not take effect after restarting.
• job-parameter: The parameter of the job, the value obtainedMyFirstJob#execute in the method is this value.shardingContext.getJobParameter()
• sharding-item-parameters: sharding parameters, 0, 1, and 2 respectively represent the number of shards, and the value obtainedMyFirstJob#execute in the method is this value.shardingContext.getShardingParameter()

OK, now the configuration is complete.

3. Run

Now we start the Spring Boot project directly. After startup, the console will print the following log:

No problem, the log is printed every three seconds.

Now we start an instance of the current project again, check Allow parallel runto start multiple instances (remember to modify the port number when starting a new instance):

When the new instance is started, we find that there is no log printed in the instance started for the first time, and instead print the log in the instance started the second time, this is because we have sharding-total-countconfigured 1, that is, there is only one instance at the same time The scheduled task in is running.

3. Operation and maintenance platform

ElasticJob provides an operation and maintenance platform through which scheduled tasks can be dynamically managed. The operation and maintenance platform address:

• https://github.com/apache/shardingsphere-elasticjob-ui

Steps to use the operation and maintenance platform:

1. Clone the project down: git clone https://github.com/apache/shardingsphere-elasticjob-ui.git.
2. Enter into the directory: cd shardingsphere-elasticjob-ui.
3. Packaging: mvn clean package -Prelease.
4. After the package is complete, unzip the shardingsphere-elasticjob-ui/shardingsphere-elasticjob-ui-distribution/shardingsphere-elasticjob-lite-ui-bin-distribution/target/apache-shardingsphere-elasticjob-3.1.0-SNAPSHOT-lite-ui-bin.tar.gzfile , and then execute the startup.sh script in its bin directory to start.

The third step of packaging above is easy to make mistakes due to network reasons, so if the small partners fail to package, they can reply in the background of the public account Jiangnan shardingsphere-elasticjob-uiYiyuyu to obtain the packaged files of Songge.

After the operation and maintenance platform is started, the browser input http://localhost:8088will jump to the login page, as follows:

The default username and password are both root.

After the registration is successful, click the registration center configuration, and then select the Add button to add the registration center. After the registration center is added, the operation and maintenance platform will automatically read the timing task information from the registration center:

Just fill it in truthfully. Be careful not to write the namespace wrong. If it is written otherwise, it will not be able to read the scheduled task.

Next, click the connect button to establish a connection with zk:

Click the job dimension to view the detailed information of the job, including job name, total number of shards, cron expressions, etc.:

There are four action buttons at the back:

• Modify: Modify the details of the job, such as modifying the job's cron expression.
• Details: View the details of the job.
• Trigger: triggers the execution of the job.
• Invalidation: It is equivalent to suspending the execution of the job. After clicking the invalidation button, the validating button will appear. Clicking the validating button will make the job valid and continue to execute.
• Terminate: Stop the job.

Click the server dimension to view server information:

4. Summary

Well, today I will show you how to play ElasticJob with my friends through a simple case. Regarding other ways to play ElasticJob, we will continue to talk later~

The download address of the case in this article: https://github.com/lenve/javaboy-code-samples