Still writing task scheduling code? Try this visual distributed scheduling framework!

In a microservice system, there are often scenarios for task scheduling. For example, regularly synchronize data every day, generate business reports regularly, and clean up logs regularly. Today I recommend a distributed scheduling framework to everyone to help you complete task scheduling easily!

Introduction to PowerJob

PowerJob is a new generation of distributed scheduling and computing framework that allows you to easily complete task scheduling and complex task distributed computing.

Main features:

Easy to use: Provide a front-end web interface, allowing developers to visually complete the management of scheduled tasks and view task running status and logs.
Perfect timing strategy: supports four timing scheduling strategies: CRON expression, fixed frequency, fixed delay and API.
Rich execution modes: supports four execution modes of stand-alone, broadcast, Map, and MapReduce.
Reliance reduction: The minimum relies on only relational databases (MySQL, etc.), and the extended reliance is MongoDB (used to store huge online logs).

Why is there a dispatch center

We will use the general QuartZor Spring Taskthose implemented in the framework of applications scheduled task to task scheduling, but in the micro-service architecture, if many applications are flooded with the task scheduler code seemed a little awkward. A reasonable plan should be like this. The task execution method exists in the application, and we have a dispatch center, which is responsible for scheduling these methods. We only need to configure the tasks in the dispatch center. PowerJob is just such a distributed Scheduling framework.

Installation preparation

Since PowerJob's scheduling center (powerjob-server) needs to use MySQL to store data and MongoDB to store logs, we first install and start these two services.

Start the MySQL service in the Docker container;

docker run -p 3306:3306 --name mysql \
-v /mydata/mysql/log:/var/log/mysql \
-v /mydata/mysql/data:/var/lib/mysql \
-v /mydata/mysql/conf:/etc/mysql \
-e MYSQL_ROOT_PASSWORD=root  \
-d mysql:5.7

Create the database required by PowerJob in MySQL powerjob-product;

CREATE DATABASE IF NOT EXISTS `powerjob-product` DEFAULT CHARSET utf8mb4

Start the MongoDB service in the Docker container.

docker run -p 27017:27017 --name mongo \
-v /mydata/mongo/db:/data/db \
-d mongo:4.2.5

Install dispatch center

Installing PowerJob's dispatch center in Docker environment is very simple, it takes a minute to get it!

Download the mirrored powerjob-serverDocker image:

docker pull tjqq/powerjob-server:latest

Run the powerjob-serverservice in a Docker container :

docker run -p 7700:7700 -p 10086:10086 --name powerjob-server \
--link mysql:db \
--link mongo:mongo \
-e TZ="Asia/Shanghai" \
-e JVMOPTIONS="" \
-e PARAMS="--spring.profiles.active=product --spring.datasource.core.jdbc-url=jdbc:mysql://db:3306/powerjob-product?useUnicode=true&characterEncoding=UTF-8 --spring.datasource.core.username=root --spring.datasource.core.password=root --spring.data.mongodb.uri=mongodb://mongo:27017/powerjob-product" \
-v ~/docker/powerjob-server:/mydata/powerjob/powerjob-server \
-v ~/.m2:/mydata/powerjob/.m2 \
-d tjqq/powerjob-server:latest

After running successfully, you can access powerjob-serverthe web interface. Note that the Linux firewall needs to open 7700and 10086two ports. The access address: http://192.168.3.101:7700/

Initialize the actuator in the application

After installing the dispatch center, we need to initialize the PowerJob executor (powerjob-worker) in the SpringBoot application.

First pom.xmladd powerjob-workerrelated dependencies in:

<dependency>
    <groupId>com.github.kfcfans</groupId>
    <artifactId>powerjob-worker-spring-boot-starter</artifactId>
    <version>3.2.3</version>
</dependency>

After application.ymladding the powerjob-workerrelevant configuration in the configuration file , pay attention to powerjob.worker.app-namethis configuration;

powerjob:
  worker:
    akka-port: 27777 # akka 工作端口
    app-name: mall-tiny-powerjob # 接入应用名称，用于分组隔离
    server-address: 192.168.3.101:7700 # 调度服务器地址
    store-strategy: disk # 持久化方式

To add a stand-alone processor StandaloneProcessor, just inherit the BasicProcessorinterface and implement the processmethod;

package com.macro.mall.tiny.job;

@Slf4j
@Component
public class StandaloneProcessor implements BasicProcessor {

    @Override
    public ProcessResult process(TaskContext context){
        //OmsLogger可以直接将日志上报到powerjob-server
        OmsLogger omsLogger = context.getOmsLogger();
        omsLogger.info("StandaloneProcessor start process,context is {}.", context);
        log.info("jobParams is {}", context.getJobParams());
        return new ProcessResult(true, "Process success!");
    }
}

After packaging and uploading the image, run the SpringBoot application service in the Docker container. Note that the configured time zone must be consistent with the scheduling center.

docker run -p 8080:8080 --name mall-tiny-powerjob \
--link mysql:db \
-v /etc/localtime:/etc/localtime \
-v /mydata/app/mall-tiny-powerjob/logs:/var/logs \
-e TZ="Asia/Shanghai" \
-d mall-tiny/mall-tiny-powerjob:1.0-SNAPSHOT

Configuration and execution of tasks

With the actuator and dispatch center, we only need to configure the task in the dispatch center to achieve task scheduling.

First, we need to register the application (integrated actuator) in the dispatch center, the application name is application.ymlthe powerjob.worker.app-nameattribute in, and here is mall-tiny-powerjob:123456;