Still writing task scheduling code? Try this visual distributed scheduling framework

In a microservice system, there are often scenarios for task scheduling. For example, regularly synchronize data every day, generate business reports regularly, and clean up logs regularly. Today I recommend a distributed scheduling framework to everyone to help you complete task scheduling easily!

Introduction to PowerJob

PowerJob is a new generation of distributed scheduling and computing framework that allows you to easily complete task scheduling and complex task distributed computing.

Main features:

• Easy to use: Provide a front-end web interface, allowing developers to visually complete the management of scheduled tasks and view task running status and logs.
• Perfect timing strategy: supports four timing scheduling strategies: CRON expression, fixed frequency, fixed delay and API.
• Rich execution modes: supports four execution modes of stand-alone, broadcast, Map, and MapReduce.
• Reliance reduction: The minimum relies on only relational databases (MySQL, etc.), and the extended reliance is MongoDB (used to store huge online logs).

Why is there a dispatch center

Under normal circumstances, we will use QuartZ or Spring Task frameworks to implement timing tasks in applications for task scheduling, but under the microservice architecture, if many applications are flooded with this task scheduling code, it seems a bit inappropriate. A reasonable plan should be like this. The task execution method exists in the application, and we have a dispatch center, which is responsible for scheduling these methods. We only need to configure the tasks in the dispatch center. PowerJob is just such a distributed Scheduling framework.

Installation preparation

Since PowerJob's scheduling center (powerjob-server) needs to use MySQL to store data and MongoDB to store logs, we first install and start these two services.

• Start the MySQL service in the Docker container;

docker run -p 3306:3306 --name mysql \
-v /mydata/mysql/log:/var/log/mysql \-v /mydata/mysql/data:/var/lib/mysql \-v /mydata/mysql/conf:/etc/mysql \-e MYSQL_ROOT_PASSWORD=root  \-d mysql:5.7

• Create the database powerjob-product required by PowerJob in MySQL;

CREATE DATABASE IF NOT EXISTS `powerjob-product` DEFAULT CHARSET utf8mb4

• Start the MongoDB service in the Docker container.

docker run -p 27017:27017 --name mongo \
-v /mydata/mongo/db:/data/db \
-d mongo:4.2.5

Install dispatch center

Installing PowerJob's dispatch center in Docker environment is very simple, it takes a minute to get it!

• Download the Docker image of the image powerjob-server:

docker pull tjqq/powerjob-server:latest

• Run the powerjob-server service in a Docker container:

docker run -p 7700:7700 -p 10086:10086 --name powerjob-server \
--link mysql:db \--link mongo:mongo \-e TZ="Asia/Shanghai" \
-e JVMOPTIONS="" \
-e PARAMS="--spring.profiles.active=product --spring.datasource.core.jdbc-url=jdbc:mysql://db:3306/powerjob-product?useUnicode=true&characterEncoding=UTF-8 --spring.datasource.core.username=root --spring.datasource.core.password=root --spring.data.mongodb.uri=mongodb://mongo:27017/powerjob-product" \
-v ~/docker/powerjob-server:/mydata/powerjob/powerjob-server \
-v ~/.m2:/mydata/powerjob/.m2 \
-d tjqq/powerjob-server:latest

• After running successfully, you can access the web interface of powerjob-server. Note that the Linux firewall needs to open two ports 7700 and 10086. The access address is: http://192.168.3.101:7700/

 

Initialize the actuator in the application

After installing the dispatch center, we need to initialize the PowerJob executor (powerjob-worker) in the SpringBoot application.

• First add powerjob-worker dependencies in pom.xml:

<dependency>
    <groupId>com.github.kfcfans</groupId>
    <artifactId>powerjob-worker-spring-boot-starter</artifactId>
    <version>3.2.3</version>
</dependency>

• Then add powerjob-worker related configuration in the application.yml configuration file, pay attention to the configuration of powerjob.worker.app-name;

powerjob:
  worker:
    akka-port: 27777 # akka 工作端口
    app-name: mall-tiny-powerjob # 接入应用名称，用于分组隔离
    server-address: 192.168.3.101:7700 # 调度服务器地址
    store-strategy: disk # 持久化方式

• To add a standalone processor StandaloneProcessor, you only need to inherit the BasicProcessor interface and implement the process method;

package com.macro.mall.tiny.job;
@Slf4j
@Component
public class StandaloneProcessor implements BasicProcessor {
    @Override
    public ProcessResult process(TaskContext context){
        //OmsLogger可以直接将日志上报到powerjob-server
        OmsLogger omsLogger = context.getOmsLogger();
        omsLogger.info("StandaloneProcessor start process,context is {}.", context);
        log.info("jobParams is {}", context.getJobParams());
        return new ProcessResult(true, "Process success!");
    }
}

• After packaging and uploading the image, run the SpringBoot application service in the Docker container. Note that the configured time zone must be consistent with the scheduling center.

docker run -p 8080:8080 --name mall-tiny-powerjob \
--link mysql:db \-v /etc/localtime:/etc/localtime \-v /mydata/app/mall-tiny-powerjob/logs:/var/logs \-e TZ="Asia/Shanghai" \
-d mall-tiny/mall-tiny-powerjob:1.0-SNAPSHOT

Configuration and execution of tasks

With the actuator and dispatch center, we only need to configure the task in the dispatch center to achieve task scheduling.

• First of all, we need to register the application (integrated actuator) in the dispatch center. The application name is the powerjob.worker.app-name property in application.yml, where mall-tiny-powerjob:123456 is used;

 

• After that, we can see a machine information on the homepage;

 

• Then add a task to the task management function, here we use the CRON method to set the processing method in the executor to be executed every 20 seconds;

 

• Click Run in the task list to start the task;

 

• Click More -> Run Log in the task list to view the run log of the task;

 

• Click on the log to view the log reported in the processor, jobParams is the parameter set when we created the task before;

 

• Click Details to view the result of this triggered task, that is, the information we returned in ProcessResult.

Get the project source code address: forward + comment, follow my private message and reply "666" to get it for free

Recommended reading for high-quality articles:

Alibaba advanced interview questions (first issue, 136 high-frequency questions, including answers)

https://blog.csdn.net/weixin_45132238/article/details/107251285

GitHub Biaoxing 20w's 4 low-level interview guides (computer bottom layer + operating system + algorithm), interview headlines/Tencent is right!

https://blog.csdn.net/weixin_45132238/article/details/108640805

Alibaba internal architecture combat: SpringBoot/SpringCloud/Docker/Nginx/distributed

https://blog.csdn.net/weixin_45132238/article/details/108666255