Distributed task scheduling platform XXL-JOB
Introduction:
XXL-JOB is a lightweight distributed task scheduling platform. Its core design goals are rapid development, simple learning, lightweight, and easy expansion. The source code is now open and connected to the online product lines of many companies, ready to use out of the box.
Official website document
characteristic
1. Simple: support CRUD operation of tasks through the Web page, simple operation, one minute to get started;
2. Dynamic: support dynamic modification of task status, start/stop tasks, and terminate running tasks, with immediate effect;
3. Dispatch center HA (Central): Scheduling adopts a central design, the "dispatch center" self-developed scheduling components and supports cluster deployment, which can ensure the dispatch center HA;
4. Actuator HA (distributed): task is executed in a distributed manner, task "executor" Support cluster deployment to ensure task execution HA;
5. Registration center: The executor will automatically register tasks periodically, and the dispatch center will automatically discover the registered tasks and trigger execution. At the same time, it also supports manual entry of the actuator address;
6. Flexible expansion and shrinkage: Once a new actuator machine is online or offline, the task will be redistributed during the next scheduling;
7. Routing strategy: Provide rich actuator cluster deployment The routing strategies include: first, last, polling, random, consistent HASH, least frequently used, least recently used, failover, busy transfer, etc.;
8. Failover: Task routing strategy selection "Failure" In the case of "transfer", if a machine in the actuator cluster fails, it will automatically failover to a normal actuator to send scheduling requests.
9. Blocking processing strategy: the processing strategy when scheduling is too intensive and the executor is too late to process. Strategies include: single machine serial (default), discarding subsequent scheduling, and overwriting previous scheduling;
10. Task timeout control: support custom task timeout time, task Running overtime will actively interrupt the task;
11. Task failure retry: support custom task failure retry times, when the task fails, it will actively retry according to the preset failure retry times; the fragmentation task supports fragmentation Granular failure retry;
12. Task failure alarm; default email failure alarm is provided, and expansion interface is reserved at the same time, which can easily expand the alarm methods such as SMS and nailing;
13. Fragmented broadcast task: When the executor cluster is deployed and the task routing strategy selects "fragmented broadcasting", a task scheduling will broadcast to trigger all executors in the cluster to perform a task, and the fragmented task can be developed according to the sharding parameters ;
fragmentation fragmentation broadcasting task to the actuator as a dimension to support dynamic expansion actuator cluster dynamically increasing the number of fragments, collaborative business processes; during large amount of data traffic operation can significantly improve tasks: 14, dynamic fragmentation Processing power and speed.
15. Event triggering: In addition to triggering task execution in "Cron mode" and "task-dependent mode", it supports event-based triggering of tasks. The dispatch center provides API services that trigger a single execution of tasks, which can be flexibly triggered according to business events.
16. Task progress monitoring: support real-time monitoring of task progress;
17. Rolling real-time log: support online viewing of scheduling results, and support real-time viewing of the complete execution log output by the actuator in Rolling mode;
18. GLUE: provide Web IDE, support online Develop task logic code, publish dynamically, compile in real time, and omit the process of deployment and online. Support 30 versions of historical version backtracking.
19. Script tasks: support the development and operation of script tasks in GLUE mode, including Shell, Python, NodeJS, PHP, PowerShell and other types of scripts;
20. Command line tasks: natively provide generic command line task Handler (Bean task, "CommandJobHandler") ; The business side only needs to provide the command line;
21. Task dependency: support the configuration of sub-task dependencies. When the parent task is executed and successfully executed, it will automatically trigger the execution of a sub-task, and multiple sub-tasks are separated by commas;
22. Consistency: The "dispatch center" guarantees the consistency of cluster distributed scheduling through DB locks,
25. Data encryption: The communication between the dispatch center and the actuator is encrypted to improve the security of dispatching information;
26. Email alarm: Support email alarm when the task fails, and support the configuration of multiple email addresses to send alarm emails;
27. Push maven central Warehouse: The latest stable version will be pushed to the maven central warehouse, which is convenient for users to access and use;
28. Running reports: Support real-time viewing of running data, such as the number of tasks, scheduling times, and the number of executors; and scheduling reports, such as scheduling Date distribution diagram, scheduling success distribution diagram, etc.;
29. Fully asynchronous: fully asynchronous design and implementation of task scheduling process, such as asynchronous scheduling, asynchronous operation, asynchronous callback, etc., effectively peaking traffic for intensive scheduling, theoretically supporting tasks of any duration operation;
30, cross-language: dispatch center and actuators provide RESTful API service language-independent, third-party can accordingly any language docking dispatch center or implement actuator. In addition, other cross-language solutions such as "multitasking mode" and "httpJobHandler" are also provided;
31. Internationalization: The dispatch center supports international settings and provides two optional languages, Chinese and English, and the default is Chinese;
32 、Containerization: Provide official docker images and update and push dockerhub in real time to further realize product out-of-the-box use;
33. Thread pool isolation: schedule thread pool for isolation and split, slow tasks are automatically degraded into "Slow" thread pool to avoid consumption Dispatch threads as much as possible to improve system stability;
34. User management: support online management of system users, and there are two roles of administrator and ordinary user;
35. Authority control: authority control in the dimension of actuators, administrators have full authority, ordinary users Need to assign executor permissions before allowing relevant operations;
deploy
拉取代码：这是我放到了自己 的 gitee 上，拉取速度非常快
git clone https://gitee.com/Liu_YiHong/xxl-job.git 导入sql 文件到数据库MySQL
cd xxl-job/doc/db //会有一份tables_xxl_job.sql 文件，导入到数据库就好。 修改xxl-job-admin配置文件：
cd xxl-job/xxl-job-admin/src/main/resources
vim application.properties
修改如下 参数：
server.port=8086
spring.datasource.url=jdbc:mysql://数据库IP:3306/xxl_job?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=Asia/Shanghai
spring.datasource.username=root
spring.datasource.password=0GXwwchW4rP修改 xxl-job-executor-sample-springboot 配置文件：
cd xxl-job/xxl-job-executor-samples/xxl-job-executor-sample-springboot/src/main/resources
vim application.properties
修改如下参数：
server.port=8087
xxl.job.admin.addresses=http://xxxxxxIP地址:8086/xxl-job-admincd xxl-job //项目得根目录下打包
1·打包之前最好修改maven 得仓库地址，否则太慢。
Maven 配置
打开 Maven 的配置文件(windows机器一般在maven安装目录的conf/settings.xml)，在<mirrors></mirrors>标签中添加 mirror 子节点:
<mirror>
<id>aliyunmaven</id>
<mirrorOf>*</mirrorOf>
<name>阿里云公共仓库</name>
<url>https://maven.aliyun.com/repository/public</url>
</mirror>
2· 执行打包命令
mvn clean package -Dmaven.test.skip=true3·制作xxl-job-admin镜像
cd xxl-job-admin
docker build -t xxl-job-admin .4`制作 xxl-jobs-pringboot 镜像
cd xxl-job/xxl-job-executor-samples/xxl-job-executor-sample-springboot
docker build -t xxl-jobs-pringboot .5·编写docker-compose.yml 文件
version: '3'
services:
xxl-job-admin:
image: xxl-job-admin
restart: always
container_name: xxl-job-admin
ports:
- 8086:8086
environment:
- TZ=Asia/Shanghai
volumes:
- ./data/applogs:/data/applogs
xxl-job-springboot:
image: xxl-jobs-pringboot
restart: always
container_name: xxl-job-springboot
ports:
- 8087:8087
environment:
- TZ=Asia/Shanghai6·启动服务：
docke-compose up -d7·访问方式：
http://IP+端口号/xxl-job-admin