Distributed Task Scheduling System
1. The contab instruction quickly implements simple scheduled tasks.
You can edit a simple scheduling task by directly using the crontab -e command on a Linux machine, which is suitable for situations where the request volume is not very large.
* * * * * echo "timer">/root/test.out
Disadvantages: It can only be executed on this machine; it can only be executed at the minute level.
When configuring the expression of each part, you can specify a specific value. For example, the minute bit is set to 0, which means it will be executed at the 0th minute each time
. In addition to these basic numbers, several special characters are allowed:
- represent all possible values.
- Indicates the specified range. For example, configure 1-10 in the day bit, indicating the 1st to 10th day of each month.
- , represents an enumeration. For example, minute bit configuration 3,5 means execution in the 3rd minute and 5th minute.
- / indicates the specified increment. For example, configuring 0/15 in the minute bit means starting from minute 0 and executing every 15 minutes. 3/20 means the third minute starts and is executed every 20 minutes.
There are also several special symbols that are rarely used. For example, day and week positions. To avoid possible conflicts, you can set one of the values to ?
xxljob quickly implements lightweight distributed task scheduling
There are many frameworks for implementing scheduled tasks, and elasticjob and xxljob are currently used more frequently in China. These two projects have a lot of application scenarios. Among them, elasticjob relies on zookeeper for distributed task coordination, while xxljob only needs to coordinate through the database.
Case
package com.xxl.job.service.handler;
import com.xxl.job.core.context.XxlJobHelper;
import com.xxl.job.core.handler.IJobHandler;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class HttpJobHandler extends IJobHandler {
@Override
public void execute() throws Exception {
String target = XxlJobHelper.getJobParam();
XxlJobHelper.log("received job target:"+target);
boolean isPostMethod= false;
// request
HttpURLConnection connection = null;
BufferedReader bufferedReader = null;
try {
// connection
URL realUrl = new URL(target);
connection = (HttpURLConnection) realUrl.openConnection();
connection.setDoOutput(isPostMethod);
connection.setDoInput(true);
connection.setUseCaches(false);
connection.setReadTimeout(5 * 1000);
connection.setConnectTimeout(3 * 1000);
connection.setRequestProperty("connection", "Keep-Alive");
connection.setRequestProperty("Content-Type","application/json;charset=UTF-8");
connection.setRequestProperty("Accept-Charset","application/json;charset=UTF-8");
// do connection
connection.connect();
// valid StatusCode
int statusCode = connection.getResponseCode();
if (statusCode != 200) {
throw new RuntimeException("Http Request StatusCode(" +
statusCode + ") Invalid.");
}
// result
bufferedReader = new BufferedReader(new
InputStreamReader(connection.getInputStream(), "UTF-8"));
StringBuilder result = new StringBuilder();
String line;
while ((line = bufferedReader.readLine()) != null) {
result.append(line);
}
String responseMsg = result.toString();
XxlJobHelper.log(responseMsg);
return;
} catch (Exception e) {
XxlJobHelper.log(e);
XxlJobHelper.handleFail();
return;
} finally {
try {
if (bufferedReader != null) {
bufferedReader.close();
}
if (connection != null) {
connection.disconnect();
}
} catch (Exception e2) {
XxlJobHelper.log(e2);
}
}
}
}
In this implementation, the accessed target address, target, is passed in as a task parameter. Therefore, next you only need to configure the URL in the form of parameters in the task configuration interface of xxljob.
The core of timing tasks - time wheel algorithm
The time wheel algorithm can be simply seen as being implemented by 循环数组+双向链表a 数据结构. The loop array forms a ring structure, and the pointer moves one step every tickDuration time. Each node in the array is mounted with a scheduled task list in a doubly linked list structure.
The tasks on the doubly linked list have an attribute called remainingRounds, which indicates the remaining rounds of the current task. Whenever the pointer reaches the data node corresponding to the task, remainingRounds is reduced by 1. When remainingRounds reaches 0, the current scheduled task is triggered. It can be seen from this principle that the smaller the tickDuration, the more accurate the timing task will be, but the response will be heavier, and the computing burden on the system will be heavier.