Taobao official reference manual: http://code.taobao.org/p/tbschedule/wiki/index/
http://www.jianshu.com/p/e66aa342f0d1
1. The TBSchedule project can actually be divided into two parts:
①TBSchedule management console. Responsible for monitoring, monitoring task execution status
②The client program that actually executes the job
In actual use, first start zookeeper, then deploy the management console of the TBSchedule web interface, and finally start the client machine that actually executes the job. Here zookeeper does not actually control task scheduling, it is only responsible for communicating with N clients that execute jobs, coordinating, managing, and monitoring the running information of these machines. The driver assigns tasks to the TBSchedule management console, and the console obtains job running information from zookeeper.
TBSchedule indirectly controls the execution of Job by controlling the creation, modification, and deletion of ZNode. Clients executing Job will monitor their corresponding ZNode state update events, so as to achieve the purpose of controlling Job execution through TBSchedule.
1. Deploy zookeeper (for the simple installation of stand-alone mode, please refer to my other blog for cluster mode)
Download address: http://zookeeper.apache.org/releases.html#download, here I downloaded zookeeper-3.3.6.tar.gz.
1. After the download is complete, extract it to the /application/software directory
2. Establish a soft connection ln -s /application/software/zookeeper-3.3.6 /opt/zookeeker
3. Rename zoo_sample.cfg in the /conf directory to zoo.cfg, because zookeeper will find zoo.cfg in this directory to read the configuration information when it starts. There are several important parameters in this file that need to be explained:
tickTime=2000
Defines the unit of measure for time. This means that a tick is 2 seconds. In the future, when configuring time-related things, the unit will be ticks.
dataDir=/opt/zookeer/data
Defines the storage location for snapshot files. Zookeeper will regularly write node information to this directory. This directory must exist, otherwise an error will be reported at startup.
clientPort=2181
Specifies the client connection port. Zookeeper will listen for connection requests on this port.
server.1=127.0.0.1:2000:3000
This parameter only takes effect when the cluster is deployed. The format is: server.id=host:port:port. id represents the unique identifier of the server, generally counting from 1. The first port indicates the communication port between the zookeeper cluster machines, and the second port indicates the communication port used when the cluster machine elects the leader. Leader election is only done when the cluster starts up for the first time, or when the master crashes.
4. After the configuration is complete, switch to the /bin directory and execute: ./zkServer.sh start
You can start zookeeper, which will run in the background by default. If you want to run in the front end, you need to execute: ./zkServer.sh start-foreground
Note: The linux server where Zookeeper is located should be the same time as the machine where the TBSchedule client code is located. For simplicity, you can use the linux command to set the time and date
Second, deploy ScheduleConsole
Download address: http://code.taobao.org/p/tbschedule/wiki/index/ , here I downloaded ScheduleConsole.war directly
ScheduleConsole is a web project written with servlet/JSP. We can directly publish the downloaded ScheduleConsole.war to the tomcat server, and then access it in the browser:
http://localhost:8080/ScheduleConsole
That's it.
The following configuration page appears the first time you access the console:
The first line specifies the address and port of zookeeper
The second line is the timeout. Usernames and passwords are of no use here, just ignore them.
The root directory of the Zookeeper in the third line does not refer to a directory of the centos system where the zookeeper is located, but the directory node znode of the zookeeper. The TBSchedule management console will save the configuration information of the task (such as execution start time, scheduling policy) to the In this directory, the configuration information can be directly read from the directory when the management console is started next time.
Click Save after filling out. Then click the management home page to enter the management page and create a scheduling policy:
At this point, the TBSchedule console is deployed.
TBSchedule client writing
Project Architecture
springBoot + TBSchedule implements task scheduling
Project structure
Project construction
1.pom.xml configuration
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>springboot_tbschedule_demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.3.6.RELEASE</version>
</parent>
<dependencies>
<!-- add typical dependencies for a web application -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>com.taobao.pamirs.schedule</groupId>
<artifactId>tbschedule</artifactId>
<version>3.2.14</version>
</dependency>
<!-- use log4j -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.21</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<!-- Add zookeeper dependency-->
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.3.6</version>
</dependency>
<!-- Dependency-->
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.7</version>
</dependency>
</dependencies>
<build>
<plugins>
<!-- Spring Boot provides a Maven plugin for creating executable jars -->
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<dependencies>
<!-- During our development process, we need to modify frequently, in order to avoid repeated project startup, we can enable hot deployment-->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>springloaded</artifactId>
<version>1.2.5.RELEASE</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
</project>
2.application.yml placement
logging:
level:
root: INFO
job:
zkConfig:
zkConnectString: 10.10.28.111:2181 #Registration center address
rootPath: /tb-schedule/dev #root directory of scheduled tasks, arbitrarily specified, corresponding to the scheduling console configuration
zkSessionTimeout: 60000 #Timeout configuration
userName: admin #Account, arbitrarily specified, corresponding to the scheduling console configuration
password: admin #Password, arbitrarily specified, corresponding to the scheduling console configuration
isCheckParentPath: true
2. Configure TBSchedule to connect to zookeeper
package com.zto.demo.config;
import java.util.HashMap;
import java.util.Map;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import com.taobao.pamirs.schedule.strategy.TBScheduleManagerFactory;
@Configuration
public class TBScheduleJobConfig {
/**
* After creating the Bean, call the initialization method init() of TBScheduleManagerFactory to initialize
*/
@Bean(initMethod="init")
public TBScheduleManagerFactory tbScheduleManagerFactory(
@Value("${job.zkConfig.zkConnectString}")String zkConnectString,
@Value("${job.zkConfig.rootPath}")String rootPath,
@Value("${job.zkConfig.zkSessionTimeout}")String zkSessionTimeout,
@Value("${job.zkConfig.userName}")String userName,
@Value("${job.zkConfig.password}")String password,
@Value("${job.zkConfig.isCheckParentPath}")String isCheckParentPath){
TBScheduleManagerFactory tbScheduleManagerFactory = new TBScheduleManagerFactory();
Map<String, String> zkConfig = new HashMap<String, String>();
zkConfig.put("zkConnectString", zkConnectString);
zkConfig.put("rootPath", rootPath);
zkConfig.put("zkSessionTimeout", zkSessionTimeout);
zkConfig.put("userName", userName);
zkConfig.put("password", password);
System.out.println("userName:"+userName+", password:"+password);
zkConfig.put("isCheckParentPath", isCheckParentPath);
tbScheduleManagerFactory.setZkConfig(zkConfig);
return tbScheduleManagerFactory;
}
}
3. Task entity class definition
package com.zto.demo.schedule.model;
public class TaskModel {
private String name;
private String job;
public TaskModel(String name, String job){
this.name = name;
this.job = job;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getJob() {
return job;
}
public void setJob(String job) {
this.job = job;
}
}
4. Task processing class definition
package com.zto.demo.schedule.task;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import org.apache.log4j.Logger;
import org.springframework.stereotype.Component;
import com.taobao.pamirs.schedule.IScheduleTaskDealSingle;
import com.taobao.pamirs.schedule.TaskItemDefine;
import com.zto.demo.schedule.model.TaskModel;
@Component("dataSyncABean")
public class DataSyncABean implements IScheduleTaskDealSingle<TaskModel>{
private static final Logger LOG = Logger.getLogger (DataSyncABean.class);
@Override
public Comparator<TaskModel> getComparator() {
// TODO Auto-generated method stub
return null;
}
@Override
public List<TaskModel> selectTasks(String taskParameter, String ownSign, int taskQueueNum,
List<TaskItemDefine> taskItemList, int eachFetchDataNum) throws Exception {
LOG.info("IScheduleTaskDealSingleTest configuration parameters, taskParameter:"+taskParameter
+",ownSign:"+ownSign
+",taskQueueNum:"+taskQueueNum
+ ", taskItemList:" + taskItemList
+", eachFetchDataNum:"+eachFetchDataNum
);
LOG.info("");
List<TaskModel> modelList = new ArrayList<TaskModel>();
modelList.add(new TaskModel(String.valueOf(System.currentTimeMillis()), "test1"));
modelList.add(new TaskModel(String.valueOf(System.currentTimeMillis()), "test2"));
return modelList;
}
@Override
public boolean execute(TaskModel model, String ownSign) throws Exception {
System.out.println(model.getJob()+" ************************ "+model.getName());
return true;
}
}
The calling process of tbSchedule is:
①Execute the selectTasks() method, which returns a List object representing the list of tasks you have selected.
②Execute the execute() method, tbschedule will traverse the List you returned in the selectTasks() method, and then call the execute() method for each element.
5. Finally start it in Application
package com.zto.demo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
/**
*
*
* The first annotation used on our Example class is @RestController. This is called a stereotype annotation. It provides advice for people reading the code.
* For Spring, this class plays a special role. In this example, our class is a web @Controller, so when processing incoming web requests,
* Spring will ask him
*
*/
@SpringBootApplication
public class Application{
public static void main(String[] args) throws Exception{
SpringApplication.run(Application.class, args);
}
}
Task scheduling configuration
Go to the TBSchedule management console and create a new task:
Reference: http://blog.csdn.net/yucao2015/article/details/53033628