background

Whether or Internet applications, enterprise applications, are filled with a large number of batch jobs. Task scheduling systems often need some help developers solve the problem. With the gradual evolution of the micro-service architecture, and gradually evolved into a distributed architecture monomer, micro-services architecture. In this context, many of the original task scheduling platform can not meet the needs of business systems. So there are some distributed task-based management platform. These platforms have their own characteristics, but at their own shortcomings, such as not support task scheduling, and business high coupling does not support cross-platform and so on. Very not meet the needs of next-generation micro-architecture of the service, and therefore should believe the company has developed a micro-task scheduling service platform (SIA-TASK).

SIA is appropriate letter to the company based on the development platform Simple is Awesome abbreviation, SIA-TASK (micro-task scheduling service platform) is one of the important products, SIA-TASK fit the current micro-services architecture model, with cross-platform layout, high available, non-invasive, consistency, asynchronous parallel, dynamic expansion, real-time monitoring and so on.

Project Description

SIA-TASK task scheduling is one solution. Meta data acquisition task, then the task visual layout, the final task scheduling, and take on the task to monitor the whole process, easy to use. Business completely non-invasive, task scheduling model can be generated in line with expectations through simple and flexible configuration.

SIA-TASK reference design micro-services, access to distribution task metadata on each task executor, and uploaded to the task registry. Using online scheduling task, the task to dynamically modify the clock, as the task scheduling using HTTP protocol, with JSON unified data format clocked parsed by the control center, process tasks, task notification.

Key Terms

Tasks (Task): basic execution unit, actuator exposed outside of an HTTP call interface;
Jobs (Job): the presence of one or more logical relationships to each other (serial / parallel) of the tasks, the smallest unit of scheduling tasks dispatch center;
Plan (Plan): the number of job execution order, each job has its own execution cycle, there is no plan implementation cycle;
Mission control center (Scheduler): performs scheduling according to the execution period of each job, i.e., an HTTP request in accordance with the logic program, a job, the task;
Task scheduling center (Config): choreography centers use task to create plans and operations;
Task executor (Executer): receiving a HTTP request execution of service logic;
Hunter: Spring project expansion pack, is responsible for the implementation of tasks capture, upload registration center, business can rely on this component be written Task.

Characteristics of micro-services task scheduling platform

Uploaded to the registry based on the annotation task automatically grab the task, adding @OnlineTask comment on the exposure to the HTTP service method, @ OnlineTask will automatically grab the IP address of the method is located, the port, the path request, the request method, request parameter format and other information (ZooKeeper), and synchronous write persistent store, i.e., the task of this method;
Annotation-based non-invasive multi-threaded control, single task instance must run single-threaded task scheduling framework to automatically block comment @OnlineTask single-threaded operating control, will not be scheduled again in keeping a job is running. And the whole process completely control the perception of developers.
Adaptive scheduler task allocation, the failure occurred during the execution of the task, the abnormality. According to multi-task can be customized policies to re-awaken the task to ensure uninterrupted mission.
Highly flexible task scheduling mode, SIA-TASK design philosophy is based on atomic task, multiple tasks are combined to form a working relationship in accordance with the execution. While running into the central task scheduling and task scheduling center, so that the composition of the jobs in the job scheduling and spaced apart of each other. When we need to adjust the flow of work, we only need to be processed at the center of choreography. Meanwhile choreography center support tasks in the serial, parallel, and other branches of organized relationship. Different tasks at the same task instances, also support a variety of scheduling processing.

Micro-service task scheduling platform

SIA-TASK is divided into five parts:

Task executor
Mission control center
Task scheduling center
Task Registry (zookeeper)
Persistent storage (Mysql)

SIA-TASK major operating logic:

Reported to the registry by annotation task crawl task executor of the task
Task scheduling center to obtain data from mission to save the registry orchestrate persistent storage
Mission control center stores the acquired information from the persistent scheduling
Mission control center in accordance with the scheduling task executor logical access

UI Preview

Home monitoring provides a multi-dimensional

The dispatcher information: Display scheduling information (load capacity, early warning value), as well as the distribution of work.
Scheduling information: Impressions dispatch control center is triggered, jobs, tasks, multi-dimensional scheduling statistics.
Statistical docking project: to use project statistical system, job number, job number, and so on.

Job scheduling monitoring provides real-time monitoring has been submitted to show.

Job status real-time monitoring: the project team as a unit panel, showing job runs state.
Real-time Log Correlation: Log real-time correlation can be demonstrated by coloring the status icon.