Easy Scheduler 1.1.0 Release release, distributed workflow task scheduling system

Easy Scheduler Release 1.1.0

Easy Scheduler 1.1.0 is the first version of the 1.1.x series.

New features:

• [EasyScheduler-391] run a process under a specified tenement user
• [ EasyScheduler-288 ] the Feature / qiye_weixin
• [ EasyScheduler-189 ] such as the Kerberos security support
• [ EasyScheduler-398 ] administrator, there are tenants (install.sh tenant default setting), you can create a resource, project and data sources (there is a limit administrator)
• [ EasyScheduler-293 ] Click to select the process parameters at runtime, there is no place to see, nor save
• [ EasyScheduler-401 ] Timing Timing is easy once per second, later time to complete the next page can be displayed at the trigger time
• [EasyScheduler-493]add datasource kerberos auth and FAQ modify and add resource upload s3

Enhanced:

• [EasyScheduler-227] upgrade spring-boot to 2.1.x and spring to 5.x
• [ EasyScheduler-434 ] zk worker nodes and the number of inconsistencies in mysql
• [ EasyScheduler-435 ] format mail verification
• [ EasyScheduler-441 ] Add inhibit operation node has completed node detects
• [ EasyScheduler-400 ] Home page, queue statistics discord, no statistical data command
• [ EasyScheduler-395 ] for fault-tolerant recovery process, the state is not running **
• [EasyScheduler-529] optimize poll task from zookeeper
• [ EasyScheduler-242 ] Server-worker node acquires task performance issues
• [ EasyScheduler-352 ] worker packet queue Consumption
• [ EasyScheduler-461 ] to see the data source parameters, you need to encrypt the account password information
• [ EasyScheduler-396 ] Dockerfile optimization, and the associated automatic playing Dockerfile mirror and github
• [EasyScheduler-389]service monitor cannot find the change of master/worker
• [EasyScheduler-511]support recovery process from stop/kill nodes.
• [ EasyScheduler-399 ] HadoopUtils specified user operation, rather than deployment user **
• [EasyScheduler-378]Mailbox regular match
• [EasyScheduler-625]EasyScheduler call shell "task instance not set host"
• [EasyScheduler-622]Front-end interface deployment k8s, background deployment big data cluster session error

repair:

• [ EasyScheduler-394 when] master & worker deployed on the same machine, if the restart master & worker service, will lead to the scheduled before the task can not continue scheduling
• [EasyScheduler-469]Fix naming errors,monitor page
• [EasyScheduler-392]Feature request: fix email regex check
• [ EasyScheduler-405 ] Timing modify / add a page, start time and end time can not be the same
• [ EasyScheduler-517 ] complement - sub-workflows - time parameter
• [ EasyScheduler-532 Problems] node does not execute Python
• [EasyScheduler-543]optimize datasource connection params safety
• [ EasyScheduler-569 ] can not really stop a scheduled task
• [ EasyScheduler-463 ]-mail authentication does not support uncommon suffix mailbox
• [EasyScheduler-650]Creating a hive data source without a principal will cause the connection to fail
• [EasyScheduler-641]The cellphone is not supported for 199 telecom segment when create a user
• [EasyScheduler-627]Different sql node task logs in parallel in the same workflow will be mixed
• [EasyScheduler-655]when deploy a spark task,the tentant queue not empty,set with a empty queue name
• [EasyScheduler-667]HivePreparedStatement can't print the actual SQL executed

thank:

Last but most importantly, there is no contribution of the following partners no birth of a new version:

Baoqi, jimmy201602, samz406, petersear, millionfor, hyperknob, fanguanqun, yangqinlong, qq389401879, chgxtony, Stanfan, lfyee, thisnew, hujiang75277381, sunnyingit, lgbo-ustc, ivivi, lzy305, JackIllkid, telltime, lipengbo2018, wuchunfu, telltime, chenyuan9028, zhangzhipeng621, thisnew, 307526982, crazycarry

And a micro-channel group in a large number of enthusiastic partner! In this very grateful!

Easy Scheduler

Easy Scheduler for Big Data

Design features:  a distributed and scalable visualization DAG workflow task scheduling system. We committed to resolve dependencies of complex data processing flow, the scheduling system in data processing procedures 开箱即用. Its main objectives are as follows:

• By way of the DAG will associate Task dependencies in accordance with the task of running the state can monitor real-time visualization tasks
• It supports various types of tasks: Shell, MR, Spark, SQL (mysql, postgresql, hive, sparksql), Python, Sub_Process, Procedure, etc.
• Workflow Support regular schedule, dependent scheduling, scheduling manual, manual pause / stop / recovery, while supporting failure retry / alarm, failure recovery, the operation Kill tasks from the specified node
• Support workflow priorities, failover priority tasks and tasks and task time-out alarm / failure
• Workflow support global parameters and custom parameters settings node
• Support resources online file upload / download, manage, and support for online document creation, editing,
• Support Task Log online to view and scroll, online download logs, etc.
• Implement cluster HA, achieved Master Worker clusters and cluster to the center by Zookeeper
• Support for the Master/Worker view cpu load, memory, cpu online
• Support the operation history tree workflow / Gantt chart shows, support mission state statistics, statistical process status
• Support complement
• Support for multi-tenancy
• International support
• There are more waiting for partners to explore

Comparison with similar scheduling system

Screenshot parts of the system

File

• Back-end deployment documentation

• Front-end deployment documentation

• manual

• Upgrade documentation

• I want to experience

More documents please refer  easyscheduler Chinese online documentation

Recent R & D program

EasyScheduler work plan: research and development program  , under which the card is version 1.1.0 In Develop functional, TODO card is a to-do list (including feature ideas)