Python task scheduling module APScheduler

A, APScheduler what & APScheduler four kinds part?

APScheduler the entire process for Advanced Python Scheduler, Python is a lightweight task scheduling framework. It allows you to schedule tasks regularly performed so as Cron, and supports any Python function or callable objects.

1, the scheduler (Scheduler)

The scheduler (Scheduler) is the other part. You typically use only a dispatcher, application developers do not usually deal with job storage, scheduler and trigger directly, on the contrary, the scheduler provides appropriate treatment of these interfaces. Configuration may be stored and executed in a job done in a scheduler, such as add, modify, and remove operations.　

For different scenarios not be selected scheduler:

• BlockingScheduler: When the dispatcher is the only thing you use when you want to run.
• == BackgroundScheduler ==: When you do not want to run any other frameworks and scheduler uses (used) in the background of your application execution.
• AsyncIOScheduler: When your program uses asyncio (an asynchronous frame) when using.
• GeventScheduler: When your program uses gevent (high-performance concurrent Python framework) when using.
• TornadoScheduler: When your program based Tornado (a web framework) when using.
• TwistedScheduler: When your program uses Twisted (an asynchronous frame) when using
• QtScheduler: time if your application is a Qt application can be used.

# BackgroundScheduler: 调度器在后台线程中运行，不会阻塞当前线程。


from apscheduler.schedulers.background import BackgroundScheduler
import time

scheduler = BackgroundScheduler()
 
def job1():
    print "%s: 执行任务"  % time.asctime()

scheduler.add_job(job1, 'interval', seconds=3)
scheduler.start()

while True:
    pass

2, storage job (job store)

Storing job (job store) is mainly used to store scheduled jobs, job storing default operation is simply stored in memory.

jobstore scheduler to provide deletions in the job change search interfaces, depending on the storage of, the following points:

• MemoryJobStore: No serialization, jobs there memory, CRUD operations are also in memory
• SQLAlchemyJobStore: All sqlalchemy supported databases can be used as backend, CRUD operations converted to the corresponding backend sql statement
• MongoDBJobStore：用mongodb作backend
• RedisJobStore: as the backend with redis
• RethinkDBJobStore: 用rethinkdb 作backend
• ZooKeeperJobStore：用ZooKeeper做backend

3, the actuator (Executor)

The main processing tasks running actuators (executor), mainly to the timing of tasks callable (function) submitted to a process or a thread to perform. When the task is completed, the actuator will notify the scheduler.

The most common actuator (executor) in two ways:

• ProcessPoolExecutor (process pool)
• ThreadPoolExecutor (thread pool, max: 10)

4, the trigger (Triggers)

When scheduling a task, you need to set a trigger (triggers) for it. Triggers decide what date / time, what kind of form of execution timing of this task.

APScheduler There are three built-in trigger:

• date: specify a definite point in time, job performed only once.
• interval: Specifies the time interval (fixed intervals) performed periodically.
• cron: periodically performed using a cron expression for scene (within a specified time) of the job to run periodically. Use the crontab same way under linux.

4.1, date regular schedule (work only once)

Parameters are as follows:

• run_date (datetime | str) - the job of running the date or time
• timezone (datetime.tzinfo | str) - specified 时区

<!--# 2016-12-12运行一次job_function-->
sched.add_job(job_function, 'date', run_date=date(2016, 12, 12), args=['text'])

<!--# 2016-12-12 12:00:00运行一次job_function-->
<!--args=[]中是传给job_function的参数-->
sched.add_job(job_function, 'date', run_date=datetime(2016, 12, 12, 12, 0, 0), args=['text'])

4.2, interval: to perform every once in a while

weeks=0 | days=0 | hours=0 | minutes=0 | seconds=0, start_date=None, end_date=None, timezone=None

• weeks (int) - weeks interval
• days (int) - interval days
• hours (int) - interval hours
• minutes (int) - interval minutes
• seconds (int) - the number of seconds interval
• start_date (datetime | str) - Start Date
• end_date (datetime | str) - End Date
• timezone (datetime.tzinfo|str) – 时区

<!--每隔2小时执行一次-->
scheduler.add_job(my_job, 'interval', hours=2)


<!--设置时间范围，在设置的时间范围内每隔2小时执行一次-->
scheduler.add_job(my_job, 'interval', hours=2, start_date='2017-9-8 21:30:00', end_date='2018-06-15 21:30:00)


<!--使用装饰器的方式添加定时任务-->
@scheduler.scheduled_job('interval', id='my_job_id', hours=2)
def my_job():
    print("Hello World")

4.3, cron: crontab using the same way under linux

(year=None, month=None, day=None, week=None, day_of_week=None, hour=None, minute=None, second=None, start_date=None, end_date=None, timezone=None)

In addition to week and day_of_week, their default value is *

• E.g. day = 1, minute = 20, which is equal to

year='*', month='*', day=1, week='*', day_of_week='*', hour='*', minute=20, second=0

Work will be performed in 20 minutes per hour of time on the first day of each month

expression	Parameter Type	description
*	all	Wildcards. Example: minutes = * is triggered every minute
*/a	all	Wildcards can be a divisible.
a-b	all	Ab trigger range
a-b/c	all	Range ab, and c can be triggered when it is divisible
xth y	day	The first few weeks triggered. x is the first of several, y is the day of the week
last x	day	Month, the last few weeks triggered
last	day	Last day of the month triggered
x,y,z	all	Combination of expressions may be combined to determine the value of the expression or over

• year (int | str) - years, 4 digits
• month (int | str) - May (range 1-12)
• day (int | str) - days (range 1-31)
• week (int | str) - weeks (range 1-53)
• day_of_week (int | str) - a few weeks first few days or weeks (range 0-6 or mon, tue, wed, thu, fri, sat, sun)
• hour (int | str) - When (range 0-23)
• minute (int | str) - min (range 0-59)
• second (int | str) - seconds (range 0-59)
• start_date (datetime | str) - the earliest start date (included)
• end_date (datetime | str) - latest end time (included)
• timezone (datetime.tzinfo | str) - specified 时区

sched.add_job(my_job, 'cron', hour=3, minute=30)
sched.add_job(my_job, 'cron', day_of_week='mon-fri', hour=5, minute=30, end_date='2017-10-30')

@sched.scheduled_job('cron', id='my_job_id', day='last sun')
def some_decorated_task():
    print("I am printed at 00:00:00 on the last Sunday of every month!")
    

Two, How: APSched how to use?

installation

• pip install

pip install apscheduler

• Source Installation (https://pypi.python.org/pypi/APScheduler/)

python setup.py install

Quick Start

# first.py

from apscheduler.schedulers.blocking import BlockingScheduler
import time

# 实例化一个调度器
scheduler = BlockingScheduler()
 
def job1():
    print "%s: 执行任务"  % time.asctime()

# 添加任务并设置触发方式为3s一次
scheduler.add_job(job1, 'interval', seconds=3)

# 开始运行调度器
scheduler.start()

• Perform output:

> python first.py

Fri Sep  8 20:41:55 2017: 执行任务
Fri Sep  8 20:41:58 2017: 执行任务
...

Task Action

1, add a task

Method One: Call add_job () method

Call add_job () method returns a apscheduler.job.Job instance can be used to change or remove job

job = scheduler.add_job(myfunc, 'interval', minutes=2)

Method two: decorator scheduled_job ()

Suitable for applications that run will not change during the job

from apscheduler.schedulers.blocking import BlockingScheduler
sched = BlockingScheduler()
# 装饰器
@sched.scheduled_job('interval', id='my_job_id', seconds=5)
def job_function():
    print("Hello World")
# 开始
sched.start()

2, delete the task

 <!--方法一：通过作业ID或别名调用remove_job()删除作业-->
scheduler.add_job(myfunc, 'interval', minutes=2, id='my_job_id')
scheduler.remove_job('my_job_id')


<!-- 方法二：通过add_job()返回的job实例调用remove()方法删除作业-->
job = scheduler.add_job(myfunc, 'interval', minutes=2)
job.remove()

3, pause & continue the task

By Job Scheduler instance or itself easily pause and resume operations. When a job is suspended, the next time will be cleared to run until the job recovery, no longer calculate the run time. To pause a job, use any of the following:

<!--根据任务实例-->
job = scheduler.add_job(myfunc, 'interval', minutes=2)
<!--暂停-->
job.pause() 
<!--继续-->
job.resume() 


 <!--根据任务id暂停-->
scheduler.add_job(myfunc, 'interval', minutes=2, id='my_job_id')
scheduler.pause_job('my_job_id')
scheduler.resume_job('my_job_id')

4, modify task properties

<!--修饰：-->
job.modify(max_instances=6, name='Alternate name')


<!--重设：-->
scheduler.reschedule_job('my_job_id', trigger='cron', minute='*/5')

5, access to job list

• Use get_jobs () to get all the job instances.
• Use print_jobs () outputs formatted list of all jobs.
• Use get_job (task ID) Gets the specified task list of jobs.

<!--获取所有的job实例-->
apscheduler.get_jobs()

6, start & shutdown tasks

scheduler.start() #开启
scheduler.shotdown(wait=True|False) #关闭 False 无论任务是否执行，强制关闭

Third, some of the regular tasks script

1, a scheduled task to run the script execution daily at 00:30:30

import datetime
from apscheduler.schedulers.blocking import BlockingScheduler
from app.untils.log_builder import sys_logging

scheduler = BlockingScheduler()   # 后台运行

 # 设置为每日凌晨00:30:30时执行一次调度程序
@scheduler.scheduled_job("cron", day_of_week='*', hour='1', minute='30', second='30')
def rebate():
        print "schedule execute"
        sys_logging.debug("statistic scheduler execute success" + datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"))


if __name__ == '__main__':
    try:
        scheduler.start()
        sys_logging.debug("statistic scheduler start success")
    except (KeyboardInterrupt, SystemExit):
        scheduler.shutdown()
        sys_logging.debug("statistic scheduler start-up fail")

2, 0:00 pm each day - during 8:00 am, to perform the task once every 5 seconds.

# 每天晚上0点 - 早上8点期间，每5秒执行一次任务。
scheduler.add_job(my_job, 'cron',day_of_week='*',hour = '0-8',second = '*/5')

3, to perform tasks in 0,10,20,30,40,50 sharing.

<!--# 可以被10整除的时间点执行任务，这个任务就表示在0、10、20、30、40、50分时都会执行任务-->
scheduler.add_job(my_job, 'cron',day_of_week='*',minute = '*/10')

4, until 2020-05-30, a week from Monday to Friday 5:30 am to perform a timed task both

<!--直到2020-05-30 00:00:00，每周星期从星期一到星期五的早上5:30都执行一次定时任务-->
sched.add_job(my_job(),'cron', day_of_week='mon-fri', hour=5, minute=30,end_date='2020-05-30')


<!--# 截止到2016-12-30 00:00:00，每周一到周五早上五点半运行job_function-->
sched.add_job(job_function, 'cron', day_of_week='mon-fri', hour=5, minute=30, end_date='2016-12-31')

5, the task execution timing 6, 7, 8, 11, 1,2,3 point of the third Friday of December

<!--job_function将会在6,7,8,11,12月的第3个周五的1,2,3点运行-->

sched.add_job(job_function, 'cron', month='6-8,11-12', day='3rd fri', hour='0-3')

6, the program is executed once every five seconds

<!--#表示每5秒执行该程序一次，相当于interval 间隔调度中seconds = 5-->
sched.add_job(my_job, 'cron',second = '*/5')

Reference material

• Python task scheduling module APScheduler: https: //segmentfault.com/a/1190000011084828
• The official document: https: //apscheduler.readthedocs.io/en/v3.3.0/modules/schedulers/base.html#apscheduler.schedulers.base.BaseScheduler.print_jobs
• API documentation: https: //apscheduler.readthedocs.io/en/v3.3.0/py-modindex.html
• python modules (APScheduler scheduled task): https: //blog.csdn.net/qq_37634812/article/details/79208782
• Python regular tasks APScheduler Detailed Examples Example: https: //www.jb51.net/article/165895.htm