apscheduler regular tasks framework

A, APScheduler profile:

Python frame timing of the task, performing the timing or periodically to meet the user needs to perform a task, a task based on the date DATE regular, fixed interval The time interval, and a timing crontab tasks similar to types on Linux. And the frame can not only add, delete scheduled tasks, tasks can also be stored in the database, persistence to achieve the task.

Python third party libraries, to provide the Python daemon. It comprises four components, namely:

triggers: the trigger assembly task, the task trigger offers

job stores: Task store component that provides task saving mode

executors: task scheduling component that provides task scheduling

schedulers: task scheduling component that provides task work

Two, APScheduler installation

1) use pip install (Recommended)

PIP install apscheduler #
2 based Source: https: //pypi.python.org/pypi/APScheduler/

# python setup.py install

Third, the basic concept

1, APScheduler four components and instructions

1) triggers (Trigger): Trigger comprises scheduling logic, each job has its own trigger for determining the next job run which, in addition to their initial configuration, the trigger is completely stateless.
2) job stores (store operation): for storing a scheduled job, the job memory default job tasks are simply stored in memory, additional memory can be saved job task to the various databases, support MongoDB, Redis, SQLAlchemy storage. When the job task to the persistent store, data of the job to be serialized, deserialized re-read operation.

3) executors (actuator): The actuator is used to perform regular tasks, but will need to perform the task on the new thread run or thread pool. When the job task is completed, the actuator will notify scheduler. For actuators, you can choose ThreadPoolExecutor by default, but if it comes to special tasks such as CPU-intensive tasks, you can choose the ProcessPoolExecutor, of course, can use two actuators at the same time according to the actual needs

4) schedulers (scheduler): The scheduler is linked to the other part, generally only one application scheduler, application developers do not directly manipulate the trigger, task storage and actuators. Instead scheduler provides an interface process. The task scheduler is configured by a memory and the operation of the actuator, such as can be added, removed, modified job task.

APScheduler provides multiple scheduler, the scheduler may select the appropriate according to specific needs, conventional scheduler include:

BlockingScheduler: suitable for the case to run only a single task in a process commonly used in the scheduler is the only thing you want to run when

BackgroundScheduler: suited for the task, when you want scheduler performs in the background in case the application running in the background.

AsyncIOScheduler: suitable for using asyncio framework

GeventScheduler: suitable for using gevent framework

TornadoScheduler: suitable for use application framework of Tornado

TwistedScheduler: Twisted framework for application use

QtScheduler: for using the QT

2, the configuration scheduler

APScheduler offers many different ways to configure the scheduler, you can use a dictionary or as a way to configure key parameters of the incoming. You can also create a scheduler. In the configuration and adding jobs, so you can get more flexibility in different environments.

3, a simple example

from apscheduler.schedulers.blocking import BlockingScheduler
import time
Examples of a dispatcher #
scheduler = BlockingScheduler()

the jobs that job1 DEF ():
Print "% S: mission"% time.asctime ()

# Add tasks and set a trigger to 3s

scheduler.add_job(job1, 'interval', seconds=3)
# Starts running scheduler
scheduler.start()

4. Each Component Function

1, trigger assembly

trigger trigger job offer, a total of three ways:

date: run_date only once at some point in time execution (datetime | str)

scheduler.add_job(my_job, 'date', run_date=date(2017, 9, 8), args=[])
scheduler.add_job(my_job, 'date', run_date=datetime(2017, 9, 8, 21, 30, 5), args=[])
scheduler.add_job(my_job, 'date', run_date='2019-6-12 21:30:05', args=[])
# The 'date' trigger and datetime.now() as run_date are implicit
sched.add_job(my_job, args=[[])

interval: from time to time perform a weeks = 0 | days = 0 | hours = 0 | minutes = 0 | seconds = 0,

start_date=None, end_date=None, timezone=None

scheduler.add_job(my_job, 'interval', hours=2)
scheduler.add_job(my_job, 'interval', hours=2, start_date='2017-9-8 21:30:00',
end_date='2019-06-12 21:30:00)

@scheduler.scheduled_job('interval', id='my_job_id', hours=2)
def my_job():

    print("Hello World")

cron:使用Linux下crontab的方式(year=None, month=None, day=None, week=None, day_of_week=None, hour=None, minute=None, second=None, start_date=None, end_date=None, timezone=None)

　 sched.add_job(my_job, 'cron', hour=3, minute=30)

    sched.add_job(my_job, 'cron', day_of_week='mon-fri', hour=5, minute=30, end_date='2017-10-
30')

    @sched.scheduled_job('cron', id='my_job_id', day='last sun')
    def some_decorated_task():

        print("I am printed at 00:00:00 on the last Sunday of every month!")

2, scheduler components

executing scheduler component provides a way to select suitable manner in different operating environments

Mode when running in the process scheduler only: BlockingScheduler

from apscheduler.schedulers.blocking import BlockingScheduler
import time

scheduler = BlockingScheduler()
def job1():

print "% s: mission"% time.asctime ()

scheduler.add_job(job1, 'interval', seconds=3)
scheduler.start()

Mode when not want to use any framework: BackgroundScheduler

from apscheduler.schedulers.background import BackgroundScheduler
import time

scheduler = BackgroundScheduler()
def job1():

print "% s: mission"% time.asctime ()

scheduler.add_job(job1, 'interval', seconds=3)

scheduler.start()

while True:
    pass

AsyncIOScheduler: asyncio module way (Python3)

from apscheduler.schedulers.asyncio import AsyncIOScheduler

try:

    import asyncio

except ImportError:

    import trollius as asyncio
...

...

# while True pass

try:

    asyncio.get_event_loop().run_forever()

except (KeyboardInterrupt, SystemExit):

    pass

GeventScheduler: gevent方式

from apscheduler.schedulers.gevent import GeventScheduler
...

...

g = scheduler.start()

# while True:pass

try:
    g.join()

except (KeyboardInterrupt, SystemExit):
    pass

TornadoScheduler: Tornado方式

from tornado.ioloop import IOLoop
from apscheduler.schedulers.tornado import TornadoScheduler

...

# while True:pass

try:
    IOLoop.instance().start()

except (KeyboardInterrupt, SystemExit):
    pass

TwistedScheduler: Twisted方式

from twisted.internet import reactor
from apscheduler.schedulers.twisted import TwistedScheduler

...

# while True:pass

try:
    reactor.run()

except (KeyboardInterrupt, SystemExit):
    pass

QtScheduler: Qt方式

3、executors组件

executors组件提供任务的调度方式

base

debug
gevent

pool(max_workers=10)

twisted

4、jobstore组件

jobstore提供任务的各种持久化方式

base

memory
mongodb
scheduler.add_jobstore('mongodb', collection='example_jobs')

redis

scheduler.add_jobstore('redis', jobs_key='example.jobs', run_times_key='example.run_times')
rethinkdb
scheduler.add_jobstore('rethinkdb', database='apscheduler_example')

sqlalchemy

  scheduler.add_jobstore('sqlalchemy', url=url)

zookeeper

  scheduler.add_jobstore('zookeeper', path='/example_jobs')

五、任务操作

1、添加任务add_job(如上)

如果使用了任务的存储，开启时最好添加replace_existing=True，否则每次开启时都会创建任务的副本，开启后任务不会马上启动，可修改triger参数

2、删除任务remove_job

#根据任务实例删除

job = scheduler.add_job(myfunc, 'interval', minutes=2)
job.remove()

# 根据任务id删除

scheduler.add_job(myfunc, 'interval', minutes=2, id='my_job_id')
scheduler.remove_job('my_job_id')

3、任务的暂停pause_job和继续resume_job

job = scheduler.add_job(myfunc, 'interval', minutes=2)

#根据任务实例

job.pause()

job.resume()

# 根据任务id暂停

scheduler.add_job(myfunc, 'interval', minutes=2, id='my_job_id')

scheduler.pause_job('my_job_id')

4、任务的修饰modify和重设reschedule_job

修饰：job.modify(max_instances=6, name='Alternate name')

重设：scheduler.reschedule_job('my_job_id', trigger='cron', minute='*/5')

5、调度器操作

开启：scheduler.start()

关闭：scheduler.shotdown(wait=True | False)

暂停：scheduler.pause()

继续：scheduler.resume()

监听：http://apscheduler.readthedocs.io/en/v3.3.0/modules/events.html#module-apscheduler.events

def my_listener(event):
    if event.exception:

        print('The job crashed :(')
    else:

        print('The job worked :)')
scheduler.add_listener(my_listener, EVENT_JOB_EXECUTED | EVENT_JOB_ERROR)

官方实例

from pytz import utc

from apscheduler.schedulers.background import BackgroundScheduler
from apscheduler.jobstores.mongodb import MongoDBJobStore
from apscheduler.jobstores.sqlalchemy import SQLAlchemyJobStore
from apscheduler.executors.pool import ThreadPoolExecutor, ProcessPoolExecutor

jobstores = {
    'mongo': MongoDBJobStore(),
    'default': SQLAlchemyJobStore(url='sqlite:///jobs.sqlite')

}
executors = {

    'default': ThreadPoolExecutor(20),

    'processpool': ProcessPoolExecutor(5)
}

job_defaults = {
    'coalesce': False,
    'max_instances': 3

}
scheduler = BackgroundScheduler(jobstores=jobstores, executors=executors,
job_defaults=job_default

apscheduler regular tasks framework

Guess you like