celery 配置broker redis sentinel方式，集成到django

1、prj.py 存储celery的配置信息

# _*_ coding:utf-8 _*_
from redis.sentinel import Sentinel
import time
from celery import Celery
import os
#import django

#os.environ["DJANGO_SETTINGS_MODULE"] = "settings"
#django.setup()

app=Celery("mycelery")     #集群哨兵版
username = "redis" #用户名
password = "m-SDFdfle90IUD&)+U"  #redis授权密码
app.conf.broker_url = 'sentinel://{usr}:{pwd}@127.0.0.1:9111;sentinel://{usr}:{pwd}@127.0.0.1:9222;sentinel://{usr}:{pwd}@127.0.0.1:9333'.format(
    usr = username, 
    pwd = password,
)
app.conf.broker_transport_options = { 'master_name': "mymaster",'visibility_timeout': 43200 }

app.conf.timezone = 'Asia/Shanghai'

2、task.py 写任务模块

# -*-  coding:utf-8 -*-
from prj import app
import sys
import time

@app.task(name="task.add")
def add(x,y):
    time.sleep(1)
    print(x,"add",y , x + y)
    return x+y

if __name__ == "__main__":
    print(sys.argv)
    x,y = sys.argv[1:]
    add.delay(int(x) ,int(y))

3、开启3个worker 进程

celery -A task  worker -l info -n node1
celery -A task  worker -l info -n node2
celery -A task  worker -l info -n node3

可以开启多个

4、测试执行任务

python task.py 1 4

5、查看worker进程日志

[2020-09-01 18:36:54,068: WARNING/ForkPoolWorker-1] (1, 'add', 2, 3)
[2020-09-01 18:36:54,069: INFO/ForkPoolWorker-1] Task task.add[be792083-479c-4c41-b043-9f3113cfce45] succeeded in 1.00177153759s: 3
[2020-09-01 18:36:54,709: WARNING/ForkPoolWorker-2] (1, 'add', 2, 3)
[2020-09-01 18:36:54,710: INFO/ForkPoolWorker-2] Task task.add[e5c96702-95d1-411c-b0e4-0fe077262522] succeeded in 1.00171243958s: 3
[2020-09-01 18:36:55,862: INFO/MainProcess] Received task: task.add[ba90222a-c6c5-4e28-ad44-3bca8daac04d]  
[2020-09-01 18:36:56,864: WARNING/ForkPoolWorker-1] (1, 'add', 2, 3)
[2020-09-01 18:36:56,865: INFO/ForkPoolWorker-1] Task task.add[ba90222a-c6c5-4e28-ad44-3bca8daac04d] succeeded in 1.00138912722s: 3
[2020-09-01 18:37:01,681: INFO/MainProcess] Received task: task.add[2c6b3a9f-fda0-4a96-aaa6-731183b87bd2]  
[2020-09-01 18:37:02,696: WARNING/ForkPoolWorker-1] (1, 'add', 2, 3)
[2020-09-01 18:37:02,696: INFO/ForkPoolWorker-1] Task task.add[2c6b3a9f-fda0-4a96-aaa6-731183b87bd2] succeeded in 1.00158853084s: 3
[2020-09-01 18:37:04,633: INFO/MainProcess] Received task: task.add[124d1f17-4e44-4b36-8abb-a1572038e8c7]  
[2020-09-01 18:37:05,590: INFO/MainProcess] Received task: task.add[7d2eeedf-e651-4213-84f5-2c845a02110d]  
[2020-09-01 18:37:05,636: WARNING/ForkPoolWorker-1] (1, 'add', 2, 3)
[2020-09-01 18:37:05,636: INFO/ForkPoolWorker-1] Task task.add[124d1f17-4e44-4b36-8abb-a1572038e8c7] succeeded in 1.0012263488s: 3
[2020-09-01 18:37:06,592: WARNING/ForkPoolWorker-2] (1, 'add', 2, 3)
[2020-09-01 18:37:06,593: INFO/ForkPoolWorker-2] Task task.add[7d2eeedf-e651-4213-84f5-2c845a02110d] succeeded in 1.0006608218s: 3
[2020-09-01 19:25:23,738: INFO/MainProcess] Received task: task.add[c388a795-86e5-4192-a876-fa97f3312d90]  
[2020-09-01 19:25:24,741: WARNING/ForkPoolWorker-1] (1, 'add', 4, 5)
[2020-09-01 19:25:24,742: INFO/ForkPoolWorker-1] Task task.add[c388a795-86e5-4192-a876-fa97f3312d90] succeeded in 1.00158562697s: 5
[2020-09-01 19:25:52,725: INFO/MainProcess] sync with celery@node3

6、celery集成到django参考

https://docs.celeryproject.org/en/stable/django/first-steps-with-django.html#using-celery-with-django

celery 配置broker redis sentinel方式，集成到django

猜你喜欢