We may encounter situations that require multitasking in the project, such as sending SMS verification codes, emails, etc. to users, but the process may be blocked due to network problems. At this time, we can start other processes. With Celery, when we use the producer consumer model, we only need to focus on the task itself, which greatly simplifies the programmer’s development process
Celery is simple and reliable, and it can also run on multiple servers. A single celery process can handle millions of tasks per minute
1. Install Celery
We enter the virtual environment to install celery
pip install Celery -i https://pypi.tuna.tsinghua.edu.cn/simple
2. Create instance and configuration
2.1 Add a new task package
Create a new celery_task package in the outer directory of the Django project, and create a main.py main file, config.py configuration file, and task package (the package name here is sms). The structure is as follows
2.2 Configure the database
Here is an example of Redis database
config.py
# 设置中间人，这里是redis三号库
broker_url = "redis://127.0.0.1:6379/3"
# 若是使用rabbitmq
# broker_url = "amqp://用户名:密码@ip地址:5672"
2.3 Setting up tasks
We need to mandate packet (here sms) in a new tasks.py file, the file name is fixed , do not use any other name
Then set up
tasks.py in the tasks.py file
from celery_tasks.main import celery_app # 这是在main.py里定义的对象名
# 使用装饰器装饰要执行的函数名，之后可以使用delay()方法
@celery_app.task(name='ccp_send_sms_code')
def ccp_send_sms_code(mobile, sms_code):
'''该函数就是一个任务, 用于发送短信'''
result = CCP().send_template_sms(mobile,[sms_code, 5],1)
return result
2.4 main file
We need to instantiate the Celery object in the main file, and use the above configuration and specify the task file
from celery import Celery
celery_app = Celery('main')
celery_app.config_from_object('celery_tasks.config')
# 指定任务所在包的位置，它会自动捕捉执行
celery_app.autodiscover_tasks(['celery_tasks.sms'])
2.5 call task
Because we added a decorator to the function, we just need to use the original function name and delay() to run, it will automatically start asynchronous execution (need to open the celery service)
from celery_tasks.sms.tasks import ccp_send_sms_code
# 原来的写法:
# CCP().send_template_sms(mobile, [sms_code, 5], 1)
# 改为现在的写法,注意使用delay()方法
ccp_send_sms_code.delay(mobile, sms_code)
3. Turn on the service
We need to turn on the celery service to use asynchronous
We enter the folder where the project is located (here in the virtual environment), and then use the celery command to start the service
celery -A celery_tasks.main worker -l info
Parameter Description
| parameter | Description |
|---|---|
| -A | The startup file, the main.py created above |
| worker | Started object |
| -l | Log level, such as info |
| –concurrency | Number of processes opened per CPU |
If you want to run as a coroutine, you can download eventlet
# 安装 eventlet 模块
$ pip install eventlet -i https://pypi.tuna.tsinghua.edu.cn/simple
# 启用 Eventlet 池
$ celery -A celery_tasks.main worker -l info -P eventlet -c 1000