A brief description of the difference between WSGI, uwsgi and uWSGI and the deployment of python web based on uWSGI and gunicorn

introduction

Recently, a back-end project is being developed based on the Flask Web framework. During the interaction between Web Server and Flask applications, you will always encounter the concepts of WSGI, uwsgi and uWSGI mentioned in the topic of this article, which are organized as follows.

WSGI

English full name: Web Server Gateway Interface, Web service network management interface, in simple terms, it is a communication specification between a Web server and an application.

uwsgi

uwsgi is a communication protocol, but it belongs to two things with WSGI, which is faster under this protocol.

uWSGI

uWSGI is a Web Server, and exclusive uwsgi protocol, but also supports WSGI protocol, HTTP protocol, etc., its function is to convert the HTTP protocol into a language-supported network protocol for python to use.

Although Flask is very easy to use, and its built-in app.run(host="0.0.0.0", port=7001)is very convenient for debugging, but it is lacking in both high concurrency and robustness when used in a production environment. Generally, it will cooperate with the WGSI container for [production environment deployment]

Use of uWSGI

When working on a Django project, we generally use Django's embedded Web Server directly for test development. However, if the project is to be used in a production environment and considering concurrency and other performance, we may need uwsgi and nginx. The following only explains the common usage of uwsgi. As for the configuration of nginx, I am going to write a blog post specifically for the follow-up.

1. Installation

pip install uwsgi

2. Configuration

There are generally two ways to execute uwsgi: command line and file configuration, but the command line may need to remember many parameters, so it is more common to use file configuration. The file format supports many types such as ini, xml, yaml, etc. The author recommends Using relatively simple key-value form ini mode, a simple uwsgi ini configuration example is given below:

[uwsgi]
socket = 127.0.0.1:8001
master = false
chdir = /var/www/cmpvirtmgr/
module = cmpvirtmgr.wsgi
home = /var/www/env
workers = 2
reload-mercy = 10
vacuum = true
max-requests = 1000
limit-as = 512
buffer-size = 30000
pidfile = /etc/uwsgi/uwsgi.pid

Execution: uwsgi --ini /path/to/uwsgi.ini

Parameter explanation:

socket: socket file, it can also be address + port;
master: whether to start the main process to manage other processes;
chdir: the root directory of the project;
module: relative path of wsgi file;
home: virtual environment directory;
workers: the number of open processes;
reload-mercy: Set the maximum number of seconds to wait for the end of the work in a working sub-process in a smooth restart (restart until the received request is processed);
vacuum: delete the corresponding socket and pid files when the service ends;
max_requests: The upper limit of requests set by each worker process;
limit_as: Limit the number of virtual memory occupied by each uwsgi process;
buffer_size: Set the size of the internal buffer area used for uwsgi package parsing;
pid_file: Specify the pid file;
harakiri: the timeout period of the request;
daemonize: The process executes in the background and saves the log to a specific path; if the uwsgi process is managed by the supervisor, this parameter cannot be set;

For more uwsgi parameters, please refer to the official document: https://uwsgi-docs.readthedocs.io/en/latest/

Use gunicorn to deploy flask project

1. WSGI protocol

The web framework is dedicated to how to generate HTML code or generate API interface data based on Restful, and the web server is used to process and respond to HTTP requests. The communication between the web framework and the web server requires a set of interface protocols that both parties abide by. The WSGI protocol is used to unify the two interfaces.

2, WSGI container

Commonly used WSGI containers are Gunicorn and uWSGI, but Gunicorn is started directly with commands without writing configuration files, which is much easier than uWSGI, so I also choose Gunicorn as the container here.

3. Introduction to gunicorn

Gunicorn is a python Wsgi http server, which only supports running on Unix systems, derived from Ruby's unicorn project. Gunicorn uses the prefork master-worker model (in gunicorn, the master is called arbiter), which can work with various wsgi web frameworks.

4. Gunicorn installation

The installation of gunicorn is very simple, just use the command pip install gunicorn. Generally use it, mainly to use its asynchronous worker model, but also need to install the corresponding asynchronous module.

$ pip install gunicorn 
$ pip install greenlet # 使用异步必须安装
$ pip install eventlet # 使用eventlet workers
$ pip install gevent   # 使用gevent workers

5. Use of gunicorn

Here is an example of using gunicorn to deploy a flask project. The use of the flask framework here is not too much to elaborate, and it is not the focus of this article.

The following example, save as app.py

from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello World!"

The parameters usually used by gunicorn are as follows:

-c CONFIG, --config=CONFIG
# 设定配置文件。
-b BIND, --bind=BIND
# 设定服务需要绑定的端口。建议使用HOST:PORT。
-w WORKERS, --workers=WORKERS
# 设置工作进程数。建议服务器每一个核心可以设置2-4个。
-k MODULE
# 选定异步工作方式使用的模块。

Enter your startup configuration in the shell, such as:

$ gunicorn -w 3 -b 127.0.0.1:8080 app:app
# 此处app:app中，第一个app为flask项目实例所在的启动模块，第二个app为生成的flask项目实例

The server can be started when it runs normally.

6. Binding ports

Linux usually prohibits the use of ports below 1024, unless it is root user authority. Many people tried to bind it to port 80 or 443 when using gunicorn and found it to be invalid. If you want to bind to these ports, there are several common methods as follows:

Use Nginx proxy forwarding.
sudo starts gunicorn.
Install additional programs.

7. End the gunicorn service process

Use the ps -ef | grep gunicorn command to find out all the processes of gunicorn.

[root@VM_0_12_centos ~]# ps -ef | grep gunicorn
root     16843 23035  0 Oct14 ?        00:00:02 /root/Envs/myflask/bin/python3.6 /root/Envs/myflask/bin/gunicorn -w 3 -b 172.17.0.12:80 app:app
root     22445 23035  0 Oct04 ?        00:00:15 /root/Envs/myflask/bin/python3.6 /root/Envs/myflask/bin/gunicorn -w 3 -b 172.17.0.12:80 app:app
root     22581 23035  0 Oct11 ?        00:00:05 /root/Envs/myflask/bin/python3.6 /root/Envs/myflask/bin/gunicorn -w 3 -b 172.17.0.12:80 app:app
root     23035     1  0 Sep27 ?        00:04:11 /root/Envs/myflask/bin/python3.6 /root/Envs/myflask/bin/gunicorn -w 3 -b 172.17.0.12:80 app:app

Then use the kill -9 process ID command to kill the process. Note that we can find the main process and kill it, and the child process will end. In the above example, the main process ID is 23035.

[root@VM_0_12_centos ~]# kill -9 23035
[root@VM_0_12_centos ~]# ps -ef | grep gunicorn

After killing the process, wait a few seconds, and then use ps -ef | grep gunicorn to check and find that all the gunicorn service processes have been killed.