A Graceful End - Python Flask/Gunicorn后端退出钩子

基本想法

可以把注销监听、注销服务写在flask exit hook里

Flask简单场景

1. Flask没有app.stop()方法

正常退出

1. Python有内置的atexit库

“The atexit module defines functions to register and unregister cleanup functions. Functions thus registered are automatically executed upon normal interpreter termination. atexit runs these functions in the reverse order in which they were registered; if you register A, B, and C, at interpreter termination time, they will be run in the order C, B, A.”

指令退出CTRL+C/kill

当使用ctrl+C方式关闭服务器时可以用另一个库叫signal

• 关于：Python信号处理模块signal该怎么使用

• SIGNAL模块-官方文档

题外话：可以通过注册signal.signal的形式处理一些事件，但是signal.STOP, signal.SIGKILL是没有办法拦截的（这种杀死是在内核级别实现的）。

# wsgi.py
import signal

def exit_hook(signalNumber, frame):
    print("receiving", signalNumber)
    
if __name__ == "__main__":
    signal.signal(signal.SIGINT, exit_hook)
    app.run()
    
# 可以在结束时输出
# receiving 2

# 但是如果注册signal.SIGKILL/signal.SIGSTOP则代码会报错

回答原文如下：

There is no way to handle SIGKILL

The signals SIGKILL and SIGSTOP cannot be caught, blocked, or ignored.

If you're looking to handle system shutdown gracefully, you should be handling SIGTERM or a startup/shutdown script such as an upstart job or a init.d script.

---

如何不通过Ctrl+C来关闭Flask

from flask import request
def shutdown_server():
    func = request.environ.get('werkzeug.server.shutdown')
    if func is None:
        raise RuntimeError('Not running with the Werkzeug Server')
    func()
    
@app.get('/shutdown')def shutdown():
    shutdown_server()
    return 'Server shutting down...'
    
    
---------
from multiprocessing import Process

server = Process(target=app.run)
server.start()
# ...
server.terminate()
server.join()

但其实第一种方法已经过时了....

参考文章：How To Create Exit Handlers for Your Python App

---

Gunicorn

Flask的话直接用这个可以，那Gunicorn等多线程、多进程场景该怎么办？

Gunicorn的场景下，下面都不适用

参考：how-to-configure-hooks-in-python-gunicorn

总结：gunicorn场景，可以使用configs.py文件，将configs全部写在里面，包括各种hook（Hook也算是config的一部分），但是官方文档关于hook的部份没有给出任何具体的解释，需要去查源代码。

样例：

# gunicorn_hooks_config.py
def on_starting(server):
    """
    Do something on server start
    """
    print("Server has started")


def on_reload(server):
    """
     Do something on reload
    """
    print("Server has reloaded")


def post_worker_init(worker):
    """
    Do something on worker initialization
    """
    print("Worker has been initialized. Worker Process id –>", worker.pid)

运行文件：

# wsgi.py
from app import app

if __name__ == '__main__':
    app.run()

运行

gunicorn -c gunicorn_hooks_config.py wsgi:app

但是官方关于这一块钩子的文档实在太简单了，啥都没。没办法，想知道怎么写函数，必须从源码开始查：

入口Gunicorn代码如下：

# gunicorn

#!/Users/xiao/opt/anaconda3/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from gunicorn.app.wsgiapp import run
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(run())

# wsgi.app.wsgiapp.py
def run():
    """\
    The ``gunicorn`` command line runner for launching Gunicorn with
    generic WSGI applications.
    """
    from gunicorn.app.wsgiapp import WSGIApplication
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
    
if __name__ == '__main__':
    run()

关于加载配置文件

# app.base.py

...

class Application(BaseApplication):
    ...
    
    def get_config_from_filename(self, filename):

        if not os.path.exists(filename):
            raise RuntimeError("%r doesn't exist" % filename)

        ext = os.path.splitext(filename)[1]

        try:
            module_name = '__config__'
            if ext in [".py", ".pyc"]:
                spec = importlib.util.spec_from_file_location(module_name, filename)
            else:
                msg = "configuration file should have a valid Python extension.\n"
                util.warn(msg)
                loader_ = importlib.machinery.SourceFileLoader(module_name, filename)
                spec = importlib.util.spec_from_file_location(module_name, filename, loader=loader_)
            mod = importlib.util.module_from_spec(spec)
            sys.modules[module_name] = mod
            spec.loader.exec_module(mod)
        except Exception:
            print("Failed to read config file: %s" % filename, file=sys.stderr)
            traceback.print_exc()
            sys.stderr.flush()
            sys.exit(1)

        return vars(mod)

在Arbiter.py中会有Arbiter类来负责生成和杀死worker，可以说Arbiter是Gunicorn内部的Worker大管家。但是仍然，worker类是直接调取的self.cfg.worker_class

Gunicorn这个框架里使用了paste.deploy.loadapp，因此这个也很重要。

具体流程：

wsgiapplication运行 -> 检测到configs.py -> importlib.util.spec_from_file_location 以__config__的module返回 -> 该module成为app.cfg -> 传递给Arbiter的cfg -> Worker类注册signal_handler -> 方法里调用self.cfg.worker_abort(self) -> 即 同名方法

# gunicorn/workers/base.py
class Worker(object):
    def init_signals(self):
        # reset signaling
        for s in self.SIGNALS:
            signal.signal(s, signal.SIG_DFL)
        # init new signaling
        signal.signal(signal.SIGQUIT, self.handle_quit)
        signal.signal(signal.SIGTERM, self.handle_exit)
        signal.signal(signal.SIGINT, self.handle_quit)
        signal.signal(signal.SIGWINCH, self.handle_winch)
        signal.signal(signal.SIGUSR1, self.handle_usr1)
        signal.signal(signal.SIGABRT, self.handle_abort)

        # Don't let SIGTERM and SIGUSR1 disturb active requests
        # by interrupting system calls
        signal.siginterrupt(signal.SIGTERM, False)
        signal.siginterrupt(signal.SIGUSR1, False)

        if hasattr(signal, 'set_wakeup_fd'):
            signal.set_wakeup_fd(self.PIPE[1])
    def handle_abort(self, sig, frame):
        self.alive = False
        self.cfg.worker_abort(self)
        sys.exit(1)

也就是说，configs.py里写的def worker_abort(worker)方法，必须是接收一个Worker类参数的方法。

那server呢？有了原名称调用这个思路，去找就容易了，直接搜on_exit，发现server即Arbiter类。调用场景如下348行：

class Arbiter(Object):
    def halt(self, reason=None, exit_status=0):
        """ halt arbiter """
        self.stop()
        self.log.info("Shutting down: %s", self.master_name)
        if reason is not None:
            self.log.info("Reason: %s", reason)
        if self.pidfile is not None:
            self.pidfile.unlink()
        self.cfg.on_exit(self)
        sys.exit(exit_status)

总结：

这些文档正确的写法应该是

# 注意此处lib的拼写错误是代码中的，不能自己改成init
def worker_int(worker: gunicorn.workers.Worker) -> None:
    ...

# 可以有返回值，但是返回值不会被调用
    
def on_exit(server: gunicorn.Arbiter) -> None:
    pass

似乎在Gunicorn中，Arbiter名称被隐藏，重命名为SERVER了（不理解，迷惑行为）