Source code interpretation: Flask context and proxy mode

In the previous section, I took a closer look at Python's "context manager" with you. Today, it is still a topic related to the context. It's just that the context here is a bit different from the content of the previous section. The context manager is to manage the context of the code block level, and the context I will talk about today is the context in the engineering project.

Maybe you are still not clear about the concept of context . Here is a brief explanation

The operation of a program or function, in many cases, needs to rely on variables outside the program to run. Once these variables are separated, the program will not work properly. These external variables, in accordance with the conventional practice, are to pass these variables in as function parameters one by one. This is an approach, and there is no problem with simple small programs. Once such an approach is used in a larger project, it is too cumbersome and inflexible.

A better approach is to integrate the frequently used variable values ​​in the global of these projects, and the collection of these values ​​is the context. Just take it from this context when needed.

In Flask, there are two contexts

• application context
• request context

application contextWill store variables in an app that can be shared globally. And request contextstores all the information a request initiated from the outside.

There can be multiple apps in a Flask project, but there will be multiple requests in an app. This is the correspondence between them.

As mentioned above, context can be achieved 信息共享, but at the same time, one thing is very important, that is 信息隔离. When multiple apps are running at the same time, it is necessary to ensure that one app cannot access and change the variables of another app. This is very important.

So exactly how to do 信息隔离it?

Next, we will mention three very common objects in Flask, you should not feel unfamiliar.

• Local
• LocalProxy
• LocalStack

These three objects are werkzeugin the offer, the definition of local.pyinside, so they are not unique to Flask, which means we can use them directly in their own projects, rather than relying on the Flask environment.

1. Local

First Local, remember that "concurrent programming series' fifth chapter of threads in the previous 信息隔离time, the mentioned threading.local, it is designed to store the current variable thread, thread isolation in order to achieve the object.

And the Local in Flask has the same function as this one.

In order to understand the role of this Local, the best way is to directly compare two pieces of code.

First of all, without using Local, we create a new class with a name attribute, which is wangbm by default, and then start a thread. What we do is to change the name attribute to wuyanzu.

import time
import threading

class People:
    name = 'wangbm'

my_obj = People()

def worker():
    my_obj.name = 'wuyanzu'

new_task = threading.Thread(target=worker)
new_task.start()

# 休眠1s，保证子线程先执行
time.sleep(1)

print(my_obj.name)

The execution result can be imagined, it is wuyanzu. Changes made by the child thread to the object can directly affect the main thread.

Next, we use Local to achieve

import time
import threading

from werkzeug.local import Local

class People:
    name = 'wangbm'

my_obj = Local()
my_obj.name = 'wangbm'

def worker():
    my_obj.name = 'wuyanzu'
    print('in subprocess, my_obj.name: '+str(my_obj.name))

new_task = threading.Thread(target=worker)
new_task.start()

# 休眠1s，保证子线程先执行
time.sleep(1)

print('in mainprocess, my_obj.name: '+str(my_obj.name))

The print result is as follows, it can be seen that the modification of the sub-thread will not affect the main thread

in subprocess, my_obj.name: wuyanzu
in mainprocess, my_obj.name: wangbm

So how does Local do it? In fact, the principle is very simple. It uses a basic data structure: a dictionary.

When the thread modifies the variables in the Local object (including the variable name k1 and the variable value v1), it can be seen from the source code that it first obtains the id of the current thread as __storage__the key (this storage is a nested dictionary), and the value, Is a dictionary,{k1: v1}

    def __setattr__(self, name, value):
        ident = self.__ident_func__()
        storage = self.__storage__
        try:
            storage[ident][name] = value
        except KeyError:
            storage[ident] = {name: value}

Examples are as follows

# 0 和 1 是线程 id
self.__storage__['0'][k1] = v1
self.__storage__['1'][k2] = v2

Because of the use of thread id as a layer of encapsulation, thread isolation can be realized.

If you want to use a graph to represent, the first Local object is an empty box

When different threads write data in, the Local object allocates a micro-box for each thread.

local that need to be localmanagermanaged, after the request, calls the localmanager.cleanup()function, in fact, calls local.__release_local__for data cleanup. How to do it, look at the following code.

   from werkzeug.local import Local, LocalManager

   local = Local()
   local_manager = LocalManager([local])

   def application(environ, start_response):
       local.request = request = Request(environ)
       ...

   # make_middleware会确保当request结束时，所有存储于local中的对象的reference被清除
   application = local_manager.make_middleware(application)

The following is the Localcode, there is a need can be seen here directly.

class Local(object):
    __slots__ = ('__storage__', '__ident_func__')

    def __init__(self):
        object.__setattr__(self, '__storage__', {})
        object.__setattr__(self, '__ident_func__', get_ident)

    def __iter__(self):
        return iter(self.__storage__.items())

    def __call__(self, proxy):
        """Create a proxy for a name."""
        return LocalProxy(self, proxy)

    def __release_local__(self):
        self.__storage__.pop(self.__ident_func__(), None)

    def __getattr__(self, name):
        try:
            return self.__storage__[self.__ident_func__()][name]
        except KeyError:
            raise AttributeError(name)

    def __setattr__(self, name, value):
        ident = self.__ident_func__()
        storage = self.__storage__
        try:
            storage[ident][name] = value
        except KeyError:
            storage[ident] = {name: value}

    def __delattr__(self, name):
        try:
            del self.__storage__[self.__ident_func__()][name]
        except KeyError:
            raise AttributeError(name)

2. LocalStack

Through the introduction of Local, we can know that Local actually encapsulates the dictionary to achieve thread isolation.

And the next thing I will introduce LocalStackis the same idea, which LocalStackis encapsulated Local, so it has both the local thread isolation characteristics and the stack structure characteristics. You can access objects through pop, push, and top.

Also use a picture to represent

The characteristic of the stack structure is nothing more than last in, first out. Not to mention here, the focus here is how to reflect the characteristics of thread isolation, or use the above example with a slight modification.

import time
import threading

from werkzeug.local import LocalStack


my_stack = LocalStack()
my_stack.push('wangbm')

def worker():
    print('in subthread, my_stack.top is : '+str(my_stack.top) + ' before push')
    my_stack.push('wuyanzu')
    print('in subthread, my_stack.top is : ' + str(my_stack.top) + ' after push')

new_task = threading.Thread(target=worker)
new_task.start()

# 休眠1s，保证子线程先执行
time.sleep(1)

print('in main thread, my_stack.top is : '+str(my_stack.top))

The output result is as follows. It can be seen that my_stack in the child thread and my_stack in the main thread cannot be shared, and isolation is indeed achieved.

in subthread, my_stack.top is : None before push
in subthread, my_stack.top is : wuyanzu after push
in main thread, my_stack.top is : wangbm

In Flask, there are two main contexts, AppContextand RequestContext.

When a request is initiated, the Flask will first open a thread, then the request contains the context information RequestContextinto one of the LocalStackobject ( _request_ctx_stack), and before pushing, it will in fact to detect another LocalStackobject ( _app_ctx_stack) is empty (but generally _app_ctx_stackWill not be empty), if it is empty, push the context information of the app first _app_ctx_stack, and then push the context information of the request _request_ctx_stack.

There are three commonly used objects in flask

• current_app
• request
• session

These three objects always point to LocalStackthe corresponding app, request or session in the context at the top of the stack. The corresponding source code is as follows:

def _lookup_req_object(name):
    top = _request_ctx_stack.top
    if top is None:
        raise RuntimeError(_request_ctx_err_msg)
    return getattr(top, name)

def _find_app():
    top = _app_ctx_stack.top
    if top is None:
        raise RuntimeError(_app_ctx_err_msg)
    return top.app

_request_ctx_stack = LocalStack()
_app_ctx_stack = LocalStack()
current_app = LocalProxy(_find_app)
request = LocalProxy(partial(_lookup_req_object, 'request'))
session = LocalProxy(partial(_lookup_req_object, 'session'))

3. LocalProxy

Through the above code, you can find that when we access the elements in LocalStack, we all go through LocalProxyIs there?

This is very strange, why not visit Localand LocalStackdo?

This should be a difficult point. Let me give you an example. Maybe you will understand.

The first is the case of not using LocalProxy

# use Local object directly
from werkzeug.local import LocalStack

user_stack = LocalStack()
user_stack.push({'name': 'Bob'})
user_stack.push({'name': 'John'})

def get_user():
    # do something to get User object and return it
    return user_stack.pop()


# 直接调用函数获取user对象
user = get_user()
print user['name']
print user['name']

The output is

John
John

After using LocalProxy

# use LocalProxy
from werkzeug.local import LocalStack, LocalProxy
user_stack = LocalStack()
user_stack.push({'name': 'Bob'})
user_stack.push({'name': 'John'})

def get_user():
    # do something to get User object and return it
    return user_stack.pop()

# 通过LocalProxy使用user对象
user = LocalProxy(get_user)
print user['name']
print user['name']

Output result

John
Bob

How, see the difference, use the LocalStack object directly, once the user is assigned, it can no longer be dynamically updated, while using Proxy, every time the operator is called (here []操作符used to get the attribute), the user will be re-acquired, thus realizing dynamic Update the effect of user .

Every user['name']time will trigger LocalProxyclass __getitem__to invoke the class _get_current_object. And each _get_current_objectwill be returned get_user()(the corresponding function is in flask _lookup_req_object) execution results, i.e.user_stack.pop()

def __init__(self, local, name=None):
    # 【重要】将local对象（也就是一个get_user函数对象）赋值给self.__local
    object.__setattr__(self, '_LocalProxy__local', local)
    object.__setattr__(self, '__name__', name)
    if callable(local) and not hasattr(local, '__release_local__'):
        # "local" is a callable that is not an instance of Local or
        # LocalManager: mark it as a wrapped function.
        object.__setattr__(self, '__wrapped__', local)

def _get_current_object(self):
    """Return the current object.  This is useful if you want the real
    object behind the proxy at a time for performance reasons or because
    you want to pass the object into a different context.
    """
    if not hasattr(self.__local, '__release_local__'):
        # 【重要】执行传递进行的 get_user 对象。
        return self.__local()
    try:
        return getattr(self.__local, self.__name__)
    except AttributeError:
        raise RuntimeError('no object bound to %s' % self.__name__)

In this way, each operation on the top element of the stack is performed in the face of the latest element.

4. Classic mistakes

An error often encountered in Flask is:

Working outside of application context.

This error is difficult to understand if you don't understand flask's context mechanism. Based on the above knowledge background, we can try to understand why this happens.

First, let's simulate the occurrence of this error. Suppose there is now a separate file with the following content

from flask import current_app

app = Flask(__name__)

app = current_app
print(app.config['DEBUG'])

Run it, it will report the following error.

Traceback (most recent call last):
  File "/Users/MING/PycharmProjects/fisher/app/mytest/mytest.py", line 19, in <module>
    print(app.config['DEBUG'])
  File "/Users/MING/.virtualenvs/fisher-gSdA58aK/lib/python3.6/site-packages/werkzeug/local.py", line 347, in __getattr__
    return getattr(self._get_current_object(), name)
  File "/Users/MING/.virtualenvs/fisher-gSdA58aK/lib/python3.6/site-packages/werkzeug/local.py", line 306, in _get_current_object
    return self.__local()
  File "/Users/MING/.virtualenvs/fisher-gSdA58aK/lib/python3.6/site-packages/flask/globals.py", line 51, in _find_app
    raise RuntimeError(_app_ctx_err_msg)
RuntimeError: Working outside of application context.

You must be surprised. I obviously also instantiated an app object, but why would I get an error when taking current_app? And if current_app is not used, no error will be reported.

If you study the above content carefully, it is not difficult to understand here.

From previous research current_app, it is found that when used , it takes the LocalStacktop element of the stack (app context information). In fact, when we app = Flask(__name__)instantiate an app object, this context information has not been written at this time LocalStack. Naturally It will be an error to get the top element of the stack.

def _find_app():
    top = _app_ctx_stack.top
    if top is None:
        raise RuntimeError(_app_ctx_err_msg)
    return top.app

As we said above, when will this context be pushed in? After an external request is initiated, it will first check whether the context information of the app has been pushed in. If not, it will be pushed in first.

The above is the way we run a single file, and no request is actually generated a request, naturally LocalStackthere is no context information in the app. It is normal to report errors.

After knowing the source of the error, how to solve this problem?

In Flask, which provides a method ctx=app.app_context()you can get a context object, as long as we will this context object to manually push LocalStackin, current_appit may be normal to take our app of the objects.

from flask import Flask, current_app

app = Flask(__name__)
ctx = app.app_context()
ctx.push()

app = current_app
print(app.config['DEBUG'])
ctx.pop()

Because the AppContext class implements the context protocol

class AppContext(object):
    def __enter__(self):
        self.push()
        return self

    def __exit__(self, exc_type, exc_value, tb):
        self.pop(exc_value)

        if BROKEN_PYPY_CTXMGR_EXIT and exc_type is not None:
            reraise(exc_type, exc_value, tb)

So you can also write like this

from flask import Flask, current_app

app = Flask(__name__)

with app.app_context():
    app = current_app
    print(app.config['DEBUG'])

Above, I learn through July Flask高级编程with their own straightforward to understand, you want to understand the context in Flask core mechanism would be helpful.