Flask source code articles: Flask routing rules and request matching process

Source series:

Flask source code articles: wsgi, Werkzeug and Flask startup workflow

Flask source code article: 2w words thoroughly understand Flask is the context principle

If you don't want to read the specific analysis process, you can read the summary directly, and you can understand it!

1 Routing-related operations at startup

The so-called routing principle is how Flask creates its own routing system, and when a request comes, how to accurately locate the processing function according to the routing system and respond to the request.

This section uses the simplest Flask application example to explain the routing principle, as follows:

from flask import Flask

app = Flask(__name__)


@app.route('/')
def hello_world():
    return 'Hello World!'


if __name__ == '__main__':
    app.run()

(1) Analyze app.route()

route()First, the registration of the route is realized through the decorator under the scaffold (Flask inherits the scaffold) package , and its source code is as follows:

def route(self, rule: str, **options: t.Any) -> t.Callable:
    def decorator(f: t.Callable) -> t.Callable:
      	# 获取endpoint
        endpoint = options.pop("endpoint", None)
        # 添加路由，rule就是app.route()传来的路由字符串，及'/'
        self.add_url_rule(rule, endpoint, f, **options)
        return f

    return decorator

You can see that this decorator mainly does two things: 1. Obtain endpoint; 2. Add routing.

The most important of these is the function add_url_rule(), which is used to add route mappings.

Replenish:

1. The endpoint is used later when Flask stores routing and function name mappings. If not specified, the default is the decorated function name. How to use it will be analyzed later;
2. Because the essence of app.route() is still the add_url_rule() function, we can also use this function directly. For usage, please refer to the article Flask routing .

(2) Analyze add_url_rule()

Let's take a look at what add_url_rule does. Its core source code is as follows:

class Flask(Scaffold):
    # 这里只有要讨论的主要代码，其他代码省略了
    
    url_rule_class = Rule
    url_map_class = Map
    
    def __init__(
        self,
        import_name: str,
        static_url_path: t.Optional[str] = None,
        static_folder: t.Optional[t.Union[str, os.PathLike]] = "static",
        static_host: t.Optional[str] = None,
        host_matching: bool = False,
        subdomain_matching: bool = False,
        template_folder: t.Optional[str] = "templates",
        instance_path: t.Optional[str] = None,
        instance_relative_config: bool = False,
        root_path: t.Optional[str] = None,
    ):
        super().__init__(
            import_name=import_name,
            static_folder=static_folder,
            static_url_path=static_url_path,
            template_folder=template_folder,
            root_path=root_path,
        )
        
        self.url_map = self.url_map_class()
        self.url_map.host_matching = host_matching
        self.subdomain_matching = subdomain_matching
  
    def add_url_rule(
        self,
        rule: str,
        endpoint: t.Optional[str] = None,
        view_func: t.Optional[t.Callable] = None,
        provide_automatic_options: t.Optional[bool] = None,
        **options: t.Any,
     ) -> None:
         # 1.如果没提供endpoint，获取默认的endpoint
         if endpoint is None:
             endpoint = _endpoint_from_view_func(view_func)  # type: ignore
         options["endpoint"] = endpoint

          # 2.获取请求方法，在装饰器@app.route('/index', methods=['POST','GET'])的method参数
          # 如果没有指定，则给个默认的元组("GET",)
          # 关于provide_automatic_options处理一些暂不看
          methods = options.pop("methods", None)
          if methods is None:
              methods = getattr(view_func, "methods", None) or ("GET",)
          if isinstance(methods, str):
              raise TypeError(
                  "Allowed methods must be a list of strings, for"
                  ' example: @app.route(..., methods=["POST"])'
              )
          methods = {
    
    item.upper() for item in methods}

          # Methods that should always be added
          required_methods = set(getattr(view_func, "required_methods", ()))

          # starting with Flask 0.8 the view_func object can disable and
          # force-enable the automatic options handling.
          if provide_automatic_options is None:
              provide_automatic_options = getattr(
                  view_func, "provide_automatic_options", None
              )

          if provide_automatic_options is None:
              if "OPTIONS" not in methods:
                  provide_automatic_options = True
                  required_methods.add("OPTIONS")
              else:
                  provide_automatic_options = False

          # Add the required methods now.
          methods |= required_methods

          # 3.重要的一步：url_rule_class方法实例化Rule对象
          rule = self.url_rule_class(rule, methods=methods, **options)
          rule.provide_automatic_options = provide_automatic_options  # type: ignore
					
          # 4.重要的一步：url_map（Map对象）的add方法
          self.url_map.add(rule)
          
          # 5.判断endpoint和view_func的映射存不存在，如果已经有其他view_func用了这个endpoint，则报错，否则新的映射加到self.view_functions里
          # self.view_functions继承自Scaffold，是一个字典对象
          if view_func is not None:
              old_func = self.view_functions.get(endpoint)
              if old_func is not None and old_func != view_func:
                  raise AssertionError(
                      "View function mapping is overwriting an existing"
                      f" endpoint function: {
      
      endpoint}"
                  )
              self.view_functions[endpoint] = view_func

Analyzing the above source code, this method mainly does the following things:

1. If not provided endpoint, get the default endpoint;
2. @app.route('/index', methods=['POST','GET'])Get the request method, in the parameters of the decorator method, if not specified, give a default tuple ("GET",);
3. self.url_rule_class(): Instantiate Rulethe object, which will be explained later in the Rule class;
4. Call self.url_map.add()method, where self.url_map is an Mapobject, which will be explained later;
5. self.view_functions[endpoint] = view_funcAdd the mapping of endpoint and view_func.

Among them, instantiating Ruleobjects and sums self.url_map.add()is the core of Falsk routing. The following analyzes Ruleclasses and Mapclasses.

(3) Analyze the Rule class

RuleThe class is werkzeug.routingunder the module, and its source code is more. Here we only extract the main code we use, as follows:

class Rule(RuleFactory):
  	def __init__(
        self,
        string: str,
        methods: t.Optional[t.Iterable[str]] = None,
        endpoint: t.Optional[str] = None,
        # 此处省略了其他参数的代码 ...
    ) -> None:
        if not string.startswith("/"):
            raise ValueError("urls must start with a leading slash")
        self.rule = string
        self.is_leaf = not string.endswith("/")
        self.map: "Map" = None  # type: ignore
        self.methods = methods
        self.endpoint: str = endpoint 
				# 省略了其他初始化的代码
        
    def get_rules(self, map: "Map") -> t.Iterator["Rule"]:
        """获取map对象的rule迭代器"""
        yield self
    
    def bind(self, map: "Map", rebind: bool = False) -> None:
        """把map对象绑定到Rule对象上，并且根据rule和map信息创建一个path正则表达式，存储在rule对象的self._regex属性里，路由匹配的时候用"""
        if self.map is not None and not rebind:
             raise RuntimeError(f"url rule {
      
      self!r} already bound to map {
      
      self.map!r}")
        # 把map对象绑定到Rule对象上
        self.map = map
        if self.strict_slashes is None:
            self.strict_slashes = map.strict_slashes
        if self.merge_slashes is None:
            self.merge_slashes = map.merge_slashes
        if self.subdomain is None:
            self.subdomain = map.default_subdomain
        # 调用compile方法创建一个正则表达式
        self.compile()
     
    def compile(self) -> None:
        """编写正则表达式并存储到属性self._regex中"""
				# 此处省略了正则的解析过程代码
        regex = f"^{
      
      ''.join(regex_parts)}{
      
      tail}$"
        self._regex = re.compile(regex)
        
    def match(
        self, path: str, method: t.Optional[str] = None
    		) -> t.Optional[t.MutableMapping[str, t.Any]]:
        """这个函数用于校验传进来path参数（路由）是否能够匹配，匹配不上返回None"""
        # 省去了部分代码，只摘录了主要代码，看一下大致逻辑即可
        if not self.build_only:
            require_redirect = False
					  # 1.根据bind后的正则结果（self._regex正则）去找path的结果集
            m = self._regex.search(path)
            if m is not None:
                groups = m.groupdict()
            
            # 2.编辑匹配到的结果集，加到一个result字典里并返回
            result = {
    
    }
            for name, value in groups.items():
                try:
                  	value = self._converters[name].to_python(value)
                except ValidationError:
                  	return None
                result[str(name)] = value
                if self.defaults:
                  	result.update(self.defaults)
            return result
        return None    

RuleThe class inherits from RuleFactorythe class, the main parameters are:

• string: route string
• methods: route method
• endpoint:endpoint parameter

A Rule instance represents a URL pattern, and a WSGI application will process many different URL patterns, and at the same time generate many Rule instances, and these instances will be passed as parameters to the Map class.

(4) Analyze the Map class

MapThe class is also under the werkzeug.routing module, and there are many source codes. Here we only extract the main codes we use. The main source codes are as follows:

class Map:
  	def __init__(
        self,
        rules: t.Optional[t.Iterable[RuleFactory]] = None
        # 此处省略了其他参数
    ) -> None:
      	# 根据传进来的rules参数维护了一个私有变量self._rules列表
        self._rules: t.List[Rule] = []
        # endpoint和rule的映射
        self._rules_by_endpoint: t.Dict[str, t.List[Rule]] = {
    
    }
        # 此处省略了其他初始化操作
        
    def add(self, rulefactory: RuleFactory) -> None:
        """
        把Rule对象或一个RuleFactory对象添加到map并且绑定到map，要求rule没被绑定过
        """
        for rule in rulefactory.get_rules(self):
            # 调用rule对象的bind方法
            rule.bind(self)
            # 把rule对象添加到self._rules列表里
            self._rules.append(rule)
            # 把endpoint和rule的映射加到属性self._rules_by_endpoint里
            self._rules_by_endpoint.setdefault(rule.endpoint, []).append(rule)
        self._remap = True
        
    def bind(
        self,
        server_name: str,
        script_name: t.Optional[str] = None,
        subdomain: t.Optional[str] = None,
        url_scheme: str = "http",
        default_method: str = "GET",
        path_info: t.Optional[str] = None,
        query_args: t.Optional[t.Union[t.Mapping[str, t.Any], str]] = None,
    ) -> "MapAdapter":
        """
        返回一个新的类MapAdapter
        """
        server_name = server_name.lower()
        if self.host_matching:
            if subdomain is not None:
                raise RuntimeError("host matching enabled and a subdomain was provided")
        elif subdomain is None:
            subdomain = self.default_subdomain
        if script_name is None:
            script_name = "/"
        if path_info is None:
            path_info = "/"

        try:
            server_name = _encode_idna(server_name)  # type: ignore
        except UnicodeError as e:
            raise BadHost() from e

        return MapAdapter(
            self,
            server_name,
            script_name,
            subdomain,
            url_scheme,
            path_info,
            default_method,
            query_args,
        )

MapClasses have two very important attributes:

1. self._rules, the attribute is a list that stores a series of Rule objects;
2. self._rules_by_endpoint：

There is a core method add(), here is our analysis app.add_url_rule()method is the method called in step 4. It will be explained in detail later.

(5) Analyze the MapAdapter class

In Mapthe class, MapAdapterthe class will be used. Let's get to know this class:

class MapAdapter:

    """`Map.bind`或`Map.bind_to_environ` 会返回这个类
    主要用来做匹配
    """

    def match(
        self,
        path_info: t.Optional[str] = None,
        method: t.Optional[str] = None,
        return_rule: bool = False,
        query_args: t.Optional[t.Union[t.Mapping[str, t.Any], str]] = None,
        websocket: t.Optional[bool] = None,
    ) -> t.Tuple[t.Union[str, Rule], t.Mapping[str, t.Any]]:
        """匹配请求的路由和Rule对象"""
				# 只摘摘录了主要代码，省略了大量代码...
				
        # 这里是主要步骤：遍历map对象的rule列表，依次和path进行匹配
        for rule in self.map._rules:
            try:
                # 调用rule对象的match方法返回匹配结果
                rv = rule.match(path, method)
            except RequestPath as e:
						 # 下面省略了大量代码...
         
         # 返回rule对象（或endpoint）和匹配的路由结果
         if return_rule:
            return rule, rv
         else:
             return rule.endpoint, rv

Map.bindor Map.bind_to_environmethod returns MapAdapteran object.

MapAdapterThe core method of the object is that the matchmain step is to traverse the rule list of the map object and match it with the path in turn. Of course, the match method of the rule object is called to return the matching result.

MapNext, you can look at how classes and objects are combined to use a little independently Rlue. See the following example:

from werkzeug.routing import Map, Rule

m = Map([
    Rule('/', endpoint='index'),
    Rule('/blog', endpoint='blog/index'),
    Rule('/blog/<int:id>', endpoint='blog/detail')
])

# 返回一个MapAdapter对象
map_adapter = m.bind("example.com", "/")

# MapAdapter对象的 match方法会返回匹配的结果
print(map_adapter.match("/", "GET"))
# ('index', {})

print(map_adapter.match("/blog/42"))
# ('blog/detail', {'id': 42})

print(map_adapter.match("/blog"))
# ('blog/index', {})

It can be seen that Mapthe object bindreturns an MapAdapterobject, and the method MapAdapterof the object matchcan find the matching result of the route.

(6) Analyze url_rule_class()

add_url_ruleThe first major step is rule = self.url_rule_class(rule, methods=methods, **options)to create an Ruleobject.

When analyzing Rluethe class, I know that the Rule object mainly has string(routing string), methods, and endpoint3 attributes. Take a concrete example below to see what the instantiated Ruleobject looks like.

Still the top example at the beginning, let’s look at @app.route('/')the specific properties of the object when the Rule object is instantiated after passing the code, as follows through debugging:

It can be seen that rulethe attribute of the rule object is the passed route, endpointthe attribute is obtained through the function name, methodthe attribute is the supported request method, and 'HEAD' OPTIONSis added by default.

(7) Analysis map.add(rule)

add_url_ruleThe second major step is self.url_map.add(rule)to call Mapthe object's add method.

When analyzing the Map object in step 4, it was mentioned, now let's go back and take a closer look at what this method does:

def add(self, rulefactory: RuleFactory) -> None:
        """
        把Rule对象或一个RuleFactory对象添加到map并且绑定到map，要求rule没被绑定过
        """
        for rule in rulefactory.get_rules(self):
            # 调用rule对象的bind方法
            rule.bind(self)
            # 把rule对象添加到self._rules列表里
            self._rules.append(rule)
            # 把endpoint和rule的映射加到属性self._rules_by_endpoint里
            self._rules_by_endpoint.setdefault(rule.endpoint, []).append(rule)
        self._remap = True

In fact, the main thing is to add the Rule object or a RuleFactory object instantiated in the previous step to the Map object and bind it to the map.

Mainly did two things:

1. Call the bind method of the rule object rule.bind(self): When analyzing the Rule class, this method was mentioned. Its main function is to bind the map object to the Rule object, and create a regular expression (the property of the Rule object) based on the rule and map information self._regex . .
2. Add the rule object to the list Mapof objects self._rules;
3. Add the mapping of endpointand to the object's properties (a dictionary);ruleMapself._rules_by_endpoint

We can use an example to see what the Map object becomes after adding. Through debugging, the results are as follows:

It can be seen that Mapboth the self._rulesand self._rules_by_endpointattributes contain the data corresponding to the newly added '/' route (the /static/<path:filename>route is the location route of the static file added by default).

The above analysis is over, how to add a routing map.

2 The route matching process when the request comes in

(1) Analyze wsgi_app

Let's analyze how to match the route according to the previous request Mapand the object when the previous request comes in .Rlue

In the previous chapter, we analyzed __call__the method of invoking the Flask app after processing the request through the wsgi server when the request comes in. The code is as follows:

def __call__(self, environ: dict, start_response: t.Callable) -> t.Any:
    """The WSGI server calls the Flask application object as the
    WSGI application. This calls :meth:`wsgi_app`, which can be
    wrapped to apply middleware.
    """
    return self.wsgi_app(environ, start_response)

You can see that the method of the app is actually called wsgi_app(). Its code is as follows:

def wsgi_app(self, environ: dict, start_response: t.Callable) -> t.Any:
  	# 1.获取请求上下文
    ctx = self.request_context(environ)
    error: t.Optional[BaseException] = None
    try:
        try:
          	# 2.调用请求上下文的push方法
            ctx.push()
            # 3.调用full_dispatch_request()分发请求，获取响应结果
            response = self.full_dispatch_request()
        except Exception as e:
            error = e
            response = self.handle_exception(e)
        except: 
            error = sys.exc_info()[1]
            raise
        return response(environ, start_response)
    finally:
        if self.should_ignore_error(error):
            error = None
        ctx.auto_pop(error)

There are 3 main steps in it:

1. Get request context:ctx = self.request_context(environ)
2. Call the push method of the request context:ctx.push()
3. Call full_dispatch_request() to distribute the request and get the response result:response = self.full_dispatch_request()

Let's analyze the function of each step one by one.

(2) Analyze request_context

wsgi_appThe first major step of the method.

This method is mainly to obtain a context object.

Need to pass environ(environment variables, etc.) into the method. Context is also an important concept in Flask. Of course, the next chapter will focus on the analysis of context. This chapter only focuses on what we need.

def request_context(self, environ: dict) -> RequestContext:
    return RequestContext(self, environ)

This method is to create an RequestContextobject. RequestContextThe source code of the class part is as follows:

class RequestContext:
    def __init__(
        self,
        app: "Flask",
        environ: dict,
        request: t.Optional["Request"] = None,
        session: t.Optional["SessionMixin"] = None,
    ) -> None:
        self.app = app
        if request is None:
            request = app.request_class(environ)
        self.request = request
        self.url_adapter = None
        try:
          	# 此处是重点，调用了Falsk对象的create_url_adapter方法获取了MapAdapter对象
            self.url_adapter = app.create_url_adapter(self.request)
        except HTTPException as e:
            self.request.routing_exception = e
        self.flashes = None
        self.session = session
				# 其他代码省略...
    # 其他方法的源码省略...

In the initialization method of creating RequestContextan object, a very important step is to obtain MapAdapterthe object .

We have analyzed its function in Section 4 above, and it is mainly used to match routes.

Let's look at create_url_adapterthe source code:

def create_url_adapter(
    self, request: t.Optional[Request]
) -> t.Optional[MapAdapter]:
    if request is not None:
        if not self.subdomain_matching:
            subdomain = self.url_map.default_subdomain or None
        else:
            subdomain = None
				# 此处是重点，调用了Map对象的bind_to_environ方法
        return self.url_map.bind_to_environ(
            request.environ,
            server_name=self.config["SERVER_NAME"],
            subdomain=subdomain,
        )
    if self.config["SERVER_NAME"] is not None:
      	# 此处是重点，调用了Map对象的bind方法
        return self.url_map.bind(
            self.config["SERVER_NAME"],
            script_name=self.config["APPLICATION_ROOT"],
            url_scheme=self.config["PREFERRED_URL_SCHEME"],
        )

    return None

As you can see, the main function of this method is to call the method or method Mapof the object . I also analyzed it when I talked about the Map class earlier. These two methods mainly return objects.bind_to_environbindMapAdapter

(3) Analyze ctx.push

wsgi_appThe second main step of the method.

After the context object is obtained in the wsgi method, the method is called push, and the code is as follows (only the core code is kept):

class RequestContext:
    def __init__(
        self,
        app: "Flask",
        environ: dict,
        request: t.Optional["Request"] = None,
        session: t.Optional["SessionMixin"] = None,
    ) -> None:
        # 代码省略
        pass

    def match_request(self) -> None:
        try:
          	# 1.调用了MapAdapter对象的match方法，返回了rule对象和参数对象
            result = self.url_adapter.match(return_rule=True) 
            # 2.把rule对象和参数对象放到请求上下文中
            self.request.url_rule, self.request.view_args = result
        except HTTPException as e:
            self.request.routing_exception = e

    def push(self) -> None:
        """Binds the request context to the current context."""
        # 此处省略了前置校验处理代码（上下文、session等处理）
        if self.url_adapter is not None:
          	# 调用了match_request方法
            self.match_request()

It can be seen pushthat the method mainly calls match_requestthe method. This method mainly does the following two things:

1. Calling the match method of the MapAdapter object will Mapmatch the route of the current request according to the routing information stored in the object, and return the rule object and parameter object.
2. Put the rule object and parameter object into the request context.

(4) Analyze full_dispatch_request

wsgi_appThe third main step of the method.

full_dispatch_requestThe method source code is as follows:

def full_dispatch_request(self) -> Response:
    self.try_trigger_before_first_request_functions()
    try:
        request_started.send(self)
        rv = self.preprocess_request()
        if rv is None:
          	# 这里是主要的步骤：分发请求
            rv = self.dispatch_request()
    except Exception as e:
        rv = self.handle_user_exception(e)
    return self.finalize_request(rv)

Among them dispatch_request()is the core method.

dispatch_request()The method source code is as follows:

def dispatch_request(self) -> ResponseReturnValue:
  	# 1.获取请求上下文对象
    req = _request_ctx_stack.top.request
    if req.routing_exception is not None:
        self.raise_routing_exception(req)
    # 2.从上下文中获取前面存在里面的Rule对象
    rule = req.url_rule
    # if we provide automatic options for this URL and the
    # request came with the OPTIONS method, reply automatically
    if (
        getattr(rule, "provide_automatic_options", False)
        and req.method == "OPTIONS"
    ):
        return self.make_default_options_response()
    # 这里是重点：根据rule对象的endpoint属性从self.view_functions属性中获取对应的视图函数，
    # 然后把上下文中的参数传到视图函数中并调用视图函数处理请求，返回处理结果
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)

The main steps of this method are as follows:

1. req = _request_ctx_stack.top.request: Get the request context object
2. rule = req.url_rule: Obtain the previously existing Ruleobject from the context, which is ctx.push()the method put into the context
3. Obtain the corresponding view function from the self.view_functions property according to the endpoint property of the rule object, then pass the parameters in the context to the view function and call the view function to process the request, and return the processing result:return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)

At this point, a complete request is processed.

3 Summary

According to the above analysis, the routing rules and request matching are summarized as follows:

When the app starts:

1. app.route()call add_url_rulemethod.
2. add_url_ruleIn the method: call the self.url_rule_class()instantiated Ruleobject;
3. add_url_ruleIn the method: call self.url_map.add()the method, and store the mapping relationship between Rulethe object endpointand the view function in the Map object.
4. add_url_ruleIn the method: self.view_functions[endpoint] = view_funcadd the mapping of endpoint and view_func.

Request matching process:

1. When the request comes in, WSGIthe server processes and calls __call__the method of the Flask app, and then calls wsgi_appthe method;
2. wsgi_appCreate a context object in the method: ctx = self.request_context(environ). Objects are then instantiated MapAdapteras context object properties;
3. wsgi_appThe method of calling the context object in the method push: ctx.push(). This method mainly uses the methods MapAdapterof the object match.
4. MapAdapterThe method of the object match, to call the method ruleof the object match. This method Mapmatches the route of the current request according to the routing information stored in the object, and gets rulethe object and parameter object into the context object.
5. wsgi_appCall full_dispatch_requestthe method in the method, and then call dispatch_request()the method in it;
6. dispatch_request()In the method: get the request context object, get the object inside Rule, get the corresponding view function from the attribute according to the endpoint attribute of the rule object self.view_functions, then pass the parameters in the context to the view function and call the view function to process the request, and return the processing result .

The whole flow chart is as follows:

In the first half of the project, how to use Ruleand Mapobject to establish routing rules; the second half is how to use routing rules to match when requests come in.