Openstack liberty 中Cinder-api启动过程源码分析1

在前面的博文中,主要分析了GlanceNova相关的代码,从这篇文章开始我将转到Cinder的源码分析上来。Cinder模块在Openstack中为云主机提供块存储,主要包含:cinder-api,cinder-scheduler,cinder-volumecinder-backup4个部分,后续将通过一系列文章逐个分析各个组件的源码。

今天先来看看cinder-api启动过程的源码分析,预计将包括如下几个方面的内容:

  • 请求路由映射(Python Routes)
  • WSGI 应用发现(Python Paste Deployment)
  • WSGI服务器

限于篇幅,可能将上述主题拆分到多篇博文,下面一起来看具体内容:

启动cinder-api服务

当你通过cinder-api命令(如:/usr/bin/cinder-api --config-file /etc/cinder/cinder.conf)启动api服务时,执行的实际上是
cinder/cmd/api.py/main()函数, 如下:

#`cinder/cmd/api.py/main`
def main():
    """省略次要代码,完成代码请查看相关文件"""

    #加载辅助对象,封装与数据库相关的操作
    objects.register_all()

    #加载配置并设置日志
    CONF(sys.argv[1:], project='cinder',
         version=version.version_string())
    logging.setup(CONF, "cinder")

    """初始化rpc:
    设置全局Transport和Notifier,Transport是
    oslo_messaging/transport.py/Transport实例,我采用的是默认的
    rpc_backend=rabbit,所以Transport采用的driver=oslo_messaging/
    _drivers/impl_rabbit.py/RabbitDriver;Notifier是一个通知消息发
    送器,它借助Transport将通知发送发送给ceilometer
    """
    rpc.init(CONF)

    #通过服务启动器启动WSGI服务(`osapi_volume`)并等待服务启动成功
    #在初始化WSGI服务时,会设置路由映射以及加载WSGI应用程序
    #在启动WSGI服务时,会启动http监听

    #下文具体分析相关内容
    launcher = service.process_launcher()
    server = service.WSGIService('osapi_volume')
    launcher.launch_service(server, workers=server.workers)
    launcher.wait()

创建WSGIService服务对象

def main():

    ......

    #创建一个名为`osapi_volume`的`WSGIService`服务对象
    server = service.WSGIService('osapi_volume')

    ......

#接上文,一起来看看`WSGIService`服务对象的初始化函数
#`cinder/service.py/WSGIService.__init__`
def __init__(self, name, loader=None):

    """Initialize, but do not start the WSGI server."""

    #服务名`osapi_volume`
    self.name = name
    #加载名为(`osapi_volume_manager`)的管理器(None)
    self.manager = self._get_manager()
    """创建WSGI应用加载器(`cinder/wsgi/common.py/Loader`)
    并根据配置文件(`cinder.conf`)设置应用配置路径:
    `config_path` = `/etc/cinder/paste-api.ini`
    """
    self.loader = loader or wsgi_common.Loader()

    """加载WSGI应用并设置路由映射
    return paste.urlmap.URLMap, 请看后文的具体分析
    """
    self.app = self.loader.load_app(name)

    """根据配置文件(`cinder.conf`)设置监听地址及工作线程数
    如果未指定监听ip及端口就分别设置为`0.0.0.0`及`0`
    如果为指定工作线程数就设置为cpu个数
    如果设置的工作线程数小于1,则抛异常
    """
    self.host = getattr(CONF, '%s_listen' % name, "0.0.0.0")
    self.port = getattr(CONF, '%s_listen_port' % name, 0)
    self.workers = (getattr(CONF, '%s_workers' % name, None) or
                        processutils.get_worker_count())
    if self.workers and self.workers < 1:
        worker_name = '%s_workers' % name
        msg = (_("%(worker_name)s value of %(workers)d is" 
                    "invalid, must be greater than 0.") %
                    {
   
   'worker_name': worker_name,
                     'workers': self.workers})
        raise exception.InvalidInput(msg)

    """如果CONF.profiler.profiler_enabled = True就开启性能分析  
    创建一个类型为`Messaging`的通知器(`_notifier`),将性能数据发送给
    ceilometer
    """
    setup_profiler(name, self.host)

    #创建WSGI服务器对象(`cinder/wsgi/eventlet_server.py/Server`)
    #下一篇博文再具体分析WSGI服务器的初始化及启动过程,敬请期待!!!
    self.server = wsgi.Server(name,
                              self.app,
                              host=self.host,
                              port=self.port)

小结:在初始化WSGIService服务对象过程中,主要完成了如下操作:

  • 加载WSGI ApplicationPython Paste Deployment
  • 设置路由映射(Python Routes
  • 创建WSGI服务器对象并完成初始化

先来看WSGI Application的加载过程:

加载WSGI应用

上文的self.loader.load_app(name),执行的是如下的调用:

#`cinder/wsgi/common.py/Loader.load_app`
def load_app(self, name):
    """Return the paste URLMap wrapped WSGI application.

    `Python Paste`系统可以用来发现以及配置`WSGI`应用及服务, 包含如下三
    种调用入口:

               `loadapp`    `loadfilter`   `loadserver`
                  |              |               |      
                         |               |
                                 |
                                 V
                             `loadobj`
                                 |
                                 V
                            `loadcontext` 
                                 |
                        |                |
                 |               |               |
                 V               V               V
           _loadconfig       _loadegg         _loadfunc

    分别用来配置`WSGI App`,`WSGI Filter`,`WSGI Server`;
    `loadcontext`方法基于配置文件类型(`config`,`egg`,`call`),调用具
    体的配置方法,在我们的示例中是:`loadapp` -> `loadobj` -> 
    `loadcontext` -> `_loadconfig`,下文依次分析:
    """

    try:
        #从`self.config_path`(`/etc/cinder/api-paste.ini`)指定的
        #配置中加载名为`name`(`osapi_volume`)的应用
        return deploy.loadapp("config:%s" % self.config_path, 
                                                    name=name)
    except LookupError:
        LOG.exception(_LE("Error loading app %s"), name)
        raise exception.PasteAppNotFound(name=name, path=self.config_path)

#接上文,直接通过`Python Paste`系统配置`WSGI`应用
#`../site-packages/paste/deploy/loadwsgi.py/loadapp`
def loadapp(uri, name=None, **kw):
    """输入参数如下:
    uri: 'config:/etc/cinder/api-paste.ini'
    name: 'osapi_volume'
    **kw: None
    """

    """APP = _APP(),是一个_APP实例对象,定义应用所支持的协议及其前缀:
    APP.name = 'application'
    APP.egg_protocols = [['paste.app_factory'], 
                         ['paste.composite_factory'],
                         ['paste.composit_factory']]
    APP.config_prefixes = [['app', 'application'],
                           ['composite', 'composit'],
                           ['pipeline], 
                           ['filter-app']]

    在后文的分析中会根据应用的协议来生成上下文(`context`)
    """
    return loadobj(APP, uri, name=name, **kw)

#接上文`loadobj`
def loadobj(object_type, uri, name=None, relative_to=None,
            global_conf=None):
    """根据应用的协议类型生成上下文并执行

    object_type: _APP对象
    uri: 'config:/etc/cinder/api-paste.ini'
    name: 'osapi_volume'
    """
    context = loadcontext(
        object_type, uri, name=name, relative_to=relative_to,
        global_conf=global_conf)
    return context.create()

#接上文:这是一个工厂方法,它根据uri中的配置文件类型
#(`config`,`egg`,`call`)分别调用具体的配置方法
#(`_loadconfig`,`_loadegg`, `_loadfunc`)
def loadcontext(object_type, uri, name=None, relative_to=None,
                global_conf=None):
    """创建应用上下文,结合输入参数,代码逻辑就很好理解了

    object_type: _APP对象
    uri: 'config:/etc/cinder/api-paste.ini'
    name: 'osapi_volume'
    relative_to: None
    global_conf: None
    """           
    if '#' in uri:
        if name is None:
            uri, name = uri.split('#', 1)
        else:
            # @@: Ignore fragment or error?
            uri = uri.split('#', 1)[0]
    if name is None:
        name = 'main'
    if ':' not in uri:
        raise LookupError("URI has no scheme: %r" % uri)
    """分割uri路径:
    scheme = 'config'
    path = '/etc/cinder/api-paste.ini'
    """
    scheme, path = uri.split(':', 1)
    scheme = scheme.lower()

    #_loaders是一个全局变量,包含:'config','egg', 'call'三种配置类型
    #方法
    if scheme not in _loaders:
        raise LookupError(
            "URI scheme not known: %r (from %s)"
            % (scheme, ', '.join(_loaders.keys())))
    """path: '/etc/cinder/api-paste.ini'
    这里_loaders['config'] = _loadconfig, 请看下文的分析
    """
    return _loaders[scheme](
        object_type,
        uri, path, name=name, relative_to=relative_to,
        global_conf=global_conf)

#接上文:_loaders[scheme] = _loadconfig
def _loadconfig(object_type, uri, path, name, relative_to,
                global_conf):
    """结合输入参数,代码也很好理解;输入参数如下:

    object_type: _APP对象
    uri: 'config:/etc/cinder/api-paste.ini'
    path: '/etc/cinder/api-paste.ini'
    name: 'osapi_volume'
    relative_to: None
    global_conf: None
    """

    isabs = os.path.isabs(path)
    # De-Windowsify the paths:
    path = path.replace('\\', '/')
    if not isabs:
        if not relative_to:
            raise ValueError(
                "Cannot resolve relative uri %r;no relative_to" 
                "keyword argument given" % uri)
        relative_to = relative_to.replace('\\', '/')
        if relative_to.endswith('/'):
            path = relative_to + path
        else:
            path = relative_to + '/' + path
    if path.startswith('///'):
        path = path[2:]
    path = unquote(path)
    """创建配置加载器ConfigLoader对象,用于加载配置文件内容,后续所有的
    配置解析操作都由该对象完成, 实际上基于不同的`WSGI`程序类型,它分别提
    供了相应的调用接口:
            `get_app`      `get_server`     `get_filter`
                |                |                |
                V                V                V
         `app_context`   `server_context`  `filter_context`
                |                |                |
                       |                  |
                                 |
                                 V
                             get_context 
                                 |
                                 V
                          object_type.invoke

    看完后文的分析,你应该会更有体会!!!
    """
    loader = ConfigLoader(path)
    #如果全局配置不为空,更新`loader`的`defaults`属性
    if global_conf:
        loader.update_defaults(global_conf, overwrite=False)
    #解析配置文件内容,获取上下文(`context`),请看下文的分析
    return loader.get_context(object_type, name, global_conf)

#接上文:`loader.get_context`
def get_context(self, object_type, name=None, 
                global_conf=None):
    """创建上下文的主要函数

    如果`name`满足正则表达式:re.compile(r'^[a-zA-Z]+:'),就再次调
    用`loadcontext`加载上下文,如果不满足条件就先解析配置,然后再根据选
    项条件进入不同分支做进一步的处理

    以`osapi_volume`为例分析,其在`api-paste.ini`中的内容为:
    [composite:osapi_volume]
    use = call:cinder.api:root_app_factory
    /: apiversions
    /v1: openstack_volume_api_v1
    /v2: openstack_volume_api_v2

    首次调用时(序号2)输入参数:name = `osapi_volume`,不满足正则条件,
    就先通过`find_config_section`方法从配置文件加载配置段,然后再根据配
    置前缀(如:`pipeline`)及配置选项(示例中是:`use` 选项),调用
    `_context_from_use`方法, 在该方法中再次调用`get_context`方法
    (序号5)输入参数: name = 'call:cinder.api:root_app_factory',满
    足正则条件,则调用`loadcontext`方法(序号6)加载上下文, 具体的函数调
    用链如下:

    |————>  loadcontext  ———————————————————————|
    |           | (1)                           |(7)
    |           V                               V
    |(6)    _loadconfig                     _loadcall
    |           | (2)                           | (8)
    |           V                               V
    |—— ConfigLoader.get_context      FuncLoader.get_context
                |(3)           | ———— |         |(9)
                V                 (5)|         V
    ConfigLoader.find_config_section  |  LoaderContext.create   
                |(4)                 |         |(10)
                V                     |         V
    ConfigLoader._context_from_use —— |  object_type.invoke

    在`object_type.invoke`方法中根据协议类型调用应用的`factory`方法
    (如:`cinder.api:root_app_factory`), 创建应用对象                       
    """
    if self.absolute_name(name):
        return loadcontext(object_type, name,
              relative_to=os.path.dirname(self.filename),
                               global_conf=global_conf)
    #根据配置前缀及应用名称,加载配置段                           
    section = self.find_config_section(
            object_type, name=name)

    """`defaults`配置,在创建`ConfigLoader`对象时指定, 这里是:
    {
     'here':'/etc/cinder'
     '__file__':'/etc/cinder/api-paste.ini'
    }
    """
    if global_conf is None:
        global_conf = {}
    else:
        global_conf = global_conf.copy()
    defaults = self.parser.defaults()
    #用`defaults`更新`global_conf`
    global_conf.update(defaults)
    #根据配置端中的选项设置属性
    for option in self.parser.options(section):
        #全局选项(`set`用来重写全局选项)
        if option.startswith('set '):
            name = option[4:].strip()
            global_additions[name] = global_conf[name] = (
                            self.parser.get(section, option))
        #全局选项(`get`使用全局变量值)
        elif option.startswith('get '):
            name = option[4:].strip()
            get_from_globals[name] = self.parser.get(section, 
                                                      option)
        else:
            if option in defaults:
                # @@: It's a global option (?), so skip it
                continue
            #其他的局部选项
            local_conf[option] = self.parser.get(section, 
                                                        option)
    #用全局变量值更新局部变量                                                   
    for local_var, glob_var in get_from_globals.items():
        local_conf[local_var] = global_conf[glob_var]

    #取得属性中包含的过滤器(如果有的话),在`Paste Deployment`规则中,
    #过滤器(filter)及应用(app)中可以包含其他的过滤器
    if object_type in (APP, FILTER) and 'filter-with' in 
                                                    local_conf:
        filter_with = local_conf.pop('filter-with')
    else:
        filter_with = None

    #加载指定的资源
    if 'require' in local_conf:
        for spec in local_conf['require'].split():
            pkg_resources.require(spec)
        del local_conf['require']
    #根据前缀创建上下文(根据配置api-paste.ini文件中的内容就能很容易
    #知道该走那个分支了,如:`composite:osapi_volume`走的就是
    #`'use' in local_conf`分支)
    if section.startswith('filter-app:'):
        context = self._filter_app_context(
                object_type, section, name=name,
                global_conf=global_conf, local_conf=local_conf,
                global_additions=global_additions)
    elif section.startswith('pipeline:'):
        context = self._pipeline_app_context(
                object_type, section, name=name,
                global_conf=global_conf, local_conf=local_conf,
                global_additions=global_additions)
    elif 'use' in local_conf:
        #该方法涉及的函数调用链,请看上文的简图
        context = self._context_from_use(
                object_type, local_conf, global_conf, 
                global_additions,
                section)
    else:
        #过滤器,走这里。下文再具体分析
         context = self._context_from_explicit(
                object_type, local_conf, global_conf, 
                global_additions,
                section)
    #过滤器(filter)及应用(app)中包含其他的过滤器
    if filter_with is not None:
        filter_with_context = LoaderContext(
                obj=None,
                object_type=FILTER_WITH,
                protocol=None,
                global_conf=global_conf, local_conf=local_conf,
                loader=self)
        filter_with_context.filter_context = 
                    self.filter_context(
                    name=filter_with, global_conf=global_conf)
        filter_with_context.next_context = context
            return filter_with_context
    return context

经过上文的分析我们知道Python Paste Deployment系统是如何根据api-paste.ini配置文件一步一步找到osapi_volume应用的加载入口(cinder.api.root_app_factory)的,完成的函数调用链条如下:

            `loadapp`   
                |                  
                V
            `loadobj`
                |
                V
    |————>  `loadcontext`  ———————————————————————|
    |           | (1)                             |(7)
    |           V                                 V
    |(6`_loadconfig`                     `_loadcall`
    |           | (2)                             | (8)
    |           V                                 V
    |—— `ConfigLoader.get_context`     `FuncLoader.get_context`
                |(3)           ^——————|           |(9)
                V                     |           V
`ConfigLoader.find_config_section`    | `LoaderContext.create`   
                |(4)             (5)|           |(10)
                V                     |           V
    `ConfigLoader._context_from_use`—`_APP.invoke`
                                                  |(11)
                                                  V
                                 `cinder.api:root_app_factory`    

下面继续来看·osapi_volume应用的加载过程

加载应用

经过上文Python Paste的解析,我们找到了osapi_volume应用的处理函数,如下:

#/cinder/api/__init__.py/root_app_factory`
def root_app_factory(loader, global_conf, **local_conf):
    """输入参数:
    loader ConfigLoader实例
    global_conf 全局配置字典 {'here':'/etc/cinder', 
                        '__file__':'/etc/cinder/api-paste.ini'}
    **local_conf 局部配置字典 {'/v2': 'openstack_volume_api_v2', 
                             '/v1': 'openstack_volume_api_v1', 
                             '/': 'apiversions'}

    发现了吧, local_conf字典就是`api-paste.ini`中
    `[composite:osapi_volume]`中包含的选项内容
    """

    #根据(`cinder.conf`)中的配置执行相关的处理
    #我的示例中v1及v2都是开启的
    if CONF.enable_v1_api:
        LOG.warning(_LW('The v1 api is deprecated and will be' 
        'removed in the Liberty release. You should set'
        'enable_v1_api=false and enable_v2_api=true in your'
        ' cinder.conf file.'))
    else:
        del local_conf['/v1']
    if not CONF.enable_v2_api:
        del local_conf['/v2']
    #再次调用`Python Paste`处理应用的加载,请看下文的具体分析
    return paste.urlmap.urlmap_factory(loader, global_conf, 
                                                **local_conf)   
#接上文:`paste.urlmap.urlmap_factory`
def urlmap_factory(loader, global_conf, **local_conf):

    #加载`not_found_app`应用,我的示例中为None
    if 'not_found_app' in local_conf:
        not_found_app = local_conf.pop('not_found_app')
    else:
        not_found_app = global_conf.get('not_found_app')
    if not_found_app:
        not_found_app = loader.get_app(not_found_app, 
                                global_conf=global_conf)
    #创建URLMap对象,用于存储<path, app>映射
    urlmap = URLMap(not_found_app=not_found_app)
    #逐一加载`local_conf`中的应用
    for path, app_name in local_conf.items():
        path = parse_path_expression(path)
        """调用`ConfigLoader.get_app`加载应用,, 下文以: 
        `'/v2': 'openstack_volume_api_v2'`为例,分析应用的加载过程
        请看下文的具体分析
        """
        app = loader.get_app(app_name, global_conf=global_conf)
        urlmap[path] = app
    #返回URLMap对象给调用者
    return urlmap
#接上文:`loader.get_app`
#`../site-packages/paste/deploy/loadwsgi.py/_Loader/get_app`

#还记得上文`_loadconfig`中所说的吧:`ConfigLoader`对外提供三个接口
#(`get_app`, `get_server`, `get_filter`)分别用于加载不同的`WSGI`程
#序,这里加载应用使用的就是`get_app`方法,请看:
def get_app(self, name=None, global_conf=None):
    #先获取上下文,然后创建(激活)
     return self.app_context(
            name=name, global_conf=global_conf).create()

def app_context(self, name=None, global_conf=None):
    """获取name=`openstack_volume_api_v2`应用的上下文
    看到ConfigLoader.get_context方法调用,是否有点印象!!!

    上文获取`osapi_volume`上下文就是通过该方法完成了的,再对比下`api-
    paste.ini`文件中两个应用的配置,很相似吧!下文就不在重复分析了,直接
    给出函数流程图表:
        `ConfigLoader.get_app`
                 |(1)
                 V
     `ConfigLoader.app_context`
                 |(2)
                 V                  (6)
     `ConfigLoader.get_context` ---------- `loadcontext`
                |(3)           ^——————|           |(7)
                V                     |           V
`ConfigLoader.find_config_section`    |       `_loadcall`   
                |(4)             (5)|           |(8)
                V                     |           V
`ConfigLoader._context_from_use`—————— `FuncLoader.get_context`  
                                                  |(9)
                                                  V
                                        `LoaderContext.create`
                                                  |(10)
                                                  V
                                             `_APP.invoke`
                                                  |(11)
                                                  V
                 `cinder.api.middleware.auth:pipeline_factory`

    对比上文`osapi_volume`应用的函数流程图表,可以发现处理过程基本是
    一样的,入口不一样罢了!!!
    """
    return self.get_context(
            APP, name=name, global_conf=global_conf)

通过上文的分析,我们得到了openstack_volume_api_v2应用的加载入口
cinder.api.middleware.auth:pipeline_factory,下面一起来看看该方法的处理过程:

#`/cinder/api/middleware/auth.py/pipeline_factory`
def pipeline_factory(loader, global_conf, **local_conf):
    """A paste pipeline replica that keys off of 
    auth_strategy.

    输入参数:
    loader ConfigLoader实例
    global_conf = {'__file__': '/etc/cinder/api-paste.ini', 
                    'here': '/etc/cinder'}
    local_conf = {
    'keystone': 'request_id faultwrap sizelimit osprofiler' 
                'authtoken keystonecontext apiv2', 
    'noauth': 'request_id faultwrap sizelimit osprofiler'
              'noauth apiv2', 
    'keystone_nolimit': 'request_id faultwrap sizelimit'
                 'osprofiler authtoken keystonecontext apiv2'}
    """
    #基于配置选择选项,我的例子中是:`keystone`
    pipeline = local_conf[CONF.auth_strategy]
    if not CONF.api_rate_limit:
        limit_name = CONF.auth_strategy + '_nolimit'
        pipeline = local_conf.get(limit_name, pipeline)
    #链表化:['request_id', 'faultwrap', 'sizelimit', 
    #'osprofiler', 'authtoken', 'keystonecontext', 'apiv2']
    pipeline = pipeline.split()
    """逐个加载过滤器
    加载过滤器的过程和上文加载应用的逻辑类似!唯一不同的
    是:object_type = _Filter,  直接给出函数调用流程图表:

        ConfigLoader.get_filter
                |
                V
        ConfigLoader.filter_context
                |
                V
        ConfigLoader.get_context
                |
                V
    ConfigLoader._context_from_explicit
                |
                V
        LoaderContext.create
                |
                V
        _Filter.invoke

    _Filter.invoke会调用各个过滤器的工厂方法(`factory`)生成对应的过
    滤对象(各个过滤器的工厂方法请查看`api-paste.ini`文件中对应的
    section)
    """
    filters = [loader.get_filter(n) for n in pipeline[:-1]]
    """加载应用, 由于与前述的加载过滤器的逻辑相似,这里直接给函数调用链:
        ConfigLoader.get_app
                |
                V
        ConfigLoader.app_context
                |
                V
        ConfigLoader.get_context
                |
                V
        ConfigLoader._context_from_explicit
                |
                V
        LoaderContext.create
                |
                V
          _APP.create  

    `_APP.create`会创建`cinder.api.v2.router.APIRouter`对象,在构造
    对象过程中会加载`cinder.api.contrib`下定义的扩展并建立路径映射,相
    关内容下一篇博文节再具体分析,敬请期待!!!
    """
    app = loader.get_app(pipeline[-1])
    #反转过滤器
    filters.reverse()
    """逐一创建各个过滤器,并以`前一个过滤器`作为参数,所以最终得到的:
    app = RequestId(FaultWrapper(RequestBodySizeLimiter(WsgiMiddleware(AuthProtocol(CinderKeystoneContext(APIRouter()))))))
    """
    for filter in filters:
        app = filter(app)
    return app

经过上面的分析,WSGI应用就加载完成了。最终返回给调用者的是paste.urlmap.URLMap对象,里面包含三个<path, app>程序,这样cinder-api就能根据path,调用指定的app了。

请求路由映射(Python Routes)

在上文中提到,在加载apiv2应用时会初始化APIRouter对象,该对象的顶层依赖关系如下:

            `cinder.wsgi.common:Router`
                        ^(继承)
                        |
        `cinder.api.openstack.__init__:APIRouter`
                        ^(继承)
                        |
`cinder.api.v2.router:APIRouter`---> (依赖)
                       `cinder.api.extensions:ExtensionManager`

下面来看APIRouter的初始化过程:

def __init__(self, ext_mgr=None):
    if ext_mgr is None:
        #ExtensionManager是个对象变量(类似C语言中的类变量),在对象实
        #例化前赋值: `cinder.api.extensions:ExtensionManager`
        if self.ExtensionManager:
            #实例化扩展管理器,内部会加载`cinder.api.contrib.*`下定义
            #的扩展(模块),模块加载完后,以<alias, ext>字典保存在扩展
            #管理的的`extentions`字典中,请看下文的具体分析
            ext_mgr = self.ExtensionManager()
        else:
            raise Exception(_("Must specify an "
                                    "ExtensionManager class"))

        #创建路由映射,后文分析
        ........

加载扩展

#`cinder.api.extensions:ExtensionManager`
def __init__(self):
    # 基于`cinder.conf`文件:CONF.osapi_volume_extension = 
    #cinder.api.contrib.standard_extensions
    self.cls_list = CONF.osapi_volume_extension
    self.extensions = {}
    #加载扩展(模块)
    self._load_extensions()

#接上文:
 def _load_extensions(self):
     #extensions = [cinder.api.contrib.standard_extensions]
     extensions = list(self.cls_list)

    #循环加载扩展,基于我的配置extensions中其实只有一个对象
     for ext_factory in extensions:
         """这里省略try{}except异常代码块
         加载扩展`cinder.api.contrib.standard_extensions`
         """ 
         self.load_extension(ext_factory)

#接上文:
def load_extension(self, ext_factory):
    # Load the factory
    #导入`cinder.api.contrib.standard_extensions`
    factory = importutils.import_class(ext_factory)
    #执行
    factory(self)

#接上文:`cinder.api.contrib.standard_extensions`
def standard_extensions(ext_mgr):
    """参数如下:
    ext_mgr cinder.api.extensions.ExtensionManager对象
    LOG 全局日志对象
    __path__ opt/stack/cinder/api/contrib, 包路径
    __package__ cinder.api.contrib  
    """
    extensions.load_standard_extensions(ext_mgr, LOG, __path__, 
                                                   __package__)

#接上文:`cinder.api.extensions.py/load_standard_extensions`
def load_standard_extensions(ext_mgr, logger, path, package, 
                                               ext_list=None):
    """Registers all standard API extensions.

    参数如上所示
    """
    #our_dir = `opt/stack/cinder/api/contrib`
    our_dir = path[0]

    # Walk through all the modules in our directory...
    #逐个加载目录下的模块
    for dirpath, dirnames, filenames in os.walk(our_dir):
        # Compute the relative package name from the dirpath
        #计算包的相对路径:'.'
        relpath = os.path.relpath(dirpath, our_dir)
        if relpath == '.':
            relpkg = ''
        else:
            relpkg = '.%s' % '.'.join(relpath.split(os.sep))

        # Now, consider each file in turn, only considering 
        #.py files
        #遍历`.py`文件
        for fname in filenames:
            #将文件名按<filename, ext>拆分
            root, ext = os.path.splitext(fname)

            #跳过`__init__.py`文件
            if ext != '.py' or root == '__init__':
                continue

            #由文件名(如:availability_zones)得到类名
            #(Availability_zones)
            classname = "%s%s" % (root[0].upper(), root[1:])
            """得到类路径(如):
            `cinder.api.contrib.availability_zones
            .Availability_zones`
            """
            classpath = ("%s%s.%s.%s" %
                         (package, relpkg, root, classname))

            if ext_list is not None and classname not in 
                                                    ext_list:
                logger.debug("Skipping extension: %s" % 
                                                   classpath)
                continue

            """省略try{}except异常处理,先来看看函数调用流程:

                `ExtensionManager._load_extensions`
                                |
                                V
            |-> `ExtensionManager.load_extension`
            |                   |加载`path`模块
            |                   V
            |        `standard_extensions` or  `xxx`
            |                   |
            |                   V
            ------  `load_standard_extensions`

            可以看到,是通过ExtensionManager来加载模块的,如:
            `cinder.api.contrib.availability_zones
            .Availability_zones`, 创建`Availability_zones`实例,
            并注册到`ExtensionManager`中;其他的模块也是按照相同的模式
            加载注册的,这里就不多说了,最后会得到一个:
            `self.extensions[alias]` = ext 字典
            """
            ext_mgr.load_extension(classpath)   

            #加载包,由于我们的例子中没有子目录,代码就不再给出了
            #其实原理也很简单:直接导入包,然后实例化就好了                    
            subdirs = []
            for dname in dirnames:   
                ......

创建映射路由

上文完成了扩展模块的加载,下面继续来看路由的映射过程:

#`cinder/api/openstack/__init__.py/APIRouer.__init__`
def __init__(self, ext_mgr=None):
    #扩展模块部分,请看上文
    .......

    #创建ProjectMapper对象,为后文的路径映射做准备
    #类继承关系:`ProjectManager`->APIMapper->routes.Mapper
    mapper = ProjectMapper()
    self.resources = {}
    #映射路由,请看下文的分析
    self._setup_routes(mapper, ext_mgr)
    #映射扩展路由,请看下文的分析
    self._setup_ext_routes(mapper, ext_mgr)
    #扩展`资源扩展`,请看下文的分析
    self._setup_extensions(ext_mgr)
    #创建`RoutesMiddleware`对象,并设置回调方法及`mapper`
    super(APIRouter, self).__init__(mapper)

#接上文:映射路由
def _setup_routes(self, mapper, ext_mgr):

    """创建获取版本信息的路由

    1.创建一个`WSGI`应用(Resource->Applications),所使用的
    `controller` = `cinder.api.versions:VolumeVersion`,
    2.通过`Python Routes``建立一条名为`versions`的路由,在这里不深究
    `Python Routes`的代码实现,知道具有下述的<path, action>映射就行:

    `GET`  `/versions/show`    `VolumeVersion.show`
    """
    self.resources['versions'] = versions.create_resource()
    mapper.connect("versions", "/",
                   controller=self.resources['versions'],
                   action='show')

    #路径重定向(没有指定根路径的映射都定向到`/`)
    mapper.redirect("", "/")

    """1.创建一个`WSGI`应用(Resource->Applications),所使用的
    `controller` = `cinder.api.v2.volumes:VolumeController`
    2.创建路由,路径映射为(等):

 `GET` `/{project_id}/volumes/detail` `VolumeController.detail` 
  `POST` `/{project_id}/volumes/create` 
                                      `VolumeController.create`    
  `POST` `/{project_id}/volumes/:{id}/action`
                                      `VolumeController.action`
  `PUT` `/{project_id}/volumes/:{id}/action`
                                      `VolumeController.action`

    """
    self.resources['volumes'] = 
                               volumes.create_resource(ext_mgr)
    mapper.resource("volume", "volumes",
                        controller=self.resources['volumes'],
                        collection={
   
   'detail': 'GET'},
                        member={
   
   'action': 'POST'})

    #后文的路径映射大同小异,就不再列出了,读者可以自行查阅
    ......

#接上文:映射扩展路由
 def _setup_ext_routes(self, mapper, ext_mgr):
     #还记得上文中加载了`cinder.api.contrib`下面的扩展模块吧
     #这里返回的是那些包含资源扩展(`ResourceExtentson`)的扩展模块
     #下文以`cinder.api.contrib.hosts:Hosts`扩展模块为例, 
     for resource in ext_mgr.get_resources():
         """把资源扩展添加到`resources`字典
         resource = `cinder.api.extensions.ResourceExtension`
         resource.collection = 'os-hosts'
         resource.controller = 
                     `cinder.api.contrib.hosts.HostController`
         """
         wsgi_resource = wsgi.Resource(resource.controller)
         self.resources[resource.collection] = wsgi_resource

         """字典信息如下:
         {'member': {'startup': 'GET', 'reboot': 'GET', 
                                         'shutdown': 'GET'}, 
         'controller': <cinder.api.openstack.wsgi.Resource 
                                       object at 0x5a09550>, 
         'collection': {'update': 'PUT'}
         }
         """
         kargs = dict(
                controller=wsgi_resource,
                collection=resource.collection_actions,
                member=resource.member_actions)

         if resource.parent:
                kargs['parent_resource'] = resource.parent
        #为资源扩展建立路由
        mapper.resource(resource.collection, 
                                 resource.collection, **kargs)

        if resource.custom_routes_fn:
            resource.custom_routes_fn(mapper, wsgi_resource)

#接上文:扩展`资源扩展`的方法
def _setup_extensions(self, ext_mgr):
    #还记得上文中加载了`cinder.api.contrib`下面的扩展模块吧
    #这里返回的是那些包含控制器扩展(`ControllerExtension`)的扩展模块
    #下文以`cinder.api.contrib.scheduler_hints:Scheduler_hints`
    #为例
    for extension in ext_mgr.get_controller_extensions():
        #collection = `volumes`
        #controller = `cinder.api.contrib.scheduler_hints
                                     .SchedulerHintsController`
        collection = extension.collection
        controller = extension.controller

        #排除不包含资源扩展的控制器扩展
        if collection not in self.resources:
            LOG.warning(_LW('Extension %(ext_name)s: Cannot'
            ' extend resource %(collection)s: No such'
            ' resource'),{
   
   'ext_name': extension.extension.name,
                          'collection': collection})
            continue

        #将控制器扩展注册到资源扩展中
        resource = self.resources[collection]
        #注册`wsgi actions`(wsgi_actions字典),如果包含
        #`wsgi_actions`属性的话,`SchedulerHintsController`不包含该
        #属性,所以为None
        resource.register_actions(controller)
        #注册`wsgi extensions`(wsgi_extensions字典),如果包含
        #`wsgi_extensions`属性的话,`SchedulerHintsController`包含
        #{create, None}的属性,所以会建立如下映射: 
        #wsgi_extensions['create'] 
        #                 = `SchedulerHintsController.create`
        resource.register_extensions(controller)

至此cinder-api启动过程中,加载WSGI应用及建立路由的过程就分析完成了。下一篇博文将分析cinder-api启动过程中WSGI服务器的启动过程以及它是如何处理客户端请求的。敬请期待!!!

猜你喜欢

转载自blog.csdn.net/lzw06061139/article/details/52211384