[Heat]Heat中资源的扩展与加载

Heat的核心是stack，从某方面讲，heat就是围绕stack的生命周期在玩，stack又是由各种各样的资源组成的，heat除了自定义的大量资源外，还允许用户自定义自己需要的资源，提供了很强的扩展性，下面将简单分析heat资源加载的流程，来明白heat是如何识别这些资源并如何使用它们的。

if __name__ == '__main__':

    cfg.CONF(project='heat', prog='heat-engine')
    logging.setup('heat')
    messaging.setup()

    mgr = None
    try:

<span style="white-space:pre">	</span># 加载heat的模板类
        mgr = template._get_template_extension_manager()
    except template.TemplatePluginNotRegistered as ex:
        LOG.critical(_LC("%s"), ex)
    if not mgr or not mgr.names():
        sys.exit("ERROR: No template format plugins registered")

    from heat.engine import service as engine

    srv = engine.EngineService(cfg.CONF.host, rpc_api.ENGINE_TOPIC)

上面为heat-egine进程启动的代码，在创建EngineService的时候，这里面有个resources初始化的步骤，很明显，这里应该就是完成资源加载的关键，如下面的代码所示

def __init__(self, host, topic, manager=None):
        super(EngineService, self).__init__()
        resources.initialise()
        self.host = host
        self.topic = topic

具体分析这个方法，首先是有个全局变量_environment，由于是初次加载，所以会执行clients.initialise()

def initialise():
    global _environment
    if _environment is not None:
        return

    clients.initialise()

    global_env = environment.Environment({}, user_env=False)
    _load_global_environment(global_env)
    _environment = global_env

clients其实就是heat与各个openstack组件通信的关键，类似的，这里采用了全局_mgr来管理clients的加载，初次加载的时候由于全局变量_mgr为None，所以采用stevedore加载heat需要用到的各种clients，

_mgr = None


def has_client(name):
    return _mgr and name in _mgr.names()


def initialise():
    global _mgr
    if _mgr:
        return

    _mgr = extension.ExtensionManager(
        namespace='heat.clients',
        invoke_on_load=False,
        verify_requirements=False)

这里的clients为entrypoints中定义的名称空间为heat.clients中的各种client，以Juno版本为例，包含如下组件的clients:

[heat.clients]
ceilometer = heat.engine.clients.os.ceilometer:CeilometerClientPlugin
cinder = heat.engine.clients.os.cinder:CinderClientPlugin
glance = heat.engine.clients.os.glance:GlanceClientPlugin
heat = heat.engine.clients.os.heat_plugin:HeatClientPlugin
keystone = heat.engine.clients.os.keystone:KeystoneClientPlugin
neutron = heat.engine.clients.os.neutron:NeutronClientPlugin
nova = heat.engine.clients.os.nova:NovaClientPlugin
sahara = heat.engine.clients.os.sahara:SaharaClientPlugin
swift = heat.engine.clients.os.swift:SwiftClientPlugin
trove = heat.engine.clients.os.trove:TroveClientPlugin

加载完这些clients后，会定义一个Environment对象，接着往下面看，

def __init__(self, env=None, user_env=True):
        """Create an Environment from a dict of varying format.
        1) old-school flat parameters
        2) or newer {resource_registry: bla, parameters: foo}

        :param env: the json environment
        :param user_env: boolean, if false then we manage python resources too.
        """
        if env is None:
            env = {}
        if user_env:
            from heat.engine import resources
            global_registry = resources.global_env().registry
        else:
            global_registry = None

        self.registry = ResourceRegistry(global_registry)
        self.registry.load(env.get(env_fmt.RESOURCE_REGISTRY, {}))

        if env_fmt.PARAMETERS in env:
            self.params = env[env_fmt.PARAMETERS]
        else:
            self.params = dict((k, v) for (k, v) in six.iteritems(env)
                               if k != env_fmt.RESOURCE_REGISTRY)
        self.constraints = {}
        self.stack_lifecycle_plugins = []

初始化这里并没有太多需要注意的，唯一需要注意的是下面这段，这个就是后面我们加载资源的关键了

 self.registry = ResourceRegistry(global_registry)

接着往下面看，

def _load_global_environment(env):
    _load_global_resources(env)
    environment.read_global_environment(env)


def _load_global_resources(env):
    _register_constraints(env, _get_mapping('heat.constraints'))
    _register_stack_lifecycle_plugins(
        env,
        _get_mapping('heat.stack_lifecycle_plugins'))

    manager = plugin_manager.PluginManager(__name__)
    # Sometimes resources should not be available for registration in Heat due
    # to unsatisfied dependencies. We look first for the function
    # 'available_resource_mapping', which should return the filtered resources.
    # If it is not found, we look for the legacy 'resource_mapping'.
    resource_mapping = plugin_manager.PluginMapping(['available_resource',
                                                     'resource'])
    constraint_mapping = plugin_manager.PluginMapping('constraint')

    _register_resources(env, resource_mapping.load_all(manager))

    _register_constraints(env, constraint_mapping.load_all(manager))

在_load_global_resources方法里面，首先是往env里面注册constraints和stack_lifecycle_plugins，注册的原理很简单，也是用stevedore进行插件的加载，需要加载的项在entry_points里面可以看到

[heat.constraints]
glance.image = heat.engine.clients.os.glance:ImageConstraint
iso_8601 = heat.engine.resources.iso_8601:ISO8601Constraint
neutron.network = heat.engine.clients.os.neutron:NetworkConstraint
neutron.port = heat.engine.clients.os.neutron:PortConstraint
neutron.router = heat.engine.clients.os.neutron:RouterConstraint
neutron.subnet = heat.engine.clients.os.neutron:SubnetConstraint
nova.flavor = heat.engine.resources.server:FlavorConstraint
nova.keypair = heat.engine.resources.nova_keypair:KeypairConstraint

[heat.stack_lifecycle_plugins]

然后通过下面的方法，将这些插件注册到前面定义的global_env对象中，接下面就是resources的加载的，这里定义了2个插件相关的类，PluginManager以及PluginMapping，首先看下PluginManager

class PluginManager(object):
    '''A class for managing plugin modules.'''

    def __init__(self, *extra_packages):
        '''Initialise the Heat Engine plugin package, and any others.

        The heat.engine.plugins package is always created, if it does not
        exist, from the plugin directories specified in the config file, and
        searched for modules. In addition, any extra packages specified are
        also searched for modules. e.g.

        >>> PluginManager('heat.engine.resources')

        will load all modules in the heat.engine.resources package as well as
        any user-supplied plugin modules.

        '''
        def packages():
            for package_name in extra_packages:
                yield sys.modules[package_name]

            cfg.CONF.import_opt('plugin_dirs', 'heat.common.config')
            yield plugin_loader.create_subpackage(cfg.CONF.plugin_dirs,
                                                  'heat.engine')

        def modules():
            pkg_modules = itertools.imap(plugin_loader.load_modules,
                                         packages())
            return itertools.chain.from_iterable(pkg_modules)

        self.modules = list(modules())

其属性modules主要是列出了资源所在的各个模块，这里默认加载的'heat.engine.resources'包下面的各个模块，此外，heat还允许我们自己配置加载资源插件的路径，但是默认来说，heat自身只会加载'heat.engine.resources'下面的模块，所以我们扩展资源插件的时候，可以选择放在这个目录。

接下来定义了2个PluginMapping，分别针对resources和constraints，注意到这里传入的参数，resources传入的是available_resource和resource，后面我们会看到为什么要传这个

resource_mapping = plugin_manager.PluginMapping(['available_resource',
                                                     'resource'])
constraint_mapping = plugin_manager.PluginMapping('constraint')

接着会执行下面这段代码

_register_resources(env, resource_mapping.load_all(manager))

</pre><pre name="code" class="python"># 迭代type_pairs中的内容

def _register_resources(env, type_pairs):
    for res_name, res_class in type_pairs:
        env.register_class(res_name, res_class)

那type_pairs里面到底是啥呢，我们看下相关的方法，首先是resource_mapping.load_all(manager)的返回，可以看到这里返回了个迭代器,迭代器中的内容又是一个迭代器，每个迭代器的内容又来源于 itertools.imap(function, self.modules)，也就是将 load_from_module方法作用于之前加载起来的在resources下面的各个资源模块。

def load_all(self, plugin_manager):
        '''Iterate over the mappings from all modules in the plugin manager.

        Mappings are returned as a list of (key, value) tuples.
        '''
        mod_dicts = plugin_manager.map_to_modules(self.load_from_module)
        return itertools.chain.from_iterable(six.iteritems(d) for d
def map_to_modules(self, function):
        '''Iterate over the results of calling a function on every module.'''
        return itertools.imap(function, self.modules)

我们来看下PluginMapping中的load_from_module这个方法，可以看到，之前传的available_resource和resource参数就起作用了，该方法会从这个模块尝试去取available_resource_mapping和resource_mapping，如果available_resource_mapping或resource_mapping是函数，就会取resouce_mapping方法的内容

def load_from_module(self, module):
        '''Return the mapping specified in the given module.

        If no such mapping is specified, an empty dictionary is returned.
        '''
        for mapping_name in self.names:
            mapping_func = getattr(module, mapping_name, None)
            if callable(mapping_func):
                fmt_data = {'mapping_name': mapping_name, 'module': module}
                try:
                    mapping_dict = mapping_func(*self.args, **self.kwargs)
                except Exception:
                    LOG.error(_('Failed to load %(mapping_name)s '
                                'from %(module)s') % fmt_data)
                    raise
                else:
                    if isinstance(mapping_dict, collections.Mapping):
                        return mapping_dict
                    elif mapping_dict is not None:
                        LOG.error(_('Invalid type for %(mapping_name)s '
                                    'from %(module)s') % fmt_data)

        return {}

我们以heat自带的autoscaling模块为例，它的返回如下

def resource_mapping():
    return {
        'AWS::AutoScaling::AutoScalingGroup': AutoScalingGroup,
        'OS::Heat::InstanceGroup': InstanceGroup,
        'OS::Heat::AutoScalingGroup': AutoScalingResourceGroup,
    }

所以接着上面的代码看，这里的res_name和res_class就是resource_mapping或者available_mapping返回的字典的内容，然后往之前的env对象中注册该资源，

def _register_resources(env, type_pairs):
    for res_name, res_class in type_pairs:
        env.register_class(res_name, res_class)

def register_class(self, resource_type, resource_class):
        ri = ResourceInfo(self, [resource_type], resource_class)
        self._register_info([resource_type], ri)

这里首先根据我们传入的资源创建一个合适的资源类，然后注册到env中，直至把modul中的资源加载完成。加载constraints的过程也和resource类似，这里不继续展开。当env加载完各种插件后，回到最之前的代码会把这个global_env对象赋值给全局变量_environment，

def initialise():
    global _environment
    if _environment is not None:
        return

    clients.initialise()

    global_env = environment.Environment({}, user_env=False)
    _load_global_environment(global_env)
    _environment = global_env

现在我们在resource目录下自定义一个自己的插件，按照上面的分析，可以这么写，让heat能够正确加载我们的插件

from heat.engine import resource


class MyResource(resource.Resource):
    pass


def available_resource_mapping():
    return {'OS::Openstack::MyTest': MyResource}

我们在加载资源插件之后加一句打印代码

<pre name="code" class="python">def _load_global_resources(env):
    _register_constraints(env, _get_mapping('heat.constraints'))
    _register_stack_lifecycle_plugins(
        env,
        _get_mapping('heat.stack_lifecycle_plugins'))

    manager = plugin_manager.PluginManager(__name__)
    # Sometimes resources should not be available for registration in Heat due
    # to unsatisfied dependencies. We look first for the function
    # 'available_resource_mapping', which should return the filtered resources.
    # If it is not found, we look for the legacy 'resource_mapping'.
    resource_mapping = plugin_manager.PluginMapping(['available_resource',
                                                     'resource'])
    constraint_mapping = plugin_manager.PluginMapping('constraint')

    _register_resources(env, resource_mapping.load_all(manager))

    _register_constraints(env, constraint_mapping.load_all(manager))
    print 'OS::Openstack::MyTest' in env.registry._registry

可以发现打印为True，代表我们已经成功加载我们自定义的插件了，是不是非常灵活方便呢，极大的提高了我们扩展资源的方式，也非常简便，heat很酷！

[Heat]Heat中资源的扩展与加载

猜你喜欢