[Source code analysis] The startup process of elastic search 8.0.0 (2)

Immediately after the previous article

1. Create a NodeClient object

Collect the configuration files of all the objects involved so far, again in the form of a list collection,

final List<Setting<?>> additionalSettings = new ArrayList<>();

Then a NodeClient object is created as a parameter. If Node is a static representation of a node, then NodeClient is a dynamic representation of a node, and various actions will be executed. What is an action? Let's talk about it in the corresponding scene. Here it can be simply understood as query, new document and other actions

2. Analyze Plugin

Plugin is an extension point for developers to customize functions in es. After reading the plugin file directory from the disk, we now need to extract the ScriptPlugin type plug-in for operation.

I don’t know how much you know about es api operations, whether you have used es scripts, such as the api below, delete the query results

jiaowu_school/_delete_by_query
{
    "query": {
        "match_all": {}
    }
}

Or update the results of the query

These are two apis that I often use in my daily development. These two apis correspond to the Script keywords here.

There will be the concept of context in the script module, which can be simply understood as some information that you have mastered before doing the operation.

About the detailed introduction of the script, you can open a separate article to introduce it. Since this is about the start, you can't over-introduce it.

Put into memory after parsing

public final Map<String, ScriptEngine> engines;
public final Map<String, ScriptContext<?>> contexts;

Next is the analysis plugin AnalysisPlugin

Related to the tokenizer, such as how to segment, how to filter is related to registration in this step

The next step is to encapsulate Settings into a module, which includes some built-in parameters of the cluster, etc., all encapsulated into SettingsModule objects 

public SettingsModule(
            Settings settings,
            List<Setting<?>> additionalSettings,
            List<String> settingsFilter,
            Set<SettingUpgrader<?>> settingUpgraders) {
        this(
            settings,
            additionalSettings,
            settingsFilter,
            settingUpgraders,
            ClusterSettings.BUILT_IN_CLUSTER_SETTINGS,
            IndexScopedSettings.BUILT_IN_INDEX_SETTINGS);
    }

NetworkService has also been created. During the creation process, it will be judged according to the setting whether the user has defined the method of obtaining the host address. By default, it must not implement the interface to define it yourself

The next step is to find out the plug-ins related to the cluster management behavior in the plug-ins, and these plug-ins all implement the ClusterPlugin interface.

4 by default

Up to now, with the cluster configuration, cluster name, and thread pool, you can encapsulate a ClusterService object for cluster management. It needs to know the status of the cluster and add a listener to monitor the events of the cluster, such as when The local node becomes the master node, or is no longer the master node

The statistics of some information during the operation of the cluster also encapsulates the ClusterInfoService object, but it is mainly static results.

As a framework, es needs to monitor some information and specifically encapsulates the monitoring service. What resources are mainly monitored?

public MonitorService(Settings settings, NodeEnvironment nodeEnvironment, ThreadPool threadPool) throws IOException {
        this.jvmGcMonitorService = new JvmGcMonitorService(settings, threadPool);
        this.osService = new OsService(settings);
        this.processService = new ProcessService(settings);
        this.jvmService = new JvmService(settings);
        this.fsService = new FsService(settings, nodeEnvironment);
    }

Jvm garbage collection situation, basic items of the operating system, process information, jvm information, file system information

All are collected in 1s bit units

3. Analysis of important modules

Next, a ClusterModule is encapsulated, and the configuration and services related to the cluster are processed here, which affects the entire cluster. In other words, the ClusterInfoService mentioned above is also part of the ClusterModule. And it can be seen that many of the above objects are created for the preparation of ClusterModule.

When constructing the object, in addition to using the objects that have been created above, this environment will also create some built-in decider collections, which are a collection of various AllocationDeciders. At present, I don’t know what to allocate, but These AllocationDeciders can be understood as some rules in their respective scenarios, and they can be re-allocated only if these rules are met. Then, the fragmented allocate is created, and finally the decider set and the fragmented allocate are unified into AllocationService for management.

The next step is the analysis of the IndexesModule.

When we deal with indexes, we must deal with mapping related issues, and the first thing to do when creating IndicatorsModule is to get built-in mappers, currently there are 25 types

It is required that the MapperPlugin loaded from the disk cannot be the same as the 25 existing ones, and it is also put into the final mappers.

What does the metadata mapper mean? As shown

These 2 mappers are the most important content of the index module