Nothing is done at the beginning when it starts, the actual execution starts from this method
org.elasticsearch.cli.Command#execute
1. CreateEnvironment
Will use the configuration file of conf in elasticsearch home, required
Prepare before creating Environment
The core key is to get the value of path.home passed in from the command line and put it in the setting. Because when creating the Environment, according to the parameter
config、plugins、data、logs、bin、lib、modules等解析出来，正好对应相应文件夹
xiaohuihui@xiaohuihuideMBP 8_home % ls
config data lib logs modules plugins
It is equivalent to parsing the corresponding path from path.home. At this time, there will be an Environment variable. This variable is not the Environment that is finally returned because the configuration file elasticsearch.yaml has not been parsed.
Then start parsing the elasticsearch.yaml file. If the user sets some logs and data paths in the configuration file, it will be the final path, because after parsing the configuration file, es will return with a new Environment, the logs at this time The path of, data, etc. is after parsing elasticsearch.yaml. To sum up, es itself has a default path for logs, data, etc. If the user sets it through elasticsearch.yaml, the user setting is used.
2、init
2.1 First, create an Environment based on the settings passed down above, (why don't you know)
2.2 Then start to configure log. Still based on the log path in the environment
The name of the loaded log plugin is org.elasticsearch.common.logging.
The configuration file related to log is log4j2.properties, es will load the file and parse the configuration inside
The default log level is Level.ERROR, but through the logger.level configuration, the default value will be overwritten if it is set, so the default value can also be adjusted
2.3 Check if the version of Lucene is correct
2.4 Start processing the modules module
There are 49 in total. Each one is encapsulated as a PluginInfo object
But no other processing is done, only one eligible process is started
2.5 Then check and set some resources of the system
For example, it cannot be operated by the root user
If the Linux platform is trying to apply for the maximum number of threads, it means that the number of threads will not be limited when applying for threads.
2.6 Initial SecurityManager for Environment
es customized ESPolicy and started SecurityManager. (I didn't understand)
3. Start creating Node objects
Node represents a node in the cluster, the setting information at this time
{"client.type":"node","cluster.name":"elasticsearch","node.name":null,"path.home":"/Users/xiaohuihui/Downloads/elasticsearch-master/build/elasticsearch-distros/extracted_elasticsearch_8.0.0-SNAPSHOT_archive_darwin_default/elasticsearch-8.0.0-SNAPSHOT","path.logs":"/Users/xiaohuihui/Downloads/elasticsearch-master/build/elasticsearch-distros/extracted_elasticsearch_8.0.0-SNAPSHOT_archive_darwin_default/elasticsearch-8.0.0-SNAPSHOT/logs"}What is the use of this setting? It will be used when making judgments in some scenarios below.
3.1 Encapsulate the processing of modules and plugins into an object called PluginsService. The process of creating it is the process of loading modules and plugins. After the creation is complete, continue to return to the Node creation process, and assign the PluginsService object to a variable in Node, and then extract the roles of these plugins, put them in the memory, and save them using a Map called roleMap
3.2 Create NodeEnvironment. It refers to the path of data in es home, because some index-related files are also stored in the data folder.
The NodeEnvironment here refers to the content framed in the red box.
At the same time, it will check whether the folder has write permission and whether it has atomic move permission
The file on the disk will be loaded into the memory according to the data path, and encapsulated into a NodeMetadata object. What does this object mean? It exists in the data folder of the current node. If the es instance is started repeatedly, it will still exist. This can be verified. When you start es, if you performed an operation last time, you will find that there are records in the data folder. If you start es for the first time, the data folder is empty. If the object has a value, as shown below
NodeMetadata{nodeId='zTVfnC6sRReXE9mbtqd6Aw', nodeVersion=8.0.0}In this way, the NodeEnvironment is created. The node object uses the reference nodeEnvironment to point to it.
Next, an object LocalNodeFactory is created. According to the name, it is the local node factory, but it is constructed based on nodeId and setting. I haven't seen how to play a role yet.
Next, a ThreadPool object is created. Don't be confused by its name. Here, instead of creating a thread pool, you have created several thread pools. How many have been created? 27, and each thread pool has its own name. A thread pool is created for each module. In fact, it is also used in our business system. It is impossible to share one globally, and each module uses its own. Do not interfere with each other. es is the same
The internal class ExecutorHolder is the thread pool we are familiar with, let’s look at its construction method
static class ExecutorHolder {
private final ExecutorService executor;
public final Info info;
ExecutorHolder(ExecutorService executor, Info info) {}Is even more certain
But after all, the thread pool is encapsulated, and es also divides the thread pool into types.
DIRECT("direct"),
FIXED("fixed"),
FIXED_AUTO_QUEUE_SIZE("fixed_auto_queue_size"), // TODO: remove in 9.0
SCALING("scaling");The difference between fixed and scaling is mainly in the number of threads
How to deal with the release of resources during the whole process of building Node?
The resource release here refers to IO requests for reading disk data, creating memory thread resources such as thread pools, and using a list for management. Collect those resources that are recycled in abnormal situations as the code continues to go down, rather than try finally and other code blocks for each resource that needs to be recycled, such as
resourcesToClose.add(() -> ThreadPool.terminate(threadPool, 10, TimeUnit.SECONDS));
resourcesToClose.add(() -> HeaderWarning.removeThreadContext(threadPool.getThreadContext()));Pay attention to the official account and help to increase a fan
