Elasticsearch7 builds a cluster and configures node certificates

Mobile 2023-07-01 10:35:56 views: null

Elasticsearch7 builds a cluster and configures node certificates | handsome uncle's blog

The stand-alone version is relatively simple. Try the cluster version, which has limited resources. The example in this article: a host starts with different ports to build a cluster.

Environment description:

  • Centos7
  • Elasticsearch7.9.0

Prepare to build 3 nodes

1. Download the ES installation package

Go to the official website to download

Choose the download Linux x86_64type of version 7.9.0. If you don’t understand the difference between x86_64 and AARCHS, please do some popular science.

Execute the Shell command process:

# download
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.9.0-linux-x86_64.tar.gz

# unzip
tar -zxvf elasticsearch-7.9.0-linux-x86_64.tar.gz

# After decompression, generate a folder named: elasticsearch-7.9.0
# For the convenience of management, create a new folder: elasticsearch-cluster
mkdir elasticsearch-cluster

# Copy 3 copies of elasticsearch-7.9.0 to elasticsearch-cluster and rename
cp -r  elasticsearch-7.9.0 elasticsearch-cluster/elasticsearch-9301
cp -r  elasticsearch-7.9.0 elasticsearch-cluster/elasticsearch-9302
cp -r  elasticsearch-7.9.0 elasticsearch-cluster/elasticsearch-9303

Looking at the files I copied, I know that I plan to start 3 nodes, and the startup ports are: 9301, 9302, 9303

2. Modify the configuration file of the node

Explanation of common configuration files

# cluster name
cluster.name: mtSearch

# Node name, cannot be the same
node.name: mtNode1

# Whether it is the master node
node.master: true

# Whether it is a data node
node.data: true

# log path
path.logs: /opt/elasticsearch-cluster/logs

# Lock the physical memory address to prevent elasticsearch memory from being swapped out, that is, to avoid es using swap to swap partitions
# If bootstrap.memory_lock is true, remember to modify /etc/security/limits.conf to add memlock configuration, comment it first
bootstrap.memory_lock: true

# map ip
network.host: 172.16.1.236

# http request port
http.port: 9200

# Set the tcp port for interaction between nodes, the default is 9300.
transport.tcp.port: 9300

# Set whether to compress the data during tcp transmission, the default is false, no compression.
transport.tcp.compress: true

# Cluster seed host address, if it is before the 7.* version, you can use the discovery.zen.ping.unicast.hosts parameter. This is also compatible with the 7 version, but it may be deleted later
discovery.seed_hosts: ["172.16.1.236:9300","172.16.1.236:9301","172.16.1.236:9302"]

# To prevent split-brain, how many nodes are needed to elect a Master (the minimum number of candidate nodes), generally set to N/2 + 1 (N is the number of nodes in the cluster)
discovery.zen.minimum_master_nodes: 2


# Used to get the address of the seed node that started the discovery process. By default, it is based on the setting of the seed host provider, which is the above seed address,
# There is also a file-based one, which needs to create a new unicast_hosts.txt file in the installation directory
discovery.seed_providers: file

# If you do not configure the options of discovery.seed_hosts and discovery.seed_providers, the configuration of cluster.initial_master_nodes will be used for master node election.
# If the above two configurations are configured, the configuration of cluster.initial_master_nodes will not take effect.
# What the official document says: https://www.elastic.co/guide/en/elasticsearch/reference/7.9/modules-discovery-bootstrap-cluster.html#_auto_bootstrapping_in_development_mode
cluster.initial_master_nodes: ["mtNode1", "mtNode2", "mtNode3"]

# Restart only restores data if there is so much data or the master node has joined the cluster
gateway.recover_after_nodes: 2

# Cross domain issues
http.cors.enabled: true
http.cors.allow-origin: "*"

1. Configuration files for each node

The configuration of each node is not the same

9301 node configuration
cluster.name: mtSearch
node.name: mtNode1
node.master: true
node.data: true
path.logs: /opt/elasticsearch-cluster/logs
network.host: 172.16.1.236
http.port: 9201
transport.tcp.port: 9301
transport.tcp.compress: true
discovery.seed_hosts: ["172.16.1.236:9300","172.16.1.236:9301","172.16.1.236:9302"]
cluster.initial_master_nodes: ["mtNode1","mtNode2","mtNode3"]
gateway.recover_after_nodes: 2
http.cors.enabled: true
http.cors.allow-origin: "*"
9302 node configuration
cluster.name: mtSearch
node.name: mtNode2
node.master: true
node.data: true
path.logs: /opt/elasticsearch-cluster/logs
network.host: 172.16.1.236
http.port: 9202
transport.tcp.port: 9302
transport.tcp.compress: true
discovery.seed_hosts: ["172.16.1.236:9300","172.16.1.236:9301","172.16.1.236:9302"]
cluster.initial_master_nodes: ["mtNode1","mtNode2","mtNode3"]
gateway.recover_after_nodes: 2
http.cors.enabled: true
http.cors.allow-origin: "*"
9303 node configuration
cluster.name: mtSearch
node.name: mtNode3
node.master: true
node.data: true
path.logs: /opt/elasticsearch-cluster/logs
network.host: 172.16.1.236
http.port: 9203
transport.tcp.port: 9303
transport.tcp.compress: true
discovery.seed_hosts: ["172.16.1.236:9300","172.16.1.236:9301","172.16.1.236:9302"]
cluster.initial_master_nodes: ["mtNode1","mtNode2","mtNode3"]
gateway.recover_after_nodes: 2
http.cors.enabled: true
http.cors.allow-origin: "*"

After all configurations are completed, start the three nodes respectively, and you can see that the cluster connection is successful without any accident.

/opt/elasticsearch-cluster/elasticsearch-9301/elasticsearch -d
/opt/elasticsearch-cluster/elasticsearch-9302/elasticsearch -d
/opt/elasticsearch-cluster/elasticsearch-9303/elasticsearch -d

启动成功，如下图：

如果你之前没安装过，可能会报错：像root用户运行呀，线程不够呀，内存不足呀等等问题
可以看本文最后面的踩坑笔记

集群确实是搞定了，但是没配置账号密码，集群中各节点之间的通信是也没有什么校验措施的，别人随随便便就连上集群。这样在互联网中就相当于裸奔！

三、配置证书

TLS需要X.509证书才能对与之通信的应用程序执行加密和身份验证。为了使节点之间的通信真正安全，必须对证书进行验证。在Elasticsearch集群中验证证书真实性的推荐方法是信任签署证书的证书颁发机构（CA）。这样，将节点添加到群集时，它们只需要使用由同一CA签名的证书，即可自动允许该节点加入群集。

1、生成节点证书

命令 elasticsearch-certutil 简化了生成证书的过程，它负责生成CA并与CA签署证书。

a、创建证书颁发机构CA
随便进入一个节点的bin 目录下执行elasticsearch-certutil 命令即可，如下

# 该命令输出单个文件，默认名称为elastic-stack-ca.p12。此文件是PKCS＃12密钥库
# 其中包含CA的公共证书和用于对每个节点的证书签名的私钥。
bin/elasticsearch-certutil ca

执行这个命令之后：

  • 会让你输入生成elastic-stack-ca.p12文件放在哪。（直接回车，放在当前目录）
  • 回车之后让你输入密码，该密码是让你保护文件和密钥的。如果你以后还要加集群的话，要记得输入的密码。

b、生成证书和私钥

# 此命令生成证书凭证，输出的文件是单个PKCS＃12密钥库，其中包括节点证书，节点密钥和CA证书。
bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

执行命令之后需要你操作3次：

  • 第一次，输入上面生成CA的密码，没有设置直接回车
  • 第二次，生成的文件路径，直接回车
  • 第三次，生成这次证书与私钥文件的密码，建议和上面生成CA一致（怕忘记密码，也可以直接回车）

如下图需要输入密码的地方：

命令执行完之后会生成一个elastic-certificates.p12 文件，这个就是各节点通信的凭证

只需要一个节点生成凭证即可。

2、配置证书

复制证书凭证

把证书凭证复制到各个节点一份

# 复制证书凭证到各个节点
cp elastic-certificates.p12 /opt/elasticsearch-cluster/elasticsearch-9301/config/
cp elastic-certificates.p12 /opt/elasticsearch-cluster/elasticsearch-9302/config/
cp elastic-certificates.p12 /opt/elasticsearch-cluster/elasticsearch-9303/config/
修改配置文件

在各个节点下的elasticsearch.yml文件添加如下配置

xpack.security.enabled: true
xpack.security.authc.accept_default_password: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /opt/elasticsearch-cluster/elasticsearch-9301/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /opt/elasticsearch-cluster/elasticsearch-9301/config/elastic-certificates.p12

要注意的是上面的path记得改成对应节点config下的elastic-certificates.p12。

添加密码到密码库

因为之前生成CA 和生成凭证都设置了密码，所以把密码添加到密钥库中

# 执行之后 输入上面设置的密码，回车即可
elasticsearch-9301/bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
elasticsearch-9301/bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

# 每个节点都要加
elasticsearch-9302/bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
elasticsearch-9302/bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

# 每个节点都要加
elasticsearch-9303/bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
elasticsearch-9303/bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

图片上有个警告大概的意思是说：ES未来将使用JDK11,而我现在的环境还是JDK8

之后启动各个节点

elasticsearch-9301/bin/elasticsearch -d
elasticsearch-9302/bin/elasticsearch -d
elasticsearch-9303/bin/elasticsearch -d

可以看看日志，不出意外集群启动成功了。随便请求一个节点地址：http://172.16.1.236:9201/
也可以使用elasticsearch-head连接查看，但是需要账户和密码访问
有的同学就要问了，我都没设置账号密码，去哪里看呢？

在安装Elasticsearch时，如果内置用户elastic用户没有密码，它将使用默认的引导密码。引导程序密码是一个临时密码，从随机 keystore.seed 设置派生的会在安装过程中添加到密钥库中。我们压根不知道密码是啥，所以需要为内置用户elastic设置密码。首次设置可以用elasticsearch-setup-passwords命令

Tip:下面的方法，我没试过，我没试过、我没试过，但是文档有，就提一下。
可以使用ES 提供的secure API重新加载为内置用户设置密码:

# 参考链接：https://www.elastic.co/guide/en/elasticsearch/reference/7.9/secure-settings.html#reloadable-secure-settings
POST _nodes/reload_secure_settings
{
  "secure_settings_password": "yourPassword" 
}

3、配置密码

elasticsearch-setup-passwords工具是首次设置内置用户密码的最简单方法。它使用elastic用户的引导程序密码来运行用户管理API请求。
执行命令如下：

bin/elasticsearch-setup-passwords interactive

它在“互动”模式下提示你输入：elastic，kibana_system，logstash_system，beats_system，apm_system，和remote_monitoring_user用户的密码


只需要在任意节点的bin目录下执行即可，不需要每个节点都执行。

至此ES集群的账号跟密码就设置完成了

我们设置密码之后会有一个名为.security-7的索引文档。

之后可以修改密码：

# 随便一个节点地址修改即可，一个集群共用一个账号密码
# 用Postman 请求时，选择 Authorization -> 选择 Basic Auth -> 右边选择上面设置的账号密码：elastic用户与密码
POST http://172.16.1.236:9201/_xpack/security/user/elastic/_password

{
  "password": "yourNewPassword"
}

4、踩坑记录

1、安装可能报错的问题：

查看文章链接：ES安装问题集锦

2、修改运行ES的Java环境

When starting ES7.9.0, it will prompt: future versions of Elasticsearch will require Java 11; your Java version from [/usr/local/jdk1.8.0/jre] does not meet this requirement That is to say, the future version of ES needs JDK11, and my current environment is that JDK8 does not meet the requirements. My package comes with JDK, so I simply specify the automatic JDK as the ES JDK operating environment:
modify the bin/elasticsearch file under the 3 nodes, and add the following at the front:

# Change the following path to the path of jdk under your es node
export JAVA_HOME=/opt/elasticsearch-cluster/elasticsearch-9301/jdk
export PATH=$JAVA_HOME/bin:$PATH

It can be started without modification, but it is recommended to change it. After all, the official package comes with it, so it is definitely recommended that we use the new version.

3. elastic-certificates.p12Step on the file location
  • Because I am just a host, I plan to save this certificate in another  elasticsearch-cluster folder config, but it is not satisfactory
  • A file permission problem was reported at startup. The error is as follows
  • It may be chmod 777 elastic-certificates.p12accessible, I haven't tried it, but it is still recommended to put it in the respective installation directory.

This article reference link

Guess you like

Origin blog.csdn.net/qq_32907195/article/details/131458674
Recommended
The “35-year-old curse” of Chinese coders
蘭雅 CorelDRAW 插件 2024.5.1 国际劳动节版，免费下载
Ranking
Methods and Practices of Manufacturing Data Quality Improvement
Serial port upgrade program for efficient transmission of large firmware
Use KITTI to run LIOSAM and complete the EVO evaluation
Calling methods on string literals (Java)
ActiveMQ and Spring integration (4)
You have to know the HTTPS! ! !
PAT whether Structures and Algorithms 7-4 with a binary search tree (40 rows streamlined structure)
el-upload brings request background
UVA - 1204 Fun Game-like pressure dp
2019-12-28
Daily
More
2024-05-04(18)
2024-05-03(8)
2024-05-02(0)
2024-05-01(4)
2024-04-30(36)
2024-04-29(5)
2024-04-28(12)
2024-04-27(29)
2024-04-26(22)
2024-04-25(32)

Code World

© 2018 codetd.com