ElasticSearch 7.6.2 version cluster construction and stepping on the pit

Environmental preparation

server description

This demonstration uses three RockyLinux 8.5 servers

server	IP	Remark
is-master	172.16.7.11	master node
es-node01	172.16.7.5	01 node
es-node02	172.16.7.13	02 node

kernel version

[root@es-master ~]# uname -a
Linux es-master 4.18.0-348.20.1.el8_5.x86_64 #1 SMP Thu Mar 10 20:59:28 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

modify system name

## 主节点
hostnamectl set-hostname es-master
## node1节点
hostnamectl set-hostname es-node01
## node2节点
hostnamectl set-hostname es-node02

configure hosts

All servers configure some /etc/hosts

172.16.7.11 es-master
172.16.7.5 es-node01
172.16.7.13 es-node02

Install JDK

##### 7.x开始内置了JDK不需要安装了，官网连接：
https://www.elastic.co/cn/support/matrix#logstash_plugins

create user

useradd cncfclub
passwd cncfclub
chown -R cncfclub:cncfclub  elasticsearch-7.6.2

download package

Download software

ES Chinese community download link:

Pick a version that you think is stable, I chose 7.6.2 here

Create a folder and transfer es to the folder and unzip it

mkdir -p /server/tools && cd /server/tools

Transfer the downloaded es installation package to three servers to

decompress and delete the compressed file

## 解压
tar xf elasticsearch-7.6.2-linux-x86_64.tar.gz
## 删除
rm -f elasticsearch-7.6.2-linux-x86_64.tar.gz

cluster deployment

Directory Structure Description

bin : script files, including ES startup & installation plugins, etc.

config : elasticsearch.yml (ES configuration file), jvm.options (JVM configuration file), log configuration file, etc.

JDK: Built-in JDK, JAVA_VERSION="12.0.1"

lib: class library

logs : log files

modules : all ES modules, including X-pack, etc.

plugins : ES installed plugins. no plugins by default

data : When ES starts, there will be this directory to store document data. The directory can be set

JVM optimization

According to the official "Heap Memory: Size and Exchange Editing": https://www.elastic.co/guide/cn/elasticsearch/guide/current/heap-sizing.html

The heap memory set after ES is installed by default is 1 GB, which is definitely less in actual use. The memory size of the server I am currently using for demonstration is 4GB, so I will not change it. The official said that if there is enough memory, there is no need to change it. Try not to exceed 32 GB. That is, each node memory allocation does not exceed 32 GB. Because it wastes memory, reduces CPU performance, and makes GC deal with large memory. If you want to keep it safe and reliable, setting the heap memory to 31 GB is a safe choice.

Elasticsearch.yml configuration file description

[root@es-master config]# cat elasticsearch.yml 
# 集群名称，设置一个统一的名字
cluster.name: cncfclub-es-cluster
# 节点名字，每个节点不同
node.name: es-master
# 是否有资格成为主节点
node.master: true
#是否存储数据
node.data: true
#设置最大集群节点数
node.max_local_storage_nodes: 3
#数据存储路径
path.data: /server/data
#日志存储路径
path.logs: /server/logs
# CentOS6 版本需要加入这两行
#bootstrap.memory_lock: true
#bootstrap.system_call_filter: false
#设置IP地址0.0.0.0 外网访问谁IP都能不然默认就是内网IP
network.host: 0.0.0.0
#设置端口
http.port: 9200
#集群通信接口
transport.tcp.port: 9300
#设置节点发现
discovery.seed_hosts: ["host1", "host2"]
#设置初始化集群选举
cluster.initial_master_nodes: ["node-1", "node-2"]

Master node elasticsearch.yml configuration

cluster.name: cncfclub-es-cluster
node.name: es-master
node.master: true
node.data: true
node.max_local_storage_nodes: 3
path.data: /server/data
path.logs: /server/logs
network.host: 0.0.0.0
http.port: 9200
transport.tcp.port: 9300
discovery.seed_hosts: ["es-master:9300", "es-node01:9300","es-node02:9300"]
cluster.initial_master_nodes: ["es-master", "es-node01","es-node02"]

node01 node elasticsearch.yml configuration

cluster.name: cncfclub-es-cluster
node.name: es-node01
node.master: true
node.data: true
node.max_local_storage_nodes: 3
path.data: /server/data
path.logs: /server/logs
network.host: 0.0.0.0
http.port: 9200
transport.tcp.port: 9300
discovery.seed_hosts: ["es-master:9300", "es-node01:9300","es-node02:9300"]
cluster.initial_master_nodes: ["es-master", "es-node01","es-node02"]

node02 node elasticsearch.yml configuration

cluster.name: cncfclub-es-cluster
node.name: es-node02
node.master: true
node.data: true
node.max_local_storage_nodes: 3
path.data: /server/data
path.logs: /server/logs
network.host: 0.0.0.0
http.port: 9200
transport.tcp.port: 9300
discovery.seed_hosts: ["es-master:9300", "es-node01:9300","es-node02:9300"]
cluster.initial_master_nodes: ["es-master", "es-node01","es-node02"]

Create data and log directories

mkdir -p /server/data && mkdir -p /server/logs全部赋权
#给有关es的
chown -R cncfclub:cncfclub /server

start service

## 进入cncfclub用户
su cncfclub
## 进入bin启动目录
cd /server/tools/elasticsearch-7.6.2/bin 
# 运行
./elasticsearch
# 后台运行
./elasticsearch -d

verify

head plugin

elasticsearch-head is a web front-end display plugin for browsing and interacting with Elasticsearch clusters. So elasticsearch-head is a client plugin for monitoring the status of Elasticsearch

Install the Multi elasticsearch head plug-in directly in a simple and rude browser

or docker run a web service

docker run -d -p 9100:9100 docker.io/mobz/elasticsearch-head

pit

坑一：max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

Solution

### 修改文件
vim /etc/sysctl.conf
 
### 添加参数 最后一行添加
vm.max_map_count = 262144

### 重启配置
sysctl -p

坑二：OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

### 修改
vim jvm.options
### 修改参数
-XX:+UseConcMarkSweepGC 改为 -XX:+UseG1GC

Pit 3: java.nio.file.AccessDeniedException: /server/tools/elasticsearch-7.6.2/config/elasticsearch.keystore

## 权限问题
chown -R cncfclub:cncfclub /server/tools/elasticsearch-7.6.2/config