第12篇:ElasticSearch集群搭建

背景:目前国内有大量的公司都在使用 Elasticsearch,包括阿里、京东、滴滴、今日头条、小米、vivo等诸多知名公司。除了搜索功能之外,Elasticsearch还结合Kibana、Logstash、Elastic Stack还被广泛运用在大数据近实时分析领域,包括日志分析、指标监控等多个领域。 

本节内容:今天终于迎来了大家期待已久的ElasticSearch集群搭建。

目录

1、Elastcsearch版本下载及安装

2、本地集群搭建步骤

2.1  节点复制

2.2 节点YML配置修改

2.2.1 节点1配置修改(7.9.3)

2.2.2 节点2配置修改(7.9.3.2)

2.2.3 节点3配置修改(7.9.3.3)

2.3  节点启动

2.4  Head验证集群


前提说明:实际生产业务场景下,集群模式会有多台物理机支撑。但由于老王机器有限,所以就搭建一个伪集群环境,即在一台MAC上部署3个应用,但原理与多台机器是类似的。

1、Elastcsearch版本下载及安装

我用的是Elasticsearch-7.9.3版本,下载及安装过程参考前面文章 Elasticsearch各版本特性

2、本地集群搭建步骤

2.1  节点复制

通过压缩包解压出三份,分别命名为:

elasticsearch-7.9.3、

elasticsearch-7.9.3.2、

elasticsearch-7.9.3.3。

注意:一定要使用干净的压缩文件解压出3份,防止数据污染。

如下图所示,

2.2 节点YML配置修改

主要针对解压目录下的 config/elasticsearch.yml 进行修改。

主要修改以下部分:

#集群名,所有节点需要保持集群名一样

cluster.name: my-cluster

#节点名,每个节点名称不同

node.name: node01

#host配置

network.host: 127.0.0.1

#HTTP 请求端口,每个节点不同

http.port: 9201

#TCP请求端口,每个节点不同

transport.tcp.port: 9301

#设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1,对于大的集群来说,可以设置大一点的值(2-4)

discovery.zen.minimum_master_nodes: 2

#首先遍历discovery.seed_hosts 以及最后通信的cluster的符合主节点选举条件的主机列表

discovery.seed_hosts: ["127.0.0.1:9301", "127.0.0.1:9302", "127.0.0.1:9303"]

#节点ping超时时间设置

discovery.zen.fd.ping_timeout: 1m

#节点ping重试次数

discovery.zen.fd.ping_retries: 5

# es7.x 之后新增的配置,初始化一个新的集群时,需要此配置来选举master

cluster.initial_master_nodes: ["node01", "node02", "node03"]

# 跨域设置

http.cors.enabled: true

http.cors.allow-origin: "*"

#节点设置,既可以是主节点,又可以为数据节点

node.master: true

node.data: true

2.2.1 节点1配置修改(7.9.3)

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-cluster
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node01
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 127.0.0.1
#
# Set a custom port for HTTP:
#
http.port: 9201
transport.tcp.port: 9301
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
# 设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1,对于大的集群来说,可以设置大一点的值(2-4)
discovery.zen.minimum_master_nodes: 3
discovery.seed_hosts: ["127.0.0.1:9301", "127.0.0.1:9302", "127.0.0.1:9303"] 
discovery.zen.fd.ping_timeout: 1m
discovery.zen.fd.ping_retries: 5
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
# es7.x 之后新增的配置,初始化一个新的集群时需要此配置来选举master
cluster.initial_master_nodes: ["node01", "node02", "node03"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true
node.data: true
 

2.2.2 节点2配置修改(7.9.3.2)

# ======================== Elasticsearch Configuration =========================

#

# NOTE: Elasticsearch comes with reasonable defaults for most settings.

# Before you set out to tweak and tune the configuration, make sure you

# understand what are you trying to accomplish and the consequences.

#

# The primary way of configuring a node is via this file. This template lists

# the most important settings you may want to configure for a production cluster.

#

# Please consult the documentation for further information on configuration options:

# https://www.elastic.co/guide/en/elasticsearch/reference/index.html

#

# ---------------------------------- Cluster -----------------------------------

#

# Use a descriptive name for your cluster:

#

cluster.name: my-cluster

#

# ------------------------------------ Node ------------------------------------

#

# Use a descriptive name for the node:

#

node.name: node02

#

# Add custom attributes to the node:

#

#node.attr.rack: r1

#

# ----------------------------------- Paths ------------------------------------

#

# Path to directory where to store the data (separate multiple locations by comma):

#

#path.data: /path/to/data

#

# Path to log files:

#

#path.logs: /path/to/logs

#

# ----------------------------------- Memory -----------------------------------

#

# Lock the memory on startup:

#

#bootstrap.memory_lock: true

#

# Make sure that the heap size is set to about half the memory available

# on the system and that the owner of the process is allowed to use this

# limit.

#

# Elasticsearch performs poorly when the system is swapping the memory.

#

# ---------------------------------- Network -----------------------------------

#

# Set the bind address to a specific IP (IPv4 or IPv6):

#

network.host: 127.0.0.1

#

# Set a custom port for HTTP:

#

http.port: 9202

transport.tcp.port: 9302

#

# For more information, consult the network module documentation.

#

# --------------------------------- Discovery ----------------------------------

#

# Pass an initial list of hosts to perform discovery when this node is started:

# The default list of hosts is ["127.0.0.1", "[::1]"]

#

# 设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1,对于大的集群来说,可以设置大一点的值(2-4)

discovery.zen.minimum_master_nodes: 2

# es7.x 之后新增的配置,写入候选主节点的设备地址,在开启服务后可以被选为主节点

discovery.seed_hosts: ["127.0.0.1:9301", "127.0.0.1:9302", "127.0.0.1:9303"]

discovery.zen.fd.ping_timeout: 1m

discovery.zen.fd.ping_retries: 5

#

# Bootstrap the cluster using an initial set of master-eligible nodes:

#

# es7.x 之后新增的配置,初始化一个新的集群时需要此配置来选举master

cluster.initial_master_nodes: ["node01", "node02", "node03"]

#

# For more information, consult the discovery and cluster formation module documentation.

#

# ---------------------------------- Gateway -----------------------------------

#

# Block initial recovery after a full cluster restart until N nodes are started:

#

#gateway.recover_after_nodes: 3

#

# For more information, consult the gateway module documentation.

#

# ---------------------------------- Various -----------------------------------

#

# Require explicit names when deleting indices:

#

#action.destructive_requires_name: true

http.cors.enabled: true

http.cors.allow-origin: "*"

node.master: true

node.data: true

2.2.3 节点3配置修改(7.9.3.3)

# ======================== Elasticsearch Configuration =========================

#

# NOTE: Elasticsearch comes with reasonable defaults for most settings.

# Before you set out to tweak and tune the configuration, make sure you

# understand what are you trying to accomplish and the consequences.

#

# The primary way of configuring a node is via this file. This template lists

# the most important settings you may want to configure for a production cluster.

#

# Please consult the documentation for further information on configuration options:

# https://www.elastic.co/guide/en/elasticsearch/reference/index.html

#

# ---------------------------------- Cluster -----------------------------------

#

# Use a descriptive name for your cluster:

#

cluster.name: my-cluster

#

# ------------------------------------ Node ------------------------------------

#

# Use a descriptive name for the node:

#

node.name: node03

#

# Add custom attributes to the node:

#

#node.attr.rack: r1

#

# ----------------------------------- Paths ------------------------------------

#

# Path to directory where to store the data (separate multiple locations by comma):

#

#path.data: /path/to/data

#

# Path to log files:

#

#path.logs: /path/to/logs

#

# ----------------------------------- Memory -----------------------------------

#

# Lock the memory on startup:

#

#bootstrap.memory_lock: true

#

# Make sure that the heap size is set to about half the memory available

# on the system and that the owner of the process is allowed to use this

# limit.

#

# Elasticsearch performs poorly when the system is swapping the memory.

#

# ---------------------------------- Network -----------------------------------

#

# Set the bind address to a specific IP (IPv4 or IPv6):

#

network.host: 127.0.0.1

#

# Set a custom port for HTTP:

#

http.port: 9203

transport.tcp.port: 9303

#

# For more information, consult the network module documentation.

#

# --------------------------------- Discovery ----------------------------------

#

# Pass an initial list of hosts to perform discovery when this node is started:

# The default list of hosts is ["127.0.0.1", "[::1]"]

#

discovery.zen.minimum_master_nodes: 2

# es7.x 之后新增的配置,写入候选主节点的设备地址,在开启服务后可以被选为主节点

discovery.seed_hosts: ["127.0.0.1:9301", "127.0.0.1:9302", "127.0.0.1:9303"]

discovery.zen.fd.ping_timeout: 1m

discovery.zen.fd.ping_retries: 5

#

# Bootstrap the cluster using an initial set of master-eligible nodes:

#

#cluster.initial_master_nodes: ["node-1", "node-2"]

cluster.initial_master_nodes: ["node01", "node02", "node03"]

#

# For more information, consult the discovery and cluster formation module documentation.

#

# ---------------------------------- Gateway -----------------------------------

#

# Block initial recovery after a full cluster restart until N nodes are started:

#

#gateway.recover_after_nodes: 3

#

# For more information, consult the gateway module documentation.

#

# ---------------------------------- Various -----------------------------------

#

# Require explicit names when deleting indices:

#

#action.destructive_requires_name: true

http.cors.enabled: true

http.cors.allow-origin: "*"

node.master: true

node.data: true

2.3  节点启动

如果之前启动过Elasticsearch节点,则需要先停止。

停止命令,

$ jps | grep Elasticsearch  #查询es进程
76665 Elasticsearch
$  kill -SIGTERM 76665 #杀死指定进程

启动命令,

$ cd elasticsearch-7.9.3   #根目录

$ ./bin/elasticsearch -d     #启动命令

2.4  Head验证集群

如果还没有安装head,请参考前面的内容 Mac/Windows安装ElasticSearch可视化Head插件

 出现以上内容,则表示集群安装成功。

看主节点信息,表示node01被选为了主节点。

除此之外,在head页面还可以查看索引,数据简单查询、数据复合查询。基本能满足我们日常查询需要。

至此,本地集群搭建完成。

猜你喜欢

转载自blog.csdn.net/wangyongfei5000/article/details/125110130