ElasticSearch之集群搭建

1.ElasticSearch简介

        ElasticSearch 是一个高可用开源全文检索和分析组件，核心技术在于倒排索引，一般用于复杂搜索的应用或者上亿、上百亿的数据检索。ElasticSearch为每个Term建立对应的倒排索引，并提供Term Dictionary，可以通过二分法快速查询Term，就像查询字典一样，相比传统关系型数据库采用的B+/B-树更快。

2.部署环境

编号	IP	宿主OS	硬件配置	节点名称	节点类型
1	10.*.*.147	CentOs 7.4.1708	CPU：4core,Inter i5-4590,3.3GHz Memory：16G	test1	master+data
2	10.*.*.140	CentOs 7.4.1708	CPU：4core,Inter i5-4590,3.3GHz Memory：8G	est2	data
3	10.*.*.36	CentOs 7.4.1708	CPU：4core,Inter i5-4570,3.2GHz Memory：16G	test3	data

3、安装步骤

    先升级下软件包和内核

yum update

    关闭防火墙，此坑已踩

systemctl stop firewalld.service

3.1、JDK

yum search java|grep jdk

yum install java-1.8.0-openjdk(jdk version)

java -version

      若java -version验证失败，则需配置环境变量

#set java environment
JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64
JRE_HOME=$JAVA_HOME/jre
CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
export JAVA_HOME JRE_HOME CLASS_PATH PATH

若已安装jdk，且不满足es版本要求，则可以做如下操作：

---查看软连接
ll /usr/bin/java

---删除软连接
rm -rf /etc/alternatives/java

---新建软连接
ln -s /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4.x86_64/jre/bin/java /etc/alternatives/java

3.2、ES(es安装官方文档)

• 安装最新版本

(1)key init

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

(2)/etc/yum.repo.d目录下，新建文件elasticsearch.repo，内容如下(直接x即可)：

[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

(3)安装es

sudo yum install elasticsearch

• 安装指定版本

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.3.rpm
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.3.rpm.sha512
shasum -a 512 -c elasticsearch-6.2.3.rpm.sha512 
sudo rpm --install elasticsearch-6.2.3.rpm

后续安装ik、pinyin分词插件时遇到版本不一致问题，此坑已踩。可能需要安装shasum

yum install perl-Digest-SHA -y

3.3、集群参数配置

3.3.1、关键参数

参数名称	含义	默认值
cluster.name	集群名称	true
node.name	节点名称	true
node.master	是否可被选择主节点	true
node.data	是否为数据节点	true
discovery.zen.ping.multicast.enabled	是否启用多播	true
discovery.zen.ping.unicast.hosts	单播列表	["127.0.0.1", "[::1]"]
index.number_of_shards	分片数，创建后不可更改	5
index.number_of_replicas	副本数，可通过API修改	1

PS：

（1）生产环境中最好关闭多播

（2）节点数<=主分片数*（副本数+1），原则上最小分片数

3.3.2、节点类型

es节点类型由node.master、node.data决定，默认情况下均为true。

• master节点：表示节点是否具有成为主节点的资格
• data节点：表示该节点是否存储数据
• client节点：负载均衡节点，既不成为主节点，也不存储数据，此时两个参数均为false

配置文件路径为：/etc/elasticsearch.yml，jvm.options，log4j2.properties

3.3.3、elasticsearch.yml配置

路径：/etc/elasticsearch/

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: es.test
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: es.test.1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 10.*.*.147
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
transport.tcp.port: 9300
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["10.*.*.147","10.*.*.140","10.*.*.36"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
discovery.zen.minimum_master_nodes: 1
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type

3.4、节点启动

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service

sudo systemctl start elasticsearch.service
sudo systemctl stop elasticsearch.service

启动验证：ip:9200

3.5 elasticsearch-head安装

yum install git-core
yum install git npm
git clone https://github.com/mobz/elasticsearch-head.git
cd elasticsearch-head
yum install bzip2
npm install
nohup grunt server