ElasticSearch cluster + kibana deployment

Preparation before installation

Turn off firewall and selinux

Turn off firewall

#查看防火墙状态： 
sudo systemctl status firewalld
#执行关闭命令： 
sudo systemctl stop firewalld
#执行开机禁用防火墙自启命令： 
sudo systemctl disable firewalld
#关于防火墙的其他命令：
#启动：
sudo systemctl start firewalld
#防火墙随系统开启启动：
sudo systemctl enable firewalld 

Turn off selinux

sudo sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
sudo cat /etc/selinux/config | grep SELINUX

#临时关闭(修改文件后，再临时关闭可以不用重启)
setenforce 0

Modify virtual machine memory and swappiness kernel parameters

sudo vim /etc/sysctl.conf
修改：
vm.max_map_count=262144

swappiness kernel parameters:

session    required      pam_limits.so执行命令：
sudo /usr/sbin/sysctl -w vm.swappiness=0
#添加swappiness参数设置：
sudo sh -c 'echo \"vm.swappiness = 0\" >> /etc/sysctl.conf'
#"sh -c"目的是将后面‘’的中的命令当作一句话处理，因为echo需要sudo，>>也需要sudo权限。
#使配置生效
sudo sysctl -p

Modify the maximum number of file handles (socket connections) in Linux

#查看当前设置：
ulimit -n  #默认是1024

#永久方法：（修改以下两个文件）
vi /etc/security/limits.conf，加上：
*               soft    nofile          65536 #通常soft值应小于等于hard值
*               hard    nofile          65536 #
*               soft    nproc           4096
*               hard    nproc           4096

vi /etc/pam.d/login，在后面加上：
session    required      pam_limits.so

1. Es deployment

1.1. Unzip the installation package

#解压安装包到指定位置
tar -zxvf /opt/software/elasticsearch-7.12.0-linux-x86_64.tar.gz -C /opt/module/
#修改所属组及所属者
chown -R bigdata:bigdata /opt/module/elasticsearch-7.12.0

1.2. Import ik word segmenter

#在本地解压elasticsearch-analysis-ik-7.12.0.zip
unzip elasticsearch-analysis-ik-7.12.0.zip
#移动到/opt/module/elasticsearch-7.12.0/plugins 目录
mv elasticsearch-analysis-ik-7.12.0 /opt/module/elasticsearch-7.12.0/plugins/
#分发到所有节点
scp /opt/module/elasticsearch-7.12.0 node2:/opt/module/

1.3. Modify configuration file

vi /opt/module/elasticsearch-7.12.0/config/elasticsearch.yml

1.4. Verification

Start ES

/opt/module/elasticsearch-7.12.0/bin/elasticsearch

View cluster status

curl -X GET "node1:9200/_cat/health?v&pretty"

Background process

./elasticsearch -d

2. Kibana deployment

2.1 Unzip the installation package and modify the configuration

#将安装包解压到指定目录
sudo tar -zxvf /opt/software/kibana-7.12.0-linux-x86_64.tar.gz -C /opt/module/
#修改所属组和所属用户（注意：当前目录所属一定不要是root）
sudo chown -R bigdata:bigdata /opt/module/kibana-7.12.0-linux-x86_64
#修改配置文件
vi /opt/module/kibana-7.12.0-linux-x86_64/config/kibana.yml

#创建kibana日志目录并修改所属组和所属者
mkdir /data/log/kibana
chown -R bigdata:bigdata /data/log/kibana

2.2 Verification

切换到Kibana的bin目录下执行
./kibana
登录http://node1:5601验证是否启动成功
也可用 ps -ef |grep kibana 查看

Background process

nohup /opt/module/kibana-7.12.0-linux-x86_64/bin/kibana > /data/log/kibana/kibanaout.txt 2>&1 &

3. Use systemctl to manage es cluster and kibana

3.1 Use systemctl to manage es cluster

Create elasticsearch.service on node1

sudo vim /usr/lib/systemd/system/elasticsearch.service

[Unit]
Description=elasticsearch
After=network.target

[Service]
Type=simple
User=bigdata
Group=bigdata
Restart=on-failure
Environment=ES_HOME=/opt/module/elasticsearch-7.12.0
Environment=PID_DIR=/data/ES
ExecStart=/opt/module/elasticsearch-7.12.0/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet
SuccessExitStatus=0 143
[Install]
WantedBy=multi-user.target

Distribute to other nodes
to start and shut down to verify whether the configuration is successful.

sudo systemctl start elasticsearch
sudo systemctl stop elasticsearch
sudo systemctl status elasticsearch


#验证成功配置开机自启
sudo systemctl enable elasticsearch

Create a cluster script on node1

vim /opt/bash/elasticsearch.sh

#!/bin/bash
# Elasticsearch节点列表
# 在这里修改为自己的实际节点IP地址
nodes=(node1 node2 node3 node4)

#循环遍历所有节点执行相同的命令
cmd=$1
if [ "$cmd" != "" ];then
       if [[ "start" == "$cmd" ]] || [[ "stop" == "$cmd" ]] || [[ "restart" == "$cmd" ]] || [[ "status" == "$cmd" ]] ;then
              for host in ${nodes[@]} ; do
                  echo "=================== $host =================="
                  ssh $host "sudo systemctl $cmd elasticsearch"
                  echo "$cmd Elasticsearch on $host success..."
              done
        else
               echo "  输入的参数不对"
               echo "  start   启动Elasticsearch集群"     
               echo "  stop    停止Elasticsearch集群"
               echo "  restart 重启Elasticsearch集群"
               echo "  status  查看Elasticsearch集群"
       fi
else
       echo "请传入一个参数(start|stop|restart|status)"
fi

#添加执行权限
chmod +x /opt/bash/elasticsearch.sh
#脚本使用
/opt/bash/elasticsearch.sh start/stop/status/restart

3.2 Use sytemctl to manage Kibana

Create kibana.service on node1

 vim /usr/lib/systemd/system/kibana.service

[Unit]
Description=Kibana
After=network-online.target

[Service]
Type=simple
User=bigdata
Group=bigdata
ExecStart=/opt/module/kibana-7.12.0-linux-x86_64/bin/kibana
Restart=always

[Install]
WantedBy=multi-user.target

#启动、关闭验证是否配置成功
sudo systemctl start kibana
sudo systemctl stop kibana
sudo systemctl status kibana
#验证成功配置开机自启
sudo systemctl enable kibana