[Betriebs- und Wartungswissensmaster] Zwei Methoden, Ein-Klick-Bereitstellung des ElasticSearch-Clusters (automatisierte Shell + Ansible-Bereitstellung)

Enterprise 2023-12-18 01:34:14 views: null

In diesem Artikel erfahren Sie, wie Sie mit Shell und Ansible ES-Cluster (ElasticSearch) mit einem Klick bereitstellen, Batch-Vorgänge bei der Cluster-Bereitstellung realisieren und den Charme der automatisierten Bereitstellung genießen.

Bereiten Sie den Host vor

CPU-Name IP CPU Scheibe Erinnerung
Ansible 10.0.0.61 2C 20G 2G
ELK104 10.0.0.104 2C 20G 2G
ELK105 10.0.0.105 2C 20G 2G
ELK106 10.0.0.106 2C 20G 2G

Shell-Bereitstellung mit einem Klick

Wir bereiten die erforderlichen Dateien auf Ansible vor (ES-Installationspaket ohne JDK und Oracles JDK) und schreiben Shell-Skripte in Ansible, um die Bereitstellung des ES-Clusters mit einem Klick zu erreichen. Tatsächlich gibt es viele Ideen für das Schreiben von Skripten. Hier verwende ich Do Dies sollten Sie zunächst vermeiden: Schreiben Sie nach Abschluss des Schlüssels eine for-Schleife, um eine Einzelpunktbereitstellung durchzuführen. Sie können einen Knoten auch direkt bereitstellen und kopieren.

Wenn Sie das Verzeichnis ändern möchten, müssen Sie die Skriptdateien, die Dateien elasticsearch.yml und es7.service ändern, sodass die Konfigurationsdateien gepackt und am Ende des Artikels platziert werden.

[root@Ansible ~]# cat deploy_ES.sh
#!/bin/bash
#1、先做免密钥，方便文件的推送以及远程执行命令
if [ -f /root/.ssh/id_rsa.pub ];then
	echo "公钥已经存在"
else
	ssh-keygen -t rsa
	echo "公钥不存在，自动创建公钥"
fi

hosts=(
10.0.0.104
10.0.0.105
10.0.0.106
)

for i in ${hosts[*]}
do
	expect <<EOF
      set timeout 10
      spawn ssh-copy-id -i /root/.ssh/id_rsa.pub root@$i
      expect {
        "yes/no" { send "yes\n";exp_continue }
        "password" { send "1\n" }
      }
      expect "password" { send "1\n" }
EOF
done

#2、准备配置文件



# 拼接IP字符串
ip_str=$(printf '", "%s' "${hosts[@]}")
ip_str=${ip_str:3}

# 替换文件中的IP地址
sed -i 's#^discovery.*#discovery.seed_hosts: \['"$ip_str"'"\]#g' elasticsearch.yml

sed -i 's#^cluster.initial_master_nodes.*#cluster.initial_master_nodes: \['"$ip_str"'"\]#g' elasticsearch.yml


#3、搭建ES集群

for q in ${hosts[*]}
do
	scp -r elasticsearch-7.17.5-no-jdk-linux-x86_64.tar.gz jdk-8u291-linux-x64.tar.gz elasticsearch.yml es7.service jvm.options jdk.sh $q:/root/
	ssh $q "mkdir -p /koten/{softwares,data,logs}"
	ssh $q "tar xf /root/elasticsearch-7.17.5-no-jdk-linux-x86_64.tar.gz -C /koten/softwares/"
	ssh $q "useradd koten"
	ssh $q "chown koten:koten /koten/softwares/elasticsearch-7.17.5/ -R"
	ssh $q "cat > /etc/security/limits.d/es.conf <<EOF
*          soft    nofile     65535
*          hard    nofile     131070
EOF
"	
	ssh $q "cat > /etc/sysctl.d/es.conf <<EOF
vm.max_map_count=262144
EOF"
	ssh $q "sysctl -f /etc/sysctl.d/es.conf"
	ssh $q "tar xf /root/jdk-8u291-linux-x64.tar.gz -C /koten/softwares/"
	ssh $q "\cp /root/elasticsearch.yml  /koten/softwares/elasticsearch-7.17.5/config/"
	ssh $q "\cp /root/es7.service /usr/lib/systemd/system/"
	ssh $q "\cp /root/jvm.options /koten/softwares/elasticsearch-7.17.5/config/"
	ssh $q "\cp /root/jdk.sh /etc/profile.d/"
	ssh $q "source  /etc/profile.d/jdk.sh"
	ssh $q "install -d /koten/{data,logs}/es7 -o koten -g koten"
done

#3、都部署好后再启动，防止脑裂
for e in ${hosts[*]}
do
	ssh $e "systemctl enable --now es7"
	if [ `ss -ntl | grep 19|wc -l` != 0 ];then
		echo "$e 主机ES启动成功！"
	fi	
done

[root@Ansible ~]# sh deploy_ES.sh
......
Created symlink from /etc/systemd/system/multi-user.target.wants/es7.service to /usr/lib/systemd/system/es7.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/es7.service to /usr/lib/systemd/system/es7.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/es7.service to /usr/lib/systemd/system/es7.service.

[root@ELK104 ~]# netstat -tnulp|grep 19
tcp6       0      0 :::19200                :::*                    LISTEN      2252/java           
tcp6       0      0 :::19300                :::*                    LISTEN      2252/java

 

Ansible-Batch-Bereitstellung

1. Initialisieren Sie das Rollenverzeichnis

[root@Ansible roles]# ansible-galaxy init es
- Role es was created successfully
[root@Ansible roles]# cd es 
[root@Ansible es]# pwd
/ansible/roles/es
[root@Ansible es]# ls
defaults  files  handlers  meta  README.md  tasks  templates  tests  vars

2. Schreiben Sie schlüsselfreie Playbooks für 10.0.0.104, 10.0.0.105 und 10.0.0.106

[root@Ansible es]# cat /etc/ansible/hosts | tail -4 
[es_group]
es104 ansible_ssh_host=10.0.0.104 ansible_ssh_user='root' ansible_ssh_pass='1'
es105 ansible_ssh_host=10.0.0.105 ansible_ssh_user='root' ansible_ssh_pass='1'
es106 ansible_ssh_host=10.0.0.106 ansible_ssh_user='root' ansible_ssh_pass='1'

[root@Ansible es]# cat ssh.yaml 
---
- hosts: es_group
  gather_facts: no
 
  tasks:
  - name: install ssh key
    authorized_key: user=root
                    key="{
   
   { lookup('file','/root/.ssh/id_rsa.pub')}}"
                    state=present

[root@Ansible es]# ansible-playbook ssh.yaml 

PLAY [es_group] ****************************************************************************************

TASK [install ssh key] *********************************************************************************
changed: [es106]
changed: [es105]
changed: [es104]

PLAY RECAP *********************************************************************************************
es104                      : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
es105                      : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
es106                      : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

3. Schreiben Sie ein Playbook für Daten-Push und Konfiguration

Wenn Sie den Host ändern müssen, müssen Sie nicht nur die Hosts-Datei, sondern auch Elasticsearch.yml ändern

[root@Ansible roles]# cat hosts | tail -4
[es_group]
10.0.0.104
10.0.0.105
10.0.0.106

[root@Ansible roles]# cat site.yml
- hosts: es_group
  roles:
    - role: es

[root@Ansible roles]# ls es/files/
elasticsearch-7.17.5-no-jdk-linux-x86_64.tar.gz  es7.service                 jdk.sh
elasticsearch.yml                                jdk-8u291-linux-x64.tar.gz  jvm.options

[root@Ansible roles]# cat es/tasks/main.yml 
- name: mkdir /koten/softwares /koten/data /koten/logs
  file: 
    path: "{
   
   { item }}"
    state: directory
  with_items:
    - /koten/softwares
    - /koten/data
    - /koten/logs

- name: tar xf elasticsearch-7.17.5-no-jdk-linux-x86_64.tar.gz jdk-8u291-linux-x64.tar.gz
  unarchive: 
    src: "{
   
   { item }}"
    dest: /koten/softwares
  with_items:
    - elasticsearch-7.17.5-no-jdk-linux-x86_64.tar.gz
    - jdk-8u291-linux-x64.tar.gz

- name: Create koten Group
  group:
    name: koten

- name: Create koten User
  user:
    name: koten

- name: chown -R koten.koten /koten/softwares/elasticsearch-7.17.5/
  file:
    path: /koten/softwares/elasticsearch-7.17.5/
    owner: koten
    group: koten
    recurse: yes

- name: copy content to /etc/security/limits.d/es.conf
  copy:
    content: | 
      *          soft    nofile     65535
      *          hard    nofile     131070
    dest: /etc/security/limits.d/es.conf

- name: copy test content to /etc/sysctl.d/es.conf
  copy:
    content: |
      vm.max_map_count=262144
    dest: /etc/sysctl.d/es.conf

- name: sysctl -f /etc/sysctl.d/es.conf
  command: sysctl -f /etc/sysctl.d/es.conf

- name: push es7.service to /usr/lib/systemd/system/
  copy: 
    src: es7.service
    dest: /usr/lib/systemd/system/

- name: push jvm.options to /koten/softwares/elasticsearch-7.17.5/config/
  copy:
    src: jvm.options
    dest: /koten/softwares/elasticsearch-7.17.5/config/

- name: push elasticsearch.yml to /koten/softwares/elasticsearch-7.17.5/config/
  copy:
    src: elasticsearch.yml
    dest: /koten/softwares/elasticsearch-7.17.5/config/elasticsearch.yml

- name: push jdk.sh to /etc/profile.d/
  copy:
    src: jdk.sh
    dest: /etc/profile.d/

- name: source /etc/profile.d/jdk.sh
  command: bash -c 'source /etc/profile.d/jdk.sh'

- name: mkdir /koten/data/es7 /koten/logs/es7
  file: 
    path: "{
   
   { item }}"
    state: directory
    owner: koten
    group: koten
  with_items:
    - /koten/data/es7
    - /koten/logs/es7

- name: systemctl enable --now es7
  systemd:
    name: es7
    state: started
    enabled: yes

4. Führen Sie das Daten-Push- und Konfigurations-Playbook aus und stellen Sie den ES-Cluster bereit

[root@Ansible roles]# ansible-playbook site.yml 

PLAY [es_group] ****************************************************************

TASK [Gathering Facts] *********************************************************
ok: [es105]
ok: [es106]
ok: [es104]

TASK [mkdir /koten/softwares /koten/data /koten/logs] **************************
changed: [es105] => (item=/koten/softwares)
changed: [es104] => (item=/koten/softwares)
changed: [es106] => (item=/koten/softwares)
changed: [es106] => (item=/koten/data)
changed: [es104] => (item=/koten/data)
changed: [es105] => (item=/koten/data)
changed: [es106] => (item=/koten/logs)
changed: [es105] => (item=/koten/logs)
changed: [es104] => (item=/koten/logs)

TASK [es : tar xf elasticsearch-7.17.5-no-jdk-linux-x86_64.tar.gz jdk-8u291-linux-x64.tar.gz] ***
changed: [es105] => (item=elasticsearch-7.17.5-no-jdk-linux-x86_64.tar.gz)
changed: [es104] => (item=elasticsearch-7.17.5-no-jdk-linux-x86_64.tar.gz)
changed: [es106] => (item=elasticsearch-7.17.5-no-jdk-linux-x86_64.tar.gz)
changed: [es105] => (item=jdk-8u291-linux-x64.tar.gz)
changed: [es106] => (item=jdk-8u291-linux-x64.tar.gz)
changed: [es104] => (item=jdk-8u291-linux-x64.tar.gz)

TASK [es : Create koten Group] *************************************************
changed: [es105]
changed: [es106]
changed: [es104]

TASK [es : Create koten User] **************************************************
changed: [es104]
changed: [es105]
changed: [es106]

TASK [chown -R koten.koten /koten/softwares/elasticsearch-7.17.5/] *************
changed: [es104]
changed: [es105]
changed: [es106]

TASK [copy content to /etc/security/limits.d/es.conf] **************************
changed: [es104]
changed: [es106]
changed: [es105]

TASK [copy test content to /etc/sysctl.d/es.conf] ******************************
changed: [es106]
changed: [es104]
changed: [es105]

TASK [sysctl -f /etc/sysctl.d/es.conf] *****************************************
changed: [es106]
changed: [es104]
changed: [es105]

TASK [push es7.service to /usr/lib/systemd/system/] ****************************
changed: [es104]
changed: [es105]
changed: [es106]

TASK [push jvm.options to /koten/softwares/elasticsearch-7.17.5/config/] *******
changed: [es104]
changed: [es105]
changed: [es106]

TASK [push elasticsearch.yml to /koten/softwares/elasticsearch-7.17.5/config/] ***
changed: [es104]
changed: [es106]
changed: [es105]

TASK [es : push jdk.sh to /etc/profile.d/] *************************************
changed: [es104]
changed: [es105]
changed: [es106]

TASK [es : source /etc/profile.d/jdk.sh] ***************************************
changed: [es104]
changed: [es106]
changed: [es105]

TASK [mkdir /koten/data/es7 /koten/logs/es7] ***********************************
changed: [es104] => (item=/koten/data/es7)
changed: [es105] => (item=/koten/data/es7)
changed: [es106] => (item=/koten/data/es7)
changed: [es105] => (item=/koten/logs/es7)
changed: [es104] => (item=/koten/logs/es7)
changed: [es106] => (item=/koten/logs/es7)

TASK [systemctl enable --now es7] **********************************************
changed: [es106]
changed: [es104]
changed: [es105]

PLAY RECAP *********************************************************************
es104                      : ok=16   changed=15   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
es105                      : ok=16   changed=15   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
es106                      : ok=16   changed=15   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   



# 查看es的状态
[root@ELK104 ~]# systemctl status es7.service 
● es7.service - es7
   Loaded: loaded (/usr/lib/systemd/system/es7.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2023-10-27 14:01:54 CST; 42s ago
 Main PID: 3578 (java)
   CGroup: /system.slice/es7.service
           ├─3578 /koten/softwares/jdk1.8.0_291/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=6...
           └─3711 /koten/softwares/elasticsearch-7.17.5/modules/x-pack-ml/platform/linux-x86_64/bin/c...

Oct 27 14:02:19 ELK104 elasticsearch[3578]: [2023-10-27T14:02:19,716][INFO ][o.e.p.PluginsService ...wn]
Oct 27 14:02:19 ELK104 elasticsearch[3578]: [2023-10-27T14:02:19,716][INFO ][o.e.p.PluginsService ...ql]
Oct 27 14:02:19 ELK104 elasticsearch[3578]: [2023-10-27T14:02:19,716][INFO ][o.e.p.PluginsService ...ck]
Oct 27 14:02:19 ELK104 elasticsearch[3578]: [2023-10-27T14:02:19,717][INFO ][o.e.p.PluginsService ...re]
Oct 27 14:02:19 ELK104 elasticsearch[3578]: [2023-10-27T14:02:19,717][INFO ][o.e.p.PluginsService ...de]
Oct 27 14:02:19 ELK104 elasticsearch[3578]: [2023-10-27T14:02:19,749][INFO ][o.e.p.PluginsService ...er]
Oct 27 14:02:19 ELK104 elasticsearch[3578]: [2023-10-27T14:02:19,750][INFO ][o.e.p.PluginsService ...ded
Oct 27 14:02:19 ELK104 elasticsearch[3578]: [2023-10-27T14:02:19,958][INFO ][o.e.e.NodeEnvironment...fs]
Oct 27 14:02:19 ELK104 elasticsearch[3578]: [2023-10-27T14:02:19,959][INFO ][o.e.e.NodeEnvironment...ue]
Oct 27 14:02:20 ELK104 elasticsearch[3578]: [2023-10-27T14:02:20,074][INFO ][o.e.n.Node           ...st]
Hint: Some lines were ellipsized, use -l to show in full.

 

Link zum Herunterladen der Paketdatei „deploy_es_shell“: https://pan.baidu.com/s/1l22I0aSB-50NXlh4lYZArw?pwd=umbm

Link zum Herunterladen der Paketdatei „deploy_es_ansible“: https://pan.baidu.com/s/1lIfYWMnhsg7yoyzgkBSEDA?pwd=8hpl

Mein Name ist Koten. Ich verfüge über 10 Jahre Betriebs- und Wartungserfahrung. Ich teile weiterhin Betriebs- und Wartungstipps. Vielen Dank fürs Lesen und Aufpassen!

Supongo que te gusta

Origin blog.csdn.net/qq_37510195/article/details/130855070
Recomendado
Clasificación
Diario
Más
2024-05-12(22)
2024-05-11(31)
2024-05-10(32)
2024-05-09(31)
2024-05-08(18)
2024-05-07(35)
2024-05-06(4)
2024-05-05(0)
2024-05-04(17)
2024-05-03(8)

Code World

© 2018 codetd.com