[2022-09-05T14:35:19,438][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [n32] [controller/8954] [Main.cc@114] controller (64 bit): Version 6.8.12 (Build 39bf066b214253) Copyright (c) 2020 Elasticsearch BV
[2022-09-05T14:35:19,864][DEBUG][o.e.a.ActionModule ] [n32] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2022-09-05T14:35:20,080][INFO ][o.e.d.DiscoveryModule ] [n32] using discovery type [zen] and host providers [settings]
[2022-09-05T14:35:21,009][INFO ][o.e.n.Node ] [n32] initialized
[2022-09-05T14:35:21,009][INFO ][o.e.n.Node ] [n32] starting ...
[2022-09-05T14:35:21,473][INFO ][o.e.t.TransportService ] [n32] publish_address {172.20.251.32:9300}, bound_addresses {172.20.251.32:9300}
[2022-09-05T14:35:21,492][INFO ][o.e.b.BootstrapChecks ] [n32] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2022-09-05T14:35:21,496][ERROR][o.e.b.Bootstrap ] [n32] node validation exception
[2] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
[2]: memory locking requested for elasticsearch process but memory is not locked
[2022-09-05T14:35:21,498][INFO ][o.e.n.Node ] [n32] stopping ...
[2022-09-05T14:35:21,542][INFO ][o.e.n.Node ] [n32] stopped
[2022-09-05T14:35:21,542][INFO ][o.e.n.Node ] [n32] closing ...
[2022-09-05T14:35:21,558][INFO ][o.e.n.Node ] [n32] closedAccording to the prompt, modify /etc/security/limits.conf, add nofile to 65536
Modify /etc/sysctl.conf
vm.max_map_count = 655360sudo sysctl -p makes the parameters take effect, and restarts elasticsearch after exiting.
The error still exists.
1. Modify /etc/security/limits.d/20-nproc.conf, if there is no secondary file, modify the xx-nproc.conf file
# cat 20-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.
* soft nofile 65536
* hard nofile 65536
* soft nproc 65536
root soft nproc unlimitedAfter modification, re-enter, the parameters have not yet taken effect.
[k8s@slcj-rzcl-es elasticsearch]$ ulimit -Hn
4096
[k8s@slcj-rzcl-es elasticsearch]$ sudo su -
Last login: Mon Sep 5 14:53:57 CST 2022 on pts/0
[root@slcj-rzcl-es ~]# ulimit -Hn
65536
[root@slcj-rzcl-es ~]# Ordinary users and root users execute the same command, but the displayed results are inconsistent. That is, under the root user, it will take effect after the modification.
Continue to change the configuration file, replace * in /etc/security/limits.conf with the specific user name to be operated.
k8s soft nofile 65536
k8s hard nofile 65536
k8s soft nproc 65536
k8s hard nproc 65536Modify /etc/security/limits.d/20-nproc.conf, in this file, only nproc is configured
cat /etc/security/limits.d/20-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.
* soft nproc 65536
* hard nproc 65536
root soft nproc 65536New configuration file /etc/security/limits.d/def.conf
[root@slcj-rzcl-es ~]# cat /etc/security/limits.d/def.conf
k8s soft nofile 65536
k8s hard nofile 65536After the configuration is complete, the value of ulimit -Hn does not take effect when the common user is switched.
continue
Modify /etc/pam.d/login
Create a new /etc/pam.d/sshd file and add the following content
[root@slcj-rzcl-es ~]# cat /etc/pam.d/sshd
session required pam_limits.soModify the configuration file etc/ssh/ssd_config and release the comment of UsePAM
After switching to ordinary users, the authentication is successful
[k8s@slcj-rzcl-es elasticsearch]$
[k8s@slcj-rzcl-es elasticsearch]$ ulimit -Hn
65536
[k8s@slcj-rzcl-es elasticsearch]$
[k8s@slcj-rzcl-es elasticsearch]$
[k8s@slcj-rzcl-es elasticsearch]$
[k8s@slcj-rzcl-es elasticsearch]$ sudo su - root
Last login: Mon Sep 5 18:08:06 CST 2022 on pts/3
[root@slcj-rzcl-es ~]# ulimit -Hn
65536
[root@slcj-rzcl-es ~]#
[root@slcj-rzcl-es ~]# restart elasticsearch
[2022-09-05T18:05:23,660][INFO ][o.e.x.s.a.s.FileRolesStore] [n32] parsed [0] roles from file [/data/log4x/apps/elasticsearch/config/roles.yml]
[2022-09-05T18:05:24,235][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [n32] [controller/46559] [Main.cc@114] controller (64 bit): Version 6.8.12 (Build 39bf066b214253) Copyright (c) 2020 Elasticsearch BV
[2022-09-05T18:05:24,686][DEBUG][o.e.a.ActionModule ] [n32] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2022-09-05T18:05:24,865][INFO ][o.e.d.DiscoveryModule ] [n32] using discovery type [zen] and host providers [settings]
[2022-09-05T18:05:25,577][INFO ][o.e.n.Node ] [n32] initialized
[2022-09-05T18:05:25,578][INFO ][o.e.n.Node ] [n32] starting ...
[2022-09-05T18:05:25,719][INFO ][o.e.t.TransportService ] [n32] publish_address {172.20.251.32:9300}, bound_addresses {172.20.xx.xx:9300}
[2022-09-05T18:05:25,733][INFO ][o.e.b.BootstrapChecks ] [n32] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2022-09-05T18:05:28,780][INFO ][o.e.c.s.MasterService ] [n32] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {n32}{upc2qZ3vRFS8FNSgfsRU3Q}{I-bKOB_GQWWlVLdlGg3K6w}{172.20.xx.xx}{172.20.xx.xx:9300}{ml.machine_memory=33542516736, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2022-09-05T18:05:28,784][INFO ][o.e.c.s.ClusterApplierService] [n32] new_master {n32}{upc2qZ3vRFS8FNSgfsRU3Q}{I-bKOB_GQWWlVLdlGg3K6w}{172.20.xx.xx}{172.20.xx.xx:9300}{ml.machine_memory=33542516736, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {n32}{upc2qZ3vRFS8FNSgfsRU3Q}{I-bKOB_GQWWlVLdlGg3K6w}{172.20.xx.xx}{172.20.xx.xx:9300}{ml.machine_memory=33542516736, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2022-09-05T18:05:28,825][INFO ][o.e.h.n.Netty4HttpServerTransport] [n32] publish_address {172.20.251.32:9200}, bound_addresses {172.20.xx.xx:xx}
[2022-09-05T18:05:28,825][INFO ][o.e.n.Node ] [n32] started
[2022-09-05T18:05:29,076][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [n32] Failed to clear cache for realms [[]]
[2022-09-05T18:05:29,116][INFO ][o.e.g.GatewayService ] [n32] recovered [0] indices into cluster_state
[2022-09-05T18:05:29,242][INFO ][o.e.c.m.MetaDataIndexTemplateService] [n32] adding template [.watches] for index pattThe next day, after restarting elasticsearch, it was found that it could not be restarted again. Checking the log, it still reported an error with a small number of error files.
Check the above configuration files one by one, the parameters have been set, why is it invalid again?
because of not restarting init 6
After restarting, check the host parameters, ulimit -Hn, the parameters take effect.
So far, it's over.