Environmental deployment
1. Server preparation: centos7.4 system, Jdk1.8 CAT / etc / RedHat-Release . Es6.x
If it comes openjdk need to uninstall:
rpm -qa | grep java
rpm -e –nodeps *
2.ElasticSearch installation: I used version 6.6.0 https://elasticsearch.cn/download/
( 1 ) Address: https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.6.0.tar.gz
（2）解压：tar -zxvf elasticsearch/elasticsearch-6.6.0.tar.gz
(3) modify the system configuration:
- Setting kernel parameters: vim /etc/sysctl.conf
Add the following:
fs.file-max = 65536
vm.max_map_count = 262144
sysctl -p refresh configuration,
sysctl -a to see if the entry into force,
If unsuccessful (es start or failure, not everyone get to touch, as if to 7.6 met):
rm -f /sbin/modprobe
ln -s /bin/true /sbin/modprobe
rm -f /sbin/sysctl
ln -s /bin/true /sbin/sysctl
-
Setting resource parameters
vi /etc/security/limits.conf
# add at the contents:
* Soft nofile 65536
* Hard nofile 131072
* Soft nproc 2048
* 4096 Hard nproc -
The number of modified process
vi /etc/security/limits.d/20-nproc.conf
* Soft nproc 4096
After configuration window to turn off the link, re-open a
can start with root user -
Es modify the configuration file: vim elasticsearch installation path /config/elasticsearch.yml
# Configure cluster name es, es es automatically discovers in the same network segment, if there are multiple clusters in the same network segment, you can use this property to distinguish between different clusters.
cluster.name: BI-cluster
# name node
node.name: node-master
catalog # store data
path.data: / home / elasticsearch / data
directory stored log #
path.logs: / home / elasticsearch / logs
# ip address as well as set the binding ip address of the node and other nodes interact
network.host: 0.0.0.0
# specify the http port, you use port head, kopf other related plug-ins used
http.port: 9200
# set the interaction between nodes tcp port, the default is 9300
transport.tcp.port: 9300
# set the initial list of master node in the cluster can be used to automatically discover new nodes join the cluster through these nodes.
discovery.zen.ping.unicast.hosts : [ "10.108.4.203:9300", "10.108.4.204:9300", "10.108.4.205:9300"]
# Without this setting, suffered a network failure cluster is likely to be clustered into two separate clusters - split brain - This will lead to data loss
discovery.zen.minimum_master_nodes: 3
http.cors.enabled: # es5.x to true version of the above needs, head to access
http.cors.allow-Origin: "*"
bootstrap.memory_lock: false # Some systems require because centos6.x the operating system does not support SecComp, and elasticsearch 5.5.2 default bootstrap.system_call_filter detection to true, resulting in detection failure led directly to the ES can not be started after the failure.
My configuration is as follows:# ======================== Elasticsearch Configuration ==================== ===== # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure y ou # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lis ts # the most important settings you may want to configure for a production cl uster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ------------------------------ ----- # # Use a descriptive name for your cluster: # #cluster.name: my-application # # ------------------------------------ Node ------------------------------- ----- # # Use a descriptive name for the node: # #node.name: node-1 # # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------- ----- # # Path to directory where to store the data (separate multiple locations by comma): # #path.data: /path/to/data # # Path to log files: # #path.logs: /path/to/logs # # ----------------------------------- Memory ------------------------------ ----- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ------------------------------ ----- # # Set the bind address to a specific IP (IPv4 or IPv6): # #network.host: 192.168.0.1 # # Set a custom port for HTTP: # #http.port: 9200 # # For more information, consult the network module documentation. # # --------------------------------- Discovery ----------------------------- ----- # # Pass an initial list of hosts to perform discovery when new node is start ed: # The default list of hosts is ["127.0.0.1", "[::1]"] # #discovery.zen.ping.unicast.hosts: ["host1", "host2"] # # Prevent the "split brain" by configuring the majority of nodes (total num ber of master-eligible nodes / 2 + 1): # #discovery.zen.minimum_master_nodes: # # For more information, consult the zen discovery module documentation. # # ---------------------------------- Gateway ------------------------------ ----- # # Block initial recovery after a full cluster restart until N nodes are sta rted: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ------------------------------ ----- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true cluster.name: my-es node.name: node-128 network.host: 0.0.0.0 http.port: 19200 transport.tcp.port: 19300 #discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300"] #discovery.zen.minimum_master_nodes: 3 http.cors.enabled: true http.cors.allow-origin: "*" bootstrap.memory_lock: false bootstrap.system_call_filter: false -
Start es: ./ bin / elasticsearch background start raising -d parameter
-
Open your browser to http: // your ip: 19200 / to see if can be accessed, see the following screen indicating that the startup ok
If the external network access fails, follow these steps:
- Check the following three at your elasticsearch.yml configuration file is configured correctly
network.host: 0.0.0.0
bootstrap.memory_lock: false bootstrap.system_call_filter: false
- Check the firewall is turned off (turn off the firewall needs outside the network can access)
Open, restart, shut down, firewalld.service service
# Enable
Service firewalld Start
# Restart
Service firewalld restart
# close the
service firewalld stop
to view the firewall rules:firewall-cmd --list-all
查询、开放、关闭端口：# 查询端口是否开放
执行以上步骤之后就可以在外网访问了。以下是我遇到的启动错误的问题以及解决办法：
firewall-cmd --query-port=8080/tcp
# 开放80端口
firewall-cmd --permanent --add-port=80/tcp
# 移除端口
firewall-cmd --permanent --remove-port=8080/tcp
#重启防火墙(修改配置后要重启防火墙)
firewall-cmd --reload - （1）max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
每个进程最大同时打开文件数太小，可通过下面2个命令查看当前数量 -
ulimit -Hn ulimit -Sn
修改/etc/security/limits.conf文件，增加配置，用户退出后重新登录生效
* soft nofile 65536 * hard nofile 65536
（2）max number of threads [3818] for user [es] is too low, increase to at least [4096]
问题同上，最大线程个数太低。修改配置文件/etc/security/limits.conf（和问题1是一个文件），增加配置
* soft nproc 4096 * hard nproc 4096
可通过命令查看
ulimit -Hu ulimit -Su
修改后的文件：
如果更改后发现ulimit -Su 的结果没有变化的话，此时需要在这个文件中添加如下信息：(admin是我自己的用户名称，可以根据自己的实际用户更改)
admin - nproc 4096
-
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
修改/etc/sysctl.conf文件，增加配置vm.max_map_count=262144
vi /etc/sysctl.conf
sysctl -p执行命令sysctl -p生效
4、Exception in thread "main" java.nio.file.AccessDeniedException: /usr/local/elasticsearch/elasticsearch-6.2.2-1/config/jvm.options
elasticsearch用户没有该文件夹的权限，执行命令
chown -R es:es /usr/local/elasticsearch/
安装kibana：
1、将下载的kibana安装包解压： tar -zxvf kibana-6.6.0-linux-x86_64.tar.gz
2、更改kibana的配置
# Kibana is served by a back end server. This setting specifies the port to use. server.port: 15601 # Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values. # The default is 'localhost', which usually means remote machines will not be able to connect. # To allow connections from remote users, set this parameter to a non-loopb ack address. #server.host: "localhost" server.host: "0.0.0.0" # Enables you to specify a path to mount Kibana at if you are running behin d a proxy. # Use the `server.rewriteBasePath` setting to tell Kibana if it should remo ve the basePath # from requests it receives, and to prevent a deprecation warning at startu p. # This setting cannot end in a slash. #server.basePath: "" # Specifies whether Kibana should rewrite requests that are prefixed with # `server.basePath` or require that they are rewritten by your reverse prox y. # This setting was effectively always `false` before Kibana 6.3 and will # default to `true` starting in Kibana 7.0. #server.rewriteBasePath: false # The maximum payload size in bytes for incoming server requests. #server.maxPayloadBytes: 1048576 # The Kibana server's name. This is used for display purposes. #server.name: "your-hostname" # The URLs of the Elasticsearch instances to use for all your queries. #elasticsearch.hosts: ["http://localhost:9200"] elasticsearch.hosts: ["http://localhost:19200"] # When this setting's value is true Kibana uses the hostname specified in t he server.host # setting. When the value of this setting is false, Kibana uses the hostnam e of the host # that connects to this Kibana instance. #elasticsearch.preserveHost: true # Kibana uses an index in Elasticsearch to store saved searches, visualizat ions and # dashboards. Kibana creates a new index if the index doesn't already exist . #kibana.index: ".kibana" # The default application to load. #kibana.defaultAppId: "home" # If your Elasticsearch is protected with basic authentication, these setti ngs provide # the username and password that the Kibana server uses to perform maintena nce on the Kibana # index at startup. Your Kibana users still need to authenticate with Elast icsearch, which # is proxied through the Kibana server. #elasticsearch.username: "user" #elasticsearch.password: "pass" # Enables SSL and paths to the PEM-format SSL certificate and SSL key files , respectively. # These settings enable SSL for outgoing requests from the Kibana server to the browser. #server.ssl.enabled: false #server.ssl.certificate: /path/to/your/server.crt #server.ssl.key: /path/to/your/server.key # Optional settings that provide the paths to the PEM-format SSL certificat e and key files. # These files validate that your Elasticsearch backend uses the same key fi les. #elasticsearch.ssl.certificate: /path/to/your/client.crt #elasticsearch.ssl.key: /path/to/your/client.key # Optional setting that enables you to specify a path to the PEM file for t he certificate # authority for your Elasticsearch instance. #elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ] # To disregard the validity of SSL certificates, change this setting's valu e to 'none'. #elasticsearch.ssl.verificationMode: full # Time in milliseconds to wait for Elasticsearch to respond to pings. Defau lts to the value of # the elasticsearch.requestTimeout setting. #elasticsearch.pingTimeout: 1500 # Time in milliseconds to wait for responses from the back end or Elasticse arch. This value # must be a positive integer. #elasticsearch.requestTimeout: 30000 # List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side # headers, set this value to [] (an empty list). #elasticsearch.requestHeadersWhitelist: [ authorization ] # Header names and values that are sent to Elasticsearch. Any custom header s cannot be overwritten # by client-side headers, regardless of the elasticsearch.requestHeadersWhi telist configuration. #elasticsearch.customHeaders: {} # Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable. #elasticsearch.shardTimeout: 30000 # Time in milliseconds to wait for Elasticsearch at Kibana startup before r etrying. #elasticsearch.startupTimeout: 5000 [admin@localhost kibana-6.6.0-linux-x86_64]$ cat config/kibana.yml # Kibana is served by a back end server. This setting specifies the port to use. server.port: 15601 # Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values. # The default is 'localhost', which usually means remote machines will not be able to connect. # To allow connections from remote users, set this parameter to a non-loopb ack address. #server.host: "localhost" server.host: "0.0.0.0" # Enables you to specify a path to mount Kibana at if you are running behin d a proxy. # Use the `server.rewriteBasePath` setting to tell Kibana if it should remo ve the basePath # from requests it receives, and to prevent a deprecation warning at startu p. # This setting cannot end in a slash. #server.basePath: "" # Specifies whether Kibana should rewrite requests that are prefixed with # `server.basePath` or require that they are rewritten by your reverse prox y. # This setting was effectively always `false` before Kibana 6.3 and will # default to `true` starting in Kibana 7.0. #server.rewriteBasePath: false # The maximum payload size in bytes for incoming server requests. #server.maxPayloadBytes: 1048576 # The Kibana server's name. This is used for display purposes. #server.name: "your-hostname" # The URLs of the Elasticsearch instances to use for all your queries. #elasticsearch.hosts: ["http://localhost:9200"] elasticsearch.hosts: ["http://localhost:19200"] # When this setting's value is true Kibana uses the hostname specified in t he server.host # setting. When the value of this setting is false, Kibana uses the hostnam e of the host # that connects to this Kibana instance. #elasticsearch.preserveHost: true # Kibana uses an index in Elasticsearch to store saved searches, visualizat ions and # dashboards. Kibana creates a new index if the index doesn't already exist . #kibana.index: ".kibana" # The default application to load. #kibana.defaultAppId: "home" # If your Elasticsearch is protected with basic authentication, these setti ngs provide # the username and password that the Kibana server uses to perform maintena nce on the Kibana # index at startup. Your Kibana users still need to authenticate with Elast icsearch, which # is proxied through the Kibana server. #elasticsearch.username: "user" #elasticsearch.password: "pass" # Enables SSL and paths to the PEM-format SSL certificate and SSL key files , respectively. # These settings enable SSL for outgoing requests from the Kibana server to the browser. #server.ssl.enabled: false #server.ssl.certificate: /path/to/your/server.crt #server.ssl.key: /path/to/your/server.key # Optional settings that provide the paths to the PEM-format SSL certificat e and key files. # These files validate that your Elasticsearch backend uses the same key fi les. #elasticsearch.ssl.certificate: /path/to/your/client.crt #elasticsearch.ssl.key: /path/to/your/client.key # Optional setting that enables you to specify a path to the PEM file for t he certificate # authority for your Elasticsearch instance. #elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ] # To disregard the validity of SSL certificates, change this setting's valu e to 'none'. #elasticsearch.ssl.verificationMode: full # Time in milliseconds to wait for Elasticsearch to respond to pings. Defau lts to the value of # the elasticsearch.requestTimeout setting. #elasticsearch.pingTimeout: 1500 # Time in milliseconds to wait for responses from the back end or Elasticse arch. This value # must be a positive integer. #elasticsearch.requestTimeout: 30000 # List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side # headers, set this value to [] (an empty list). #elasticsearch.requestHeadersWhitelist: [ authorization ] # Header names and values that are sent to Elasticsearch. Any custom header s cannot be overwritten # by client-side headers, regardless of the elasticsearch.requestHeadersWhi telist configuration. #elasticsearch.customHeaders: {} # Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable. #elasticsearch.shardTimeout: 30000 # Time in milliseconds to wait for Elasticsearch at Kibana startup before r etrying. #elasticsearch.startupTimeout: 5000 # Logs queries sent to Elasticsearch. Requires logging.verbose set to true. #elasticsearch.logQueries: false # Specifies the path where Kibana creates the process ID file. #pid.file: /var/run/kibana.pid # Enables you specify a file where Kibana stores log output. #logging.dest: stdout # Set the value of this setting to true to suppress all logging output. #logging.silent: false # Set the value of this setting to true to suppress all logging output othe r than error messages. #logging.quiet: false # Set the value of this setting to true to log all events, including system usage information # and all requests. #logging.verbose: false # Set the interval in milliseconds to sample system and process performance # metrics. Minimum is 100ms. Defaults to 5000. #ops.interval: 5000 # Specifies locale to be used for all localizable strings, dates and number formats. #i18n.locale: "en"
3、启动kibana: ./bin/kibana
4、外网通过ip地址访问：
注意两个点：
（1）修改启动的es链接地址
（2）修改地址绑定不然只有本地才能访问
（3）kibana的版本和elasticsearch的版本必须一致，比如都是6.6