ELK日志存储集群
ELK分别包含以下三个组件
Elasticsearch ：接收Logstash收集的日志
Logstash：负责指定收集哪些日志并发送给Elasticsearch
Kibana ：负责提供web界面
本文只是简单安装ELK日志工具（使用yum安装），如果没有特殊要求，推荐使用yum来安装，此文章只是初级入门，以后会针对logstash出一个详解。如有问题请联系微信指正（文章末尾）
安装jdk1.8
在网上下载jdk包并解压(首先创建jdk目录，并且解压jdk文件)
[root@localhost jdk1.8.0_131] mkdir /usr/java
[root@localhost jdk1.8.0_131] tar -zxf jdk1.8.0_131.tar.gz
[root@localhost jdk1.8.0_131] mv jdk1.8.0_131 /usr/java
写入linux环境变量（为jdk创建环境变量）
[root@localhost jdk1.8.0_131] tail -n 5 /etc/profile\
export JAVA_HOME=/usr/java/jdk1.8.0_131
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:$CLASSPATH
export JAVA_PATH=${JAVA_HOME}/bin:${JRE_HOME}/bin
export PATH=$PATH:${JAVA_PATH}
##注意要使用以下命令让环境变量生效
[root@localhost jdk1.8.0_131] source /etc/profile
检查是否安装成功
[root@localhost jdk1.8.0_131]# java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
引入清华大学yum源
[root@localhost jdk1.8.0_131]# cat /etc/yum.repos.d/elk.repo
[elasticsearch]
name=Elasticsearch7.0
baseurl=https://mirror.tuna.tsinghua.edu.cn/elasticstack/7.x/yum/
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
安装Elasticsearch
[root@localhost jdk1.8.0_131]# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch&&yum install -y elasticsearch
修改Elasticsearch参数，尤其是节点数量问题，如不修改会导致服务启动失败。
修改的地方有：
23行： node.name: node-1 #此节点名称
55行： network.host: 0.0.0.0 #网络主机
59行： http.port: 9200 #监听端口
72行： cluster.initial_master_nodes: [“node-1”] #此集群有哪些节点
[root@localhost jdk1.8.0_131]# cat /etc/elasticsearch/elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-1"]
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
修改服务内存
-Xms512m ##最小内存数量
-Xmx512m ##最大内存数量
[root@localhost jdk1.8.0_131]# cat /etc/elasticsearch/jvm.options
## JVM configuration
################################################################
## IMPORTANT: JVM heap size
################################################################
##
## You should always set the min and max JVM heap
## size to the same value. For example, to set
## the heap to 4 GB, set:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms512m
-Xmx512m
启动elasticsearch
[root@localhost jdk1.8.0_131]# systemctl start elasticsearch
[root@localhost jdk1.8.0_131]# netstat -tnlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 6666/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 6826/master
tcp6 0 0 :::9200 :::* LISTEN 51275/java
tcp6 0 0 :::9300 :::* LISTEN 51275/java
tcp6 0 0 :::22 :::* LISTEN 6666/sshd
tcp6 0 0 ::1:25 :::* LISTEN 6826/master
安装 kibana
[root@localhost jdk1.8.0_131]# yum install -y kibana
配置kibana参数
2行 server.port: 5601 #修改监听端口
7行 server.host: “0.0.0.0” #修改监听主机
28行 elasticsearch.hosts: [“http://localhost:9200”] #修改 elasticsearch地址
##如果想修改默认字体的话在配置文件的最后一行修改即可
##如果在生产环境建议elasticsearch.hosts: [“http://localhost:9200”]里面的localhost改成具体的elasticsearch服务器ip地址
[root@localhost jdk1.8.0_131]# cat /etc/kibana/kibana.yml
# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601
# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "0.0.0.0"
# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""
# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# This setting was effectively always `false` before Kibana 6.3 and will
# default to `true` starting in Kibana 7.0.
#server.rewriteBasePath: false
# The maximum payload size in bytes for incoming server requests.
#server.maxPayloadBytes: 1048576
# The Kibana server's name. This is used for display purposes.
#server.name: "your-hostname"
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["http://localhost:9200"]
启动kibana（此服务启动会有一些延迟，等一段时间netstat查看端口即可）
[root@localhost jdk1.8.0_131]# systemctl start kibana
[root@localhost jdk1.8.0_131]# netstat -tnlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 6666/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 6826/master
tcp 0 0 0.0.0.0:5601 0.0.0.0:* LISTEN 51548/node
tcp6 0 0 :::9200 :::* LISTEN 51275/java
tcp6 0 0 :::9300 :::* LISTEN 51275/java
tcp6 0 0 :::22 :::* LISTEN 6666/sshd
tcp6 0 0 ::1:25 :::* LISTEN 6826/master
安装logstash(logstash 是一个日志手机系统)
[root@localhost jdk1.8.0_131]# yum install -y logstash
设置内存
-Xms512m 最小512m
-Xmx512m 最大512m
[root@localhost logstash]# cat /etc/logstash/jvm.options
## JVM configuration
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms512m
-Xmx512m
启动logstash（logstash 有很多插件会在以后的文章里面体现）
[root@localhost ~]# /usr/share/logstash/bin/logstash -e 'input{stdin{}}output{stdout{codec=>rubydebug}}'
[INFO ] 2020-12-20 11:39:26.277 [Api Webserver] agent - Successfully started Logstash API endpoint {
:port=>9600}
hello（这是你输入的日志就会在下面显示出来，也可以自定义要收集的某个文件比如NGINX的日志文件）
{
"@version" => "1",
"host" => "localhost.localdomain",
"message" => "hello",
"@timestamp" => 2020-12-20T03:39:59.089Z
}
打开服务器的IP地址加上端口号例：http://192.168.182.150:5601/
微信:a1362623821