ELKログストレージクラスター
ELKには、次の3つのコンポーネントが含ま
れ
ます。Elasticsearch:Logstashによって収集されたログの受信Logstash:収集されてElasticsearchに送信されるログの指定を担当します。Kibana:
Webインターフェイスの提供を担当します。
この記事は、ELKロギングツール(yumと共にインストール)の簡単なインストールです。特別な要件がない場合は、yumを使用してインストールすることをお勧めします。この記事は単なる初心者向けガイドであり、logstashについては今後詳しく説明します。ご
不明な点がございましたら、WeChatに連絡して(記事の最後に)jdk1.8を修正し
、jdkパッケージをオンラインでダウンロードして解凍してください(最初にjdkディレクトリを作成してjdkファイルを解凍します)
[root@localhost jdk1.8.0_131] mkdir /usr/java
[root@localhost jdk1.8.0_131] tar -zxf jdk1.8.0_131.tar.gz
[root@localhost jdk1.8.0_131] mv jdk1.8.0_131 /usr/java
Linux環境変数を作成します(jdkの環境変数を作成します)
[root@localhost jdk1.8.0_131] tail -n 5 /etc/profile\
export JAVA_HOME=/usr/java/jdk1.8.0_131
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:$CLASSPATH
export JAVA_PATH=${JAVA_HOME}/bin:${JRE_HOME}/bin
export PATH=$PATH:${JAVA_PATH}
##注意要使用以下命令让环境变量生效
[root@localhost jdk1.8.0_131] source /etc/profile
インストールが成功したかどうかを確認します
[root@localhost jdk1.8.0_131]# java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
清華大学のyumソースを紹介する
[root@localhost jdk1.8.0_131]# cat /etc/yum.repos.d/elk.repo
[elasticsearch]
name=Elasticsearch7.0
baseurl=https://mirror.tuna.tsinghua.edu.cn/elasticstack/7.x/yum/
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
Elasticsearchをインストールします
[root@localhost jdk1.8.0_131]# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch&&yum install -y elasticsearch
Elasticsearchのパラメーター、特にノードの数を変更します。変更しないと、サービスの開始に失敗します。
変更は次のとおり
です。23行目:node.name:node-1#
55ノード名の行:network.host:0.0.0.0
#ネットワークホスト行59:http.port:9200#リスニングポート
行72:cluster.initial_master_nodes: ["Node-1"]#このクラスターにはどのノードがありますか
[root@localhost jdk1.8.0_131]# cat /etc/elasticsearch/elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-1"]
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
サービス
メモリの変更-Xms512m ##最小メモリ量
-Xmx512m ##最大メモリ量
[root@localhost jdk1.8.0_131]# cat /etc/elasticsearch/jvm.options
## JVM configuration
################################################################
## IMPORTANT: JVM heap size
################################################################
##
## You should always set the min and max JVM heap
## size to the same value. For example, to set
## the heap to 4 GB, set:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms512m
-Xmx512m
Elasticsearchを開始します
[root@localhost jdk1.8.0_131]# systemctl start elasticsearch
[root@localhost jdk1.8.0_131]# netstat -tnlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 6666/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 6826/master
tcp6 0 0 :::9200 :::* LISTEN 51275/java
tcp6 0 0 :::9300 :::* LISTEN 51275/java
tcp6 0 0 :::22 :::* LISTEN 6666/sshd
tcp6 0 0 ::1:25 :::* LISTEN 6826/master
安装 kibana
[root@localhost jdk1.8.0_131]# yum install -y kibana
kibanaパラメーターの構成
2行server.port:5601
#リスニングポート7の変更line server.host: "0.0.0.0"#リスニングホストの変更
28行elasticsearch.hosts :["http:// localhost:9200"] #elasticsearchアドレスの変更
##デフォルトのフォントを変更する場合は、構成ファイルの最後の行で変更します
##実稼働環境にいる場合は、elasticsearch.hosts:["http:// localhost:9200"]のlocalhostを特定のelasticsearchサーバーのIPアドレスに変更することをお勧めします。
[root@localhost jdk1.8.0_131]# cat /etc/kibana/kibana.yml
# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601
# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "0.0.0.0"
# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""
# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# This setting was effectively always `false` before Kibana 6.3 and will
# default to `true` starting in Kibana 7.0.
#server.rewriteBasePath: false
# The maximum payload size in bytes for incoming server requests.
#server.maxPayloadBytes: 1048576
# The Kibana server's name. This is used for display purposes.
#server.name: "your-hostname"
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["http://localhost:9200"]
キバナを開始します(このサービスの開始には多少の遅延があります。しばらく待って、netstatでポートを確認してください)
[root@localhost jdk1.8.0_131]# systemctl start kibana
[root@localhost jdk1.8.0_131]# netstat -tnlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 6666/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 6826/master
tcp 0 0 0.0.0.0:5601 0.0.0.0:* LISTEN 51548/node
tcp6 0 0 :::9200 :::* LISTEN 51275/java
tcp6 0 0 :::9300 :::* LISTEN 51275/java
tcp6 0 0 :::22 :::* LISTEN 6666/sshd
tcp6 0 0 ::1:25 :::* LISTEN 6826/master
logstashをインストールします(logstashはログ電話システムです)
[root@localhost jdk1.8.0_131]# yum install -y logstash
セットメモリ
-Xms512m最小512m
-Xmx512m最大512m
[root@localhost logstash]# cat /etc/logstash/jvm.options
## JVM configuration
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms512m
-Xmx512m
logstashを開始します(logstashには、将来の記事に反映される多くのプラグインがあります)
[root@localhost ~]# /usr/share/logstash/bin/logstash -e 'input{stdin{}}output{stdout{codec=>rubydebug}}'
[INFO ] 2020-12-20 11:39:26.277 [Api Webserver] agent - Successfully started Logstash API endpoint {
:port=>9600}
hello(这是你输入的日志就会在下面显示出来,也可以自定义要收集的某个文件比如NGINX的日志文件)
{
"@version" => "1",
"host" => "localhost.localdomain",
"message" => "hello",
"@timestamp" => 2020-12-20T03:39:59.089Z
}
サーバーのIPアドレスとポート番号を開きます。例:http://192.168.182.150:5601 /
WeChat:a1362623821