过程就是：logstash agent采集文件到redis队列，logstash index从redis队列将数据传输到elasticsearch全文搜索引擎，kibana通过浏览器显示数据
本文安装工具如下:
logstash-5.6.3,elasticsearch-5.6.3,kibana-5.6.3,redis-3.2.1，均是写本文时最新版本。
linux下安装教程如下：
备注：记得先修改配置，再启动
一.安装redis
下载地址:
wget http://download.redis.io/releases/redis-3.2.1.tar.gz
tar xzf redis-3.2.1.tar.gz -C /usr/local/
cd /usr/local/redis-3.2.1
make
修改配置
vi /usr/local/redis-3.2.1/redis.conf
daemonize yes
bind 127.0.0.1(改成ip)
启动:
/usr/local/redis-3.2.1/src/redis-server /usr/local/redis-3.2.1/redis.conf
停止:
ps -ef|grep redis
监控:
/usr/local/redis-3.2.1/src/redis-cli monitor
二.安装logstash
下载地址:
wget https://artifacts.elastic.co/downloads/logstash/logstash-5.6.3.tar.gz
tar zxf logstash-5.6.3.tar.gz -C /usr/local/
配置logstash的环境变量
# echo "export PATH=\$PATH:/usr/local/logstash-5.6.3/bin" > /etc/profile.d/logstash.sh
# . /etc/profile
添加logstash-file-to-redis.conf配置文件
input {
file {
path => [
"/home/rc/out"
]
}
}
output {
stdout { codec => rubydebug }
redis {
host => '172.24.132.108'
port => '6379'
data_type => 'list'
key => 'logstash:redis'
}
}
启动agent:
/usr/local/logstash-5.6.3/bin/logstash -f logstash-input-to-redis.conf //测试手动输入
nohup /usr/local/logstash-5.6.3/bin/logstash -f logstash-file-to-redis.conf &
添加logstash-redis-to-elasticsearch-5.6.3.conf配置文件
input {
redis {
host => '172.24.132.108'
data_type => 'list'
port => "6379"
key => 'logstash:redis'
type => 'redis-input'
}
}
output {
elasticsearch {
hosts => ["172.24.132.108"]
}
}
启动index:(要重新指定path.data)
nohup /usr/local/logstash-5.6.3/bin/logstash --path.data /usr/local/logstash-5.6.3/dataindex -f logstash-redis-to-elasticsearch-5.6.3.conf &
停止:
ps -ef|grep logstash
三.安装elasticsearch
下载地址
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.3.tar.gz
tar zxf elasticsearch-5.6.3.tar.gz -C /usr/local
启动:
/usr/local/elasticsearch-5.6.3/bin/elasticsearch -d
报错则加上下边这句 -Des.insecure.allow.root=true
配置地址:
vim /usr/local/elasticsearch-5.6.3/config/elasticsearch.yml
修改配置
network.host: 172.24.132.108 //指定ip
停止:
ps -ef | grep elastic
非root用户后台运行
groupadd es #增加es组
useradd es -g es -p pwd #增加es用户并附加到es组
chown -R es:es elasticsearch-5.6.3 #给目录权限
su es #使用es用户
./bin/elasticsearch -d #后台运行es
设置文件最大打开数量,root
vim /etc/security/limits.conf 修改最大值
ulimit -n 65536
查看记录
curl http://172.24.132.108:9200/_search?pretty
四.安装kibana
下载地址:
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.6.3-linux-x86_64.tar.gz
tar zxf kibana-5.6.3-linux-x86_64.tar.gz -C /usr/local
启动:
nohup /usr/local/kibana-5.6.3-linux-x86_64/bin/kibana &
配置地址:
vim /usr/local/kibana-5.6.3-linux-x86_64/config/kibana.yml
修改配置:
elasticsearch.url: "http://172.24.132.108:9200"
server.host: "0.0.0.0"
停止:
ps -ef|grep kibana
