SpringBoot integrates ELK log collection

First download ELK, official website: https://www.elastic.co/cn/
The editor downloaded version 7.16.1 here. Note that elasticsearch, logstash, and kibana are The versions must be consistent, otherwise an error may be reported.

JDK1.8 needs to be installed in advance.

The startup sequence is: Logstash >= Elasticsearch > Kibana

1. Elasticsearch

# 解压
tar -zxvf elasticsearch-7.16.1-linux-x86_64.tar.gz
# 切换目录
cd elasticsearch-7.16.1/

Modify the configuration file config/elasticsearch.yml

node.name: node-1
network.host: 192.168.12.128
http.port: 9200
cluster.initial_master_nodes: ["node-1"]

Start es:

# ./bin/elasticsearch -d 加-d后台运行
./bin/elasticsearch

Note: If there is an error when starting, the root user is not allowed to start, insufficient permissions, memory size and default configuration, etc., please check the editor's previous article:https:/ /blog.csdn.net/RookiexiaoMu_a/article/details/122023471

Started successfully, accessed 192.168.37.189:9200
Insert image description here

2. Logstash

# 解压
tar -zxvf logstash-7.16.1-linux-x86_64.tar.gz
# 切换目录
cd logstash-7.16.1/

Modify the configuration file config/logstash.yml and add at the end:

http.host: "192.168.12.128"

Edit the file in the logstash-7.16.1 directory:

vi logstash.conf

Enter the following:

input {
  tcp {
    #模式选择为server
    mode => "server"
    #ip和端口根据自己情况填写，端口默认4560,对应下文logback.xml里appender中的destination
    host => "192.168.12.128"
    port => 4560
    #格式json
    codec => json_lines
  }
}
filter {
  #过滤器，根据需要填写
}
output {
  elasticsearch {
    action => "index"
    #这里是es的地址，多个es要写成数组的形式
    hosts => "192.168.12.128:9200"
    #用于kibana过滤，可以填项目名称
    index => "api_log"
  }
}

Start using the command in the logstash-7.16.1 directory:

./bin/logstash -f logstash.conf

Access http://192.168.12.128:9600. If successful, a JSON string will be displayed.
Insert image description here

3. Kibana

# 解压
tar -zxvf kibana-7.16.1-linux-x86_64.tar.gz
# 切换目录
cd kibana-7.16.1-linux-x86_64/

Modify the configuration file config/kibana.yml

server.port: 5601
server.host: "0.0.0.0"
# 原默认是30000(30s),酌情修改,也可使用默认
elasticsearch.requestTimeout: 60000
#修改为es的地址【.url的方式会报错】
#elasticsearch.url: http://192.168.12.128:9200
elasticsearch.hosts: ["http://192.168.12.128:9200/"]

elasticsearch.url的方式会报错：
FATAL Error: [elasticsearch.url]: definition for this key is missing

Enter the kibana-7.16.1-linux-x86_64 directory to start:

# 非后台启动
./bin/kibana --allow-root
# 后台启动
nohub ./bin/kibana --allow-root &

Kibana cannot be started using the root user. You can create a new user and give permission to start it, or you can add --allow-root after the startup command.

Start the successful access page: http://192.168.12.128:5601
Insert image description here

3. SpringBoot project accesses ELK log collection

3.1 Create a new SpringBoot project, one of which contains UserController.java as follows:

package com.xiaomu.problem.controller;

import com.xiaomu.problem.entity.UserA;
import com.xiaomu.problem.entity.UserB;
import com.xiaomu.problem.service.UserService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.ArrayList;
import java.util.List;

/**
 * 用户
 *
 * @author xiaomu
 * @date 2021/12/11 14:04
 */
@Slf4j
@RestController
@RequestMapping("/user")
public class UserController {
    
    

    @Autowired
    private UserService userService;

    @GetMapping("/test1/{userId}")
    public String test1(@PathVariable Long userId) {
    
    
        UserA user = userService.getUserInfo(userId);
        log.warn("UserController.test1 userId:{}, user:{}", userId, user.toString());
        return "SUCCESS";
    }
}

Among them, User overrides the toString method.

3.2 logback-spring.xml file

<?xml version="1.0" encoding="UTF-8"?>
<!--
    scan: 当此属性设置为true时，配置文件如果发生改变，将会被重新加载，默认值为true。
    canPeriod: 设置监测配置文件是否有修改的时间间隔，如果没有给出时间单位，默认单位是毫秒。当scan为true时，此属性生效。默认的时间间隔为1分钟。
    debug: 当此属性设置为true时，将打印出logback内部日志信息，实时查看logback运行状态。默认值为false。
-->
<!--<configuration scan="true" scanPeriod="60 seconds" debug="false"></configuration>-->
<configuration debug="false">

    <!-- springProperty标签来引用yml的值 -->
    <springProperty scope="context" name="log.path" source="spring.application.name"/>

    <!--
        property标签：配置直接管理属性 我们可以直接改属性的value 格式：${name}
    -->
    <property name="FILE_LOG_PATTERN" value="logs/${log.path}-log"/>
    <!--
        日志输出格式:
        %-5level
        %d{yyyy-MM-dd HH:mm:ss.SSS}日期
        %c类的完整名称
        %M为method
        %L为行号
        %thread线程名称
        %m或者%msg为信息
        %n换行
    -->
    <!--格式化输出:%d表示日期，%thread表示线程名，%-5level∶级别从左显示5个字符宽度%msg:日志消息，%n是换行符-->
    <property name="pattern" value="%d{yyyy-MM-dd HH:mm:ss.SSS} %c [%thread] %-5level %msg%n"/>

    <appender name="stash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
        <destination>192.168.12.128:4560</destination>
        <encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder"/>
    </appender>

    <!-- root logger配置 -->
    <!-- 日志等级：OFF > FATAL > ERROR > WARN > INFO > DEBUG > ALL【OFF：关闭日志】 -->
    <root level="INFO">
        <appender-ref ref="stash"/>
    </root>

</configuration>

3.3 Start project access project

Insert image description here
Return SUCCESS, indicating success!

5. View logs in Kibana

Follow the steps:
Insert image description here
Create index:

The name editor here actually filled in api_log*
Because the name cannot be repeated, for the purpose of demonstration, the api* is filled in: api*

After the creation is completed, click Discover

Next, let’s try to search the log, enter: WARN
Insert image description here
You can see the log content, which is exactly the log printed in the program.