SpringBoot accesses a lightweight distributed log framework (GrayLog)

00. Why do you need a distributed log component?

The link to get a full set of Java learning materials: wpa.qq.com/msgrd?v=3&u…

Before the official start of the article, I will share a system I was responsible for before. Its architecture is as follows:

Every time I check a problem, I can initially locate the problem at the logic layer , but in order to explain it to the business side, I need to give evidence to the business side (log information is irrefutable evidence).

A request must be processed by one of these 8 machines, but which one is exactly, I don't know. So, I need to grep a log on each machine, and then I can find the corresponding log to prove my analysis.

Sometimes, the access layer may also need to participate together, just to troubleshoot a problem, people are stupid (it takes too long to read the log).

Later, I saw my colleague's show operation (write a script in item2: quickly log in to the bastion machine (no need to enter account and password information), cut the window according to the number of application servers and switch to the corresponding log directory). To put it bluntly, it is a one- click login to multiple application servers. Well, the speed of checking logs is much faster than before.

Later, the company's operation and maintenance side mainly pushed to log in to the application server on the Web page ( automatically log in to the bastion machine ), which can save the need to write scripts ( support batch operations ). But from the experience at the time, there was no problem with item2 accessing smoothly (it always felt stuck).

But there are still problems, because we often don't know which file info / warn / error is under. In many cases, you can only check one file at a time. Although you can directly check the wildcards , if the log is too large, it will be annoying to bring pause time.

Once the system is asked business questions, the frequency of checking logs is too high. So when I was planning a certain Q, I wanted to write the log information to the search engine myself , and learn the knowledge of the search engine by the way. Then this plan was seen by a big guy in the group, and he commented at the bottom: Why don't you try Graylog ?

It turns out that the group itself is maintaining a log framework , but I don't know... So I connected the Graylog log, and the work efficiency has been improved, and I blew a Q for this matter .

Since connecting, I haven't logged into the application server, and I almost couldn't even write grep once.

01. Lightweight ELK (Graylog)

Speaking of ELK, even if you haven't used it, you must have heard of it, and it is really popular in the back end. This time austin accesses a relatively lightweight ELK framework: Graylog

I think this framework is quite easy to use. It is very simple to connect as a user (I guess the operation and maintenance should be quite simple. Many people use Graylog to send UDP directly to the server, without having to install an agent on the machine to collect logs)

A picture is worth ten words:

Official Documentation: **
docs.graylog.org/docs **

As far as I know, quite a lot of companies use it to view logs and business monitoring alerts . In this article, I will let you experience it directly.

02、部署Graylog

老样子，直接上docker-compose，如果一直跟着我的步伐，应该对着不陌生了。 docker-compose.yml 的内容其实我也是抄官网的，这里还是贴下吧（就不用你们翻了）

version: '3'
services:
    mongo:
      image: mongo:4.2
      networks:
        - graylog
    elasticsearch:
      image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2
      environment:
        - http.host=0.0.0.0
        - transport.host=localhost
        - network.host=0.0.0.0
        - "ES_JAVA_OPTS=-Dlog4j2.formatMsgNoLookups=true -Xms512m -Xmx512m"
      ulimits:
        memlock:
          soft: -1
          hard: -1
      deploy:
        resources:
          limits:
            memory: 1g
      networks:
        - graylog
    graylog:
      image: graylog/graylog:4.2
      environment:
        - GRAYLOG_PASSWORD_SECRET=somepasswordpepper
        - GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
        - GRAYLOG_HTTP_EXTERNAL_URI=http://ip:9009/ # 这里注意要改ip
      entrypoint: /usr/bin/tini -- wait-for-it elasticsearch:9200 --  /docker-entrypoint.sh
      networks:
        - graylog
      restart: always
      depends_on:
        - mongo
        - elasticsearch
      ports:
        - 9009:9000
        - 1514:1514
        - 1514:1514/udp
        - 12201:12201
        - 12201:12201/udp
networks:
    graylog:
      driver: bridg
复制代码

这个文件里唯一需要改动的就是 ip （本来的端口是 9000 的，我由于已经占用了 9000 端口了，所以我这里把端口改成了 9009 ，你们可以随意）

嗯，写完 docker-compose.yml 文件，直接 docker-compose up -d 它就启动起来咯。

启动以后，我们就可以通过 ip:port 访问对应的Graylog后台地址了，默认的账号和密码是 admin/admin

随后，我们配置下 inputs 的配置，找到 GELF UDP ，然后点击 Launch new input ，只需要填写 Title 字段，保存就完事了（其他不用动）。

嗯，到这里，我们的GrayLog设置就完成了。

03、SpringBoot使用GrayLog

还记得我们 austin 项目使用的日志框架吗？没错，就是logback。我们要把日志数据写入Graylog很简单，只需要两步：

1、引入依赖：

<dependency>
  <groupId>de.siegmar</groupId>
  <artifactId>logback-gelf</artifactId>
  <version>3.0.0</version>
</dependency>
复制代码

2、在 logback.xml 配置graylog相关的信息：

<appender name="GELF" class="de.siegmar.logbackgelf.GelfUdpAppender">
  <!-- Graylog服务的地址 -->
  <graylogHost>ip</graylogHost>
  <!-- UDP Input端口 -->
  <graylogPort>12201</graylogPort>
  <!-- 最大GELF数据块大小（单位：字节），508为建议最小值，最大值为65467 -->
  <maxChunkSize>508</maxChunkSize>
  <!-- 是否使用压缩 -->
  <useCompression>true</useCompression>
  <encoder class="de.siegmar.logbackgelf.GelfEncoder">
    <!-- 是否发送原生的日志信息 -->
    <includeRawMessage>false</includeRawMessage>
    <includeMarker>true</includeMarker>
    <includeMdcData>true</includeMdcData>
    <includeCallerData>false</includeCallerData>
    <includeRootCauseData>false</includeRootCauseData>
    <!-- 是否发送日志级别的名称，否则默认以数字代表日志级别 -->
    <includeLevelName>true</includeLevelName>
    <shortPatternLayout class="ch.qos.logback.classic.PatternLayout">
      <pattern>%m%nopex</pattern>
    </shortPatternLayout>
    <fullPatternLayout class="ch.qos.logback.classic.PatternLayout">
      <pattern>%d - [%thread] %-5level %logger{35} - %msg%n</pattern>
    </fullPatternLayout>

    <!-- 配置应用名称（服务名称），通过staticField标签可以自定义一些固定的日志字段 -->
    <staticField>app_name:austin</staticField>
  </encoder>
</appender>
复制代码

在这个配置信息里，唯一要改的也只是 ip 的地址，到这里接入就完毕了，我们再打开控制台，就能看到日志的信息啦。

04、懂点GrayLog

懂点GrayLog查询语法：这块我日常来来去去其实就用几个，我来展示下我平时用的吧。如果觉得不够，再去官网文档捞一把就完事了：**
docs.graylog.org/docs/query-…**

1、根据字段精确查询： full_message:"13788888888"

2、查询错误日志信息： level_name:"ERROR"

3、组合多字段查询： level_name:"INFO" AND full_message:"13788888888"

在接入的时候，仔细的小伙伴可能会发现我这边在Input的时候选择的是 GELF ，然后在引入Maven依赖的时候也有 GELF 的字样。那 GELF 是啥意思呢？

这块在官网也有给出对应的解释： The Graylog Extended Log Format (GELF) is a log format that avoids the shortcomings of classic plain syslog

详细资料：**
docs.graylog.org/docs/gelf**

GELF 是一种日志格式，能避免传统意义上的 syslogs 的一些问题，而我们引入的Maven依赖则是把日志格式化成 GELF 格式然后append到GrayLog上。

05 、番外：Swagger

前几天有个老哥在GitHub给我提了个 pull request 关于 swagger 的，我昨天把他 merge 了，也升级了下 swagger 的版本。

之前我没用过 swagger 类似的文档工具，就这次 pull request 我也去体验了下 swagger 。

在初次的体验感觉是不错的：它能把项目的所有接口的 文档信息 都能在一个页面上 统一管理 ，并且就能直接通过 样例参数 直接发送请求。通过注解的方式来进行编写文档，也不用担心代码改了然后忘了更新文档这事。

但是，后来我配置好对应的参数信息文档，再在 swagger-ui 体验了下， 发现是真滴丑 ，看到这 ui 我还是阶段性放弃吧。

There are several competing products of swagger. I think the ui looks better than swagger. However, there is only one main As a proficient markdown engineer, I can easily do the document work, so I will not continue to experience other competing products.