Springboot ELK (logback-> redis-> logstash-> ES) complete record

Disclaimer: This article is a blogger original article, follow the CC 4.0 BY-SA copyright agreement, reproduced, please attach the original source link and this statement.

This link: https://blog.csdn.net/h_sn9999/article/details/102757994

Project installation: Redis, logstash, ES, kibana installation, please refer to the official documentation

Project Springboot version:

<parent>
       <groupId>org.springframework.boot</groupId>
       <artifactId>spring-boot-starter-parent</artifactId>
       <version>2.1.5.RELEASE</version>
</parent>

<spring-cloud.version>Greenwich.SR1</spring-cloud.version>

pom.xml introduced

   <dependency>
           <groupId>com.cwbase</groupId>
           <artifactId>logback-redis-appender</artifactId>
           <version>1.1.5</version>
   </dependency>

logback.xml, this file in the following sources

<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="false">

   <appender name="STDOUT"
       class="ch.qos.logback.core.ConsoleAppender">
       <encoder
           class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
           <pattern>%date [%10thread] %-5level %logger{36} - %msg%n</pattern>
       </encoder>
   </appender>

   <appender name="FILE"
       class="ch.qos.logback.core.rolling.RollingFileAppender">
       <file>logs/backend.log</file>
       <rollingPolicy
           class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
           <fileNamePattern>logs/backend.%d{yyyy-MM-dd}.log.zip
           </fileNamePattern>
           <maxHistory>1</maxHistory>
       </rollingPolicy>
       <encoder
           class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
           <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %C.%M:%L -
               %m%n</pattern>
       </encoder>
   </appender>

   <appender name="LOGSTASH"
       class="com.cwbase.logback.RedisAppender">
       <source>user_common_service</source>
       <sourcePath>node1</sourcePath>
       <type>Service</type>
       <tags>production</tags>
       <host>127.0.0.1</host>
       <port>6379</port>
       <key>logstash</key>
       <additionalField>
            <key>teamName</key>
            <value>mobile_app</value>
        </additionalField>
   </appender>

</configuration>

Attention to the following key points:

<Source> user_common_service </ Source> #### can configure the application name, followed by the ES to create an index name, note the lowercase
       <sourcePath> node1 </ sourcePath>
       <of the type> Service </ of the type>
       <Tags> Production's < / Tags>
       <Host> 127.0.0.1 </ Host> ### Redis address
       <port> 6379 </ port> ### Redis port
       <Key> logstash </ Key>
       <additionalField> ### customize Attributes! !
            <Key> teamname </ Key>
            <value> mobile_app </ value>
        </ additionalField>

logstah configuration:

input {
redis {
codec => json
host => "192.168.1.98"
port => 6379
key => "logstash"
data_type => "list"
}
}

filter {
}
output {
elasticsearch {
index => "log-%{[source]}-%{+YYYY.MM.dd}"
hosts => ["192.168.1.98:9200"]
}
stdout {codec => rubydebug}
}

create index mode, "log -% {[Source]} - YYYY.MM.DD% {+}" Here create an index according to sources and date of application, according to their need to define

kibana中 Create index pattern

Modify the shards elasticsearch

curl -XPUT 'http://172.28.161.90:19200/_template/logstash-*' -H 'Content-Type: application/json' -d'{
"index_patterns" : ["*"],
"order" : 0,
"settings" : {
"number_of_shards" : 5
}
}'

代码请参考：https://github.com/hsn999/SpringCloud_ELK

Springboot ELK (logback-> redis-> logstash-> ES) complete record

Guess you like