3 steps to complete Spring Boot's log desensitization

在我们写代码的时候，会书写许多日志代码，但是有些敏感数据是需要进行安全脱敏处理的。

copy

There are many ways to desensitize logs. The common ones are ① using conversionRulelabels, inheriting MessageConverter② writing a desensitization tool class, and desensitizing specific fields when printing logs.

Both ways have advantages and disadvantages:

• The first method needs to modify the code, which does not conform to the principle of opening and closing.
• The second method requires desensitization of the parameters of the log method, which has intrusive behavior on the original log.

Custom desensitization component (slf4j+logback)

A project has written a lot of codes for printing logs, but there is a need for desensitization later. If we manually change the codes, it will take a lot of time. If this component is introduced, desensitization can be easily completed after configuration. (Easy to configure in just three steps)

1. Custom desensitization component - desensitization effect demonstration

;

;

2. Custom desensitization components - usage method

1. Introduce Jar package dependencies

The premise is that you put the Jar package into the local warehouse. See the Jar package address below.

<dependency>
    <groupId>pers.liuchengyin</groupId>
    <artifactId>logback-desensitization</artifactId>
    <version>1.0.0</version>
</dependency>

copy

2. Replace the log file configuration class (logback.xml)

The log printing methods only need to be replaced with desensitized classes. If your business does not need it, there is no need to replace it.

①ConsoleAppender - console desensitization

// 原类
ch.qos.logback.core.ConsoleAppender
// 替换类
pers.liuchengyin.logbackadvice.LcyConsoleAppender

copy

②RollingFileAppender - rolling file

// 原类
ch.qos.logback.core.rolling.RollingFileAppender
// 替换类
pers.liuchengyin.logbackadvice.LcyRollingFileAppender

copy

③FileAppender - file

// 原类
ch.qos.logback.core.FileAppender
// 替换类
pers.liuchengyin.logbackadvice.LcyFileAppender

copy

Replacement example:

<property name="CONSOLE_LOG_PATTERN"
          value="%yellow(%date{yyyy-MM-dd HH:mm:ss}) |%highlight(%-5level) |%blue(%thread) |%blue(%file:%line) |%green(%logger) |%cyan(%msg%n)"/>

<!-- ConsoleAppender 控制台输出日志 -->
<appender name="CONSOLE" class="pers.liuchengyin.logbackadvice.LcyConsoleAppender">
    <encoder>
        <pattern>
            ${CONSOLE_LOG_PATTERN}
        </pattern>
    </encoder>
</appender>

copy

3. Add a desensitization configuration file (logback-desensitize.yml)

The configuration file should be placed under the resources file

;

3. Custom desensitization components - desensitization specification

1. Supported data types

Eight basic types and their packaging types, Map, List, Pojo object in business, List<Pojo object in business>, JSON string.

Note: When configuring in the configuration file, you only need to configure the attribute values ​​​​in the object.

2. Unsupported data types

List<eight basic types and packaging types>, because I don't know which data source is desensitized.

3. Matching rules

key + separator + value, currently only supports colon (:) and equal sign (=), examples are as follows:

log.info("your email:{}, your phone:{}", "[email protected]","15310763497");
log.info("your email={}, your cellphone={}", "[email protected]","15310763497");

copy

• key: Defines the corresponding keywords that need to be desensitized, such as email, phone, etc. of the appeal, as well as the fields in the business object, the Key in the Map, and the Key in the JSON
• value: Values ​​that need to be desensitized, such as Appeal's [email protected], 15310763497.

4. Log specification

It is recommended to be as standard as possible when writing the log. There is no way to desensitize the key in Chinese. The degree of standardization can be seen in the code in the demo of the desensitization effect.

Four, logback-desensitize.yml configuration instructions

# 日志脱敏
log-desensitize:
  # 是否忽略大小写匹配，默认为true
  ignore: true
  # 是否开启脱敏，默认为false
  open: true
  # pattern下的key/value为固定脱敏规则
  pattern:
    # 邮箱 - @前第4-7位脱敏
    email: "@>(4,7)"
    # qq邮箱 - @后1-3位脱敏
    qqemail: "@<(1,3)"
    # 姓名 - 姓脱敏，如*杰伦
    name: 1,1
    # 密码 - 所有需要完全脱敏的都可以使用内置的password
    password: password
  patterns:
    # 身份证号，key后面的字段都可以匹配以下规则(用逗号分隔)
    - key: identity,idcard
      # 定义规则的标识
      custom:
        # defaultRegex表示使用组件内置的规则：identity表示身份证号 - 内置的18/15位
        - defaultRegex: identity
          position: 9,13
        # 内置的other表示如果其他规则都无法匹配到，则按该规则处理
        - defaultRegex: other
          position: 9,10
    # 电话号码，key后面的字段都可以匹配以下规则(用逗号分隔)
    - key: phone,cellphone,mobile
      custom:
        # 手机号 - 内置的11位手机匹配规则
        - defaultRegex: phone
          position: 4,7
        # 自定义正则匹配表达式：座机号(带区号，号码七位|八位)
        - customRegex: "^0[0-9]{2,3}-[0-9]{7,8}"
        # -后面的1-4位脱敏
          position: "-<(1,4)"
        # 自定义正则匹配表达式：座机号(不带区号)
        - customRegex: "^[0-9]{7,8}"
          position: 3,5
        # 内置的other表示如果其他规则都无法匹配到，则按该规则处理
        - defaultRegex: other
          position: 1,3
    # 这种方式不太推荐 - 一旦匹配不上，就不会脱敏
    - key: localMobile
      custom:
          customRegex: "^0[0-9]{2,3}-[0-9]{7,8}"
          position: 1,3

copy

The above configuration is relatively complete, and the hierarchical configuration format must be strictly followed.

How to customize desensitization support

1. The way of key:value

• phone: 4,7, indicating that the 4-7 digits of the phone attribute are desensitized
• Raw data:13610357861
• After desensitization:136****7861

2. Use the symbol as the starting point and the ending node as the desensitization mark

emai:"@>(4,7)", @is a desensitization flag, >indicating that it is an end node, and <indicating that it is a starting node. It @>means @desensitizing the previous one, @<and desensitizing @the later one. This example is @to desensitize the 4-7 bits of the previous data.

Note: The double quotes and parentheses in this rule cannot be omitted, and the second :and =cannot be used as glyphs, because they conflict with the matching rules

• Raw data:[email protected]
• "@>(4,7)"After desensitization:123****[email protected]
• "@<(1,3)"After desensitization:123456789@***com

3. Custom regular desensitization

patterns:
  # 手机号
  - key: phone,mobile
    custom:
      # 手机号的正则
      - customRegex: "^1[0-9]{10}"
        # 脱敏范围
        position: 4,7

copy

customRegex: regular expression, if it matches the expression, use its corresponding desensitization rule (position)

4. A field, which can be customized and desensitized according to the meaning of multiple values

For example, the value of the username field can be a mobile phone number or an email address. This value changes dynamically. The previous methods cannot solve the problem, so this method can be used.

patterns:
  - key: username
    custom:
      # 手机号 - 11位
      - defaultRegex: phone
        position : 4,7
      # 邮箱 - @
   - defaultRegex: email
     position : "@>(3,12)"
   # 身份证 - 15/18位
   - defaultRegex: identity
     position : 1,3
   # 自定义正则
   - customRegex: "^1[0-9]{10}"
     position : 1,3
   # 都匹配不到时，按照这种规则来
   - defaultRegex: other
     position : 1,3

copy

Note: The double quotes and parentheses in the matching rules in the above example cannot be omitted

This component has four built-in matching rules: mobile phone number, ID number, email, other (used when other matches are not available), and a built-in desensitization method: password, which means complete desensitization, and can be used under pattren.

Note: When the keys under pattern and patterns are duplicated, only the method specified under pattern will be used for desensitization.

Jar package address and source code address

https://github.com/liuchengyin01/LogbackDesensitization/tree/master/repo/pers/liuchengyin/logback-desensitization/1.0.0

;

Github address:

https://github.com/liuchengyin01/LogbackDesensitization

How to enter the Jar package into the Maven local warehouse

1. Download the Jar package and put it in a folder

2. Open cmd in this folder (open cmd and enter this folder)

3. Execute the command (the prerequisite is to ensure that the maven configuration is normal, use mvn -vthe command to check whether it is normal, if the version number is displayed, it means normal)

mvn install:install-file -DgroupId=pers.liuchengyin -DartifactId=logback-desensitization -Dversion=1.0.0 -Dpackaging=jar -Dfile=logback-desensitization-1.0.0.jar

copy

Command description:

-DgroupId
 表示jar对应的groupId  
 <groupId>pers.liuchengyin</groupId>
-DartifactId:
 表示jar对应的artifactId
 <artifactId>logback-desensitization</artifactId>
-Dversion
 表示jar对应的 version
 <version>1.0.0</version>

copy

copy