logback - Custom log desensitization component

https://blog.csdn.net/qq_40885085/article/details/113385261?spm=1001.2014.3001.5501

Preface

When we write code, we will write a lot of logging code, but some sensitive data needs to be safely desensitized.

There are many ways to desensitize logs, the common ones are:
① Use the conversionRule tag and inherit MessageConverter
② Write a desensitization tool class to desensitize specific fields when printing logs.

Both methods have their own advantages and disadvantages: the first method requires modifying the code and does not comply with the opening and closing principle. The second method requires desensitization of the parameters of the log method to cause intrusion into the native log.

Custom desensitization components

A project has written a lot of code for printing logs, but later there is a need for desensitization. If we manually change the code, it will take a lot of time. If this component is introduced, desensitization can be easily completed after completing the configuration. (Easy to configure in just three steps.

1. Customized desensitization components - Desensitization effect demonstration

2. Customized desensitization components - how to use them

1. Introduce Jar package dependencies

The premise is that you put the Jar package into the local warehouse. The address of the Jar package is shown below.

<dependency>
    <groupId>pers.liuchengyin</groupId>
    <artifactId>logback-desensitization</artifactId>
    <version>1.0.0</version>
</dependency>

2. Replace the log file configuration class (logback.xml)

Log printing methods only need to be replaced with desensitized classes. If your business does not need it, there is no need to replace it.

①ConsoleAppender - console desensitization

// 原类
ch.qos.logback.core.ConsoleAppender
// 替换类
pers.liuchengyin.logbackadvice.LcyConsoleAppender

②RollingFileAppender - rolling files

// 原类
ch.qos.logback.core.rolling.RollingFileAppender
// 替换类
pers.liuchengyin.logbackadvice.LcyRollingFileAppender

③FileAppender - File

// 原类
ch.qos.logback.core.rolling.RollingFileAppender
// 替换类
pers.liuchengyin.logbackadvice.LcyRollingFileAppender

Replacement example:

<property name="CONSOLE_LOG_PATTERN"
          value="%yellow(%date{yyyy-MM-dd HH:mm:ss}) |%highlight(%-5level) |%blue(%thread) |%blue(%file:%line) |%green(%logger) |%cyan(%msg%n)"/>

<!-- ConsoleAppender 控制台输出日志 -->
<appender name="CONSOLE" class="pers.liuchengyin.logbackadvice.LcyConsoleAppender">
    <encoder>
        <pattern>
            ${CONSOLE_LOG_PATTERN}
        </pattern>
    </encoder>
</appender>

3. Add desensitization configuration file (logback-desensitize.yml)

This configuration file should be placed under the resources file

3. Customized desensitization components - desensitization specifications

1. Support data types

Eight basic types and their packaging types, Map, List, Pojo object in business, List <Pojo object in business>, JSON string.

Note: When configuring in the configuration file, you only need to configure the attribute values ​​​​in the object.

2. Unsupported data type

List<Eight basic types and packaging types>, because I don’t know which data source the desensitization is.

3. Matching rules

key + separator + value. Currently, only colon (:) and equal sign (=) are supported. Examples are as follows:

log.info("your email:{}, your phone:{}", "[email protected]","15310763497");
log.info("your email={}, your cellphone={}", "[email protected]","15310763497");

• key: defines the corresponding keywords that need to be desensitized, such as the email, phone, etc. of the appeal, as well as the fields in the business object, the Key in the Map, and the Key in the JSON
• value: The value that needs to be desensitized, such as [email protected], 15310763497 in the appeal.

4. Log specifications It is recommended that you try to be as standardized as possible when writing logs. There is no way to desensitize those with Chinese keys. The degree of specification can be seen in the code in the desensitization effect demonstration.

4. logback-desensitize.yml configuration instructions

# 日志脱敏
log-desensitize:
  # 是否忽略大小写匹配，默认为true
  ignore: true
  # 是否开启脱敏，默认为false
  open: true
  # pattern下的key/value为固定脱敏规则
  pattern:
    # 邮箱 - @前第4-7位脱敏
    email: "@>(4,7)"
    # qq邮箱 - @后1-3位脱敏
    qqemail: "@<(1,3)"
    # 姓名 - 姓脱敏，如*杰伦
    name: 1,1
    # 密码 - 所有需要完全脱敏的都可以使用内置的password
    password: password
  patterns:
    # 身份证号，key后面的字段都可以匹配以下规则(用逗号分隔)
    - key: identity,idcard
      # 定义规则的标识
      custom:
        # defaultRegex表示使用组件内置的规则：identity表示身份证号 - 内置的18/15位
        - defaultRegex: identity
          position: 9,13
        # 内置的other表示如果其他规则都无法匹配到，则按该规则处理
        - defaultRegex: other
          position: 9,10
    # 电话号码，key后面的字段都可以匹配以下规则(用逗号分隔)
    - key: phone,cellphone,mobile
      custom:
        # 手机号 - 内置的11位手机匹配规则
        - defaultRegex: phone
          position: 4,7
        # 自定义正则匹配表达式：座机号(带区号，号码七位|八位)
        - customRegex: "^0[0-9]{2,3}-[0-9]{7,8}"
        # -后面的1-4位脱敏
          position: "-<(1,4)"
        # 自定义正则匹配表达式：座机号(不带区号)
        - customRegex: "^[0-9]{7,8}"
          position: 3,5
        # 内置的other表示如果其他规则都无法匹配到，则按该规则处理
        - defaultRegex: other
          position: 1,3
    # 这种方式不太推荐 - 一旦匹配不上，就不会脱敏
    - key: localMobile
      custom:
          customRegex: "^0[0-9]{2,3}-[0-9]{7,8}"
          position: 1,3

The above configuration is relatively complete, and the hierarchical configuration format must be strictly followed.

Customized desensitization support methods

1. Key: value method
phone: 4,7, indicating that the 4-7 digits of the phone attribute are desensitized.
Original data: 13610357861
After desensitization: 136****7861

2. Use the symbol as the start node and the end node as the desensitization mark
emai: "@>(4,7)", @ is the desensitization mark, > means it is the end node, and < means it is the start node. That is, @> means to desensitize what comes before @, and @< means to desensitize what comes after @. This example is to desensitize bits 4-7 of the data before @.

Note: Double quotes and parentheses in this rule cannot be omitted. Secondly: and = cannot be used as sign symbols because they conflict with the matching rules.

• Original data: [email protected]
• "@>(4,7)"After desensitization: 123****[email protected]
• "@<(1,3)"After desensitization: 123456789@***com

3. Customized regular desensitization

patterns:
  # 手机号
  - key: phone,mobile
    custom:
      # 手机号的正则
      - customRegex: "^1[0-9]{10}"
        # 脱敏范围
        position: 4,7

customRegex: regular expression. If it matches the expression, its corresponding desensitization rule (position) will be used.

4. One field, customized desensitization based on multiple value meanings

For example, the value of the username field can be a mobile phone number or an email address. This value changes dynamically and cannot be solved by the previous methods. You can use this method.

patterns:
  - key: username
    custom:
      # 手机号 - 11位
      - defaultRegex: phone
        position : 4,7
      # 邮箱 - @
   - defaultRegex: email
     position : "@>(3,12)"
   # 身份证 - 15/18位
   - defaultRegex: identity
     position : 1,3
   # 自定义正则
   - customRegex: "^1[0-9]{10}"
     position : 1,3
   # 都匹配不到时，按照这种规则来
   - defaultRegex: other
     position : 1,3

Note: The double quotes and brackets in the matching rules in the above example cannot be omitted.

This component has four built-in matching rules: mobile phone number, ID number, email, other (used when other matches cannot be found), and a built-in desensitization method: password, which means complete desensitization and can be used under pattren.

Note: When patterns and keys under patterns are duplicated, only the method specified under patterns will be used for desensitization.

Jar package address and source code address:

https://github.com/liuchengyin01/LogbackDesensitization/tree/master/repo/pers/liuchengyin/logback-desensitization/1.0.0

Github address:

https://github.com/liuchengyin01/LogbackDesensitization

How to enter the Jar package into the Maven local warehouse

1. Download the Jar package and put it in a folder

2. Open cmd in this folder (open cmd and enter this folder)

3. Execute the command (prerequisite is to ensure that the maven configuration is normal, use the mvn -v command to check whether it is normal, if the version number is displayed, it means it is normal)

mvn install:install-file -DgroupId=pers.liuchengyin -DartifactId=logback-desensitization -Dversion=1.0.0 -Dpackaging=jar -Dfile=logback-desensitization-1.0.0.jar

Command description:

-DgroupId
 表示jar对应的groupId  
 <groupId>pers.liuchengyin</groupId>
-DartifactId:
 表示jar对应的artifactId
 <artifactId>logback-desensitization</artifactId>
-Dversion
 表示jar对应的 version
 <version>1.0.0</version>