mybatis-plus official artifact, a dependency to resolve data permissions.

Introduction

mybatis-mate is an mp enterprise-level module, which supports sub-database and sub-table, data auditing, data sensitive word filtering (AC algorithm), field encryption, dictionary write-back (data binding), data permissions, automatic generation of SQL maintenance for table structure, etc. It is designed to process data more agilely and elegantly.

1. Main functions

• dictionary binding

• field encryption

• Data desensitization

• Dynamic maintenance of table structure

• Data audit records

• Data Scope (Data Permissions)

• Database sub-database sub-table, dynamic data source, read-write separation, automatic switching of database health check.

2. Use

2.1. Dependency import

Spring Boot introduces automatic dependency annotation packages

<dependency>
  <groupId>com.baomidou</groupId>
  <artifactId>mybatis-mate-starter</artifactId>
  <version>1.0.8</version>
</dependency>

Annotation (used by entity subcontracting)

<dependency>
  <groupId>com.baomidou</groupId>
  <artifactId>mybatis-mate-annotation</artifactId>
  <version>1.0.8</version>
</dependency>

2.2, field data binding (dictionary write-back)

For example, the user_sex type sex dictionary results are mapped to the sexText attribute

@FieldDict(type = "user_sex", target = "sexText")
private Integer sex;

private String sexText;

Implement the IDataDict interface to provide a dictionary data source and inject it into the Spring container.

@Component
public class DataDict implements IDataDict {

    /**
     * 从数据库或缓存中获取
     */
    private Map<String, String> SEX_MAP = new ConcurrentHashMap<String, String>() {
   
   {
        put("0", "女");
        put("1", "男");
    }};

    @Override
    public String getNameByCode(FieldDict fieldDict, String code) {
        System.err.println("字段类型：" + fieldDict.type() + "，编码：" + code);
        return SEX_MAP.get(code);
    }
}

2.3, field encryption

The attribute @FieldEncrypt annotation can be encrypted and stored, and the query result will be automatically decrypted. It supports the global configuration of encryption key algorithm and annotation key algorithm, and can implement IEncryptor injection of custom algorithms.

@FieldEncrypt(algorithm = Algorithm.PBEWithMD5AndDES)
private String password;

2.4. Field desensitization

The attribute @FieldSensitive annotation can automatically desensitize the source data according to the preset strategy. By default, SensitiveType has 9 built-in commonly used desensitization strategies.

For example: Chinese name, bank card account number, mobile phone number and other desensitization strategies.

You can also customize the strategy as follows:

@FieldSensitive(type = "testStrategy")
private String username;

@FieldSensitive(type = SensitiveType.mobile)
private String mobile;

The custom desensitization strategy testStrategy is added to the default strategy and injected into the Spring container.

@Configuration
public class SensitiveStrategyConfig {

    /**
     * 注入脱敏策略
     */
    @Bean
    public ISensitiveStrategy sensitiveStrategy() {
        // 自定义 testStrategy 类型脱敏处理
        return new SensitiveStrategy().addStrategy("testStrategy", t -> t + "***test***");
    }
}

For example, article sensitive word filtering

/**
 * 演示文章敏感词过滤
 */
@RestController
public class ArticleController {
    @Autowired
    private SensitiveWordsMapper sensitiveWordsMapper;

    // 测试访问下面地址观察请求地址、界面返回数据及控制台（ 普通参数 ）
    // 无敏感词 http://localhost:8080/info?content=tom&see=1&age=18
    // 英文敏感词 http://localhost:8080/info?content=my%20content%20is%20tomcat&see=1&age=18
    // 汉字敏感词 http://localhost:8080/info?content=%E7%8E%8B%E5%AE%89%E7%9F%B3%E5%94%90%E5%AE%8B%E5%85%AB%E5%A4%A7%E5%AE%B6&see=1
    // 多个敏感词 http://localhost:8080/info?content=%E7%8E%8B%E5%AE%89%E7%9F%B3%E6%9C%89%E4%B8%80%E5%8F%AA%E7%8C%ABtomcat%E6%B1%A4%E5%A7%86%E5%87%AF%E7%89%B9&see=1&size=6
    // 插入一个字变成非敏感词 http://localhost:8080/info?content=%E7%8E%8B%E7%8C%AB%E5%AE%89%E7%9F%B3%E6%9C%89%E4%B8%80%E5%8F%AA%E7%8C%ABtomcat%E6%B1%A4%E5%A7%86%E5%87%AF%E7%89%B9&see=1&size=6
    @GetMapping("/info")
    public String info(Article article) throws Exception {
        return ParamsConfig.toJson(article);
    }


    // 添加一个敏感词然后再去观察是否生效 http://localhost:8080/add
    // 观察【猫】这个词被过滤了 http://localhost:8080/info?content=%E7%8E%8B%E5%AE%89%E7%9F%B3%E6%9C%89%E4%B8%80%E5%8F%AA%E7%8C%ABtomcat%E6%B1%A4%E5%A7%86%E5%87%AF%E7%89%B9&see=1&size=6
    // 嵌套敏感词处理 http://localhost:8080/info?content=%E7%8E%8B%E7%8C%AB%E5%AE%89%E7%9F%B3%E6%9C%89%E4%B8%80%E5%8F%AA%E7%8C%ABtomcat%E6%B1%A4%E5%A7%86%E5%87%AF%E7%89%B9&see=1&size=6
    // 多层嵌套敏感词 http://localhost:8080/info?content=%E7%8E%8B%E7%8E%8B%E7%8C%AB%E5%AE%89%E7%9F%B3%E5%AE%89%E7%9F%B3%E6%9C%89%E4%B8%80%E5%8F%AA%E7%8C%ABtomcat%E6%B1%A4%E5%A7%86%E5%87%AF%E7%89%B9&see=1&size=6
    @GetMapping("/add")
    public String add() throws Exception {
        Long id = 3L;
        if (null == sensitiveWordsMapper.selectById(id)) {
            System.err.println("插入一个敏感词：" + sensitiveWordsMapper.insert(new SensitiveWords(id, "猫")));
            // 插入一个敏感词，刷新算法引擎敏感词
            SensitiveWordsProcessor.reloadSensitiveWords();
        }
        return "ok";
    }

    // 测试访问下面地址观察控制台（ 请求json参数 ）
    // idea 执行 resources 目录 TestJson.http 文件测试
    @PostMapping("/json")
    public String json(@RequestBody Article article) throws Exception {
        return ParamsConfig.toJson(article);
    }
}

2.5, DDL data structure automatic maintenance

Solve the upgrade table structure initialization, version release update SQL maintenance problems, currently support MySql, PostgreSQL.

@Component
public class PostgresDdl implements IDdl {

    /**
     * 执行 SQL 脚本方式
     */
    @Override
    public List<String> getSqlFiles() {
        return Arrays.asList(
                // 内置包方式
                "db/tag-schema.sql",
                // 文件绝对路径方式
                "D:\\db\\tag-data.sql"
        );
    }
}

Not only fixed execution, but also dynamic execution! !

ddlScript.run(new StringReader("DELETE FROM user;\n" +
                "INSERT INTO user (id, username, password, sex, email) VALUES\n" +
                "(20, 'Duo', '123456', 0, '[email protected]');"));

It also supports multi-data source execution! ! !

@Component
public class MysqlDdl implements IDdl {

    @Override
    public void sharding(Consumer<IDdl> consumer) {
        // 多数据源指定，主库初始化从库自动同步
        String group = "mysql";
        ShardingGroupProperty sgp = ShardingKey.getDbGroupProperty(group);
        if (null != sgp) {
            // 主库
            sgp.getMasterKeys().forEach(key -> {
                ShardingKey.change(group + key);
                consumer.accept(this);
            });
            // 从库
            sgp.getSlaveKeys().forEach(key -> {
                ShardingKey.change(group + key);
                consumer.accept(this);
            });
        }
    }

    /**
     * 执行 SQL 脚本方式
     */
    @Override
    public List<String> getSqlFiles() {
        return Arrays.asList("db/user-mysql.sql");
    }
}

2.6, dynamic multi-data source master-slave free switching

The @Sharding annotation enables the data source to be freely used and switched. You can add annotations to the mapper layer, and hit it according to your needs! !

@Mapper
@Sharding("mysql")
public interface UserMapper extends BaseMapper<User> {

    @Sharding("postgres")
    Long selectByUsername(String username);
}

You can also customize the strategy and dispatch troops

@Component
public class MyShardingStrategy extends RandomShardingStrategy {

    /**
     * 决定切换数据源 key {@link ShardingDatasource}
     *
     * @param group          动态数据库组
     * @param invocation     {@link Invocation}
     * @param sqlCommandType {@link SqlCommandType}
     */
    @Override
    public void determineDatasourceKey(String group, Invocation invocation, SqlCommandType sqlCommandType) {
        // 数据源组 group 自定义选择即可， keys 为数据源组内主从多节点，可随机选择或者自己控制
        this.changeDatabaseKey(group, sqlCommandType, keys -> chooseKey(keys, invocation));
    }
}

You can turn on the master-slave strategy, and of course you can turn on the health check! ! !

Specific placement:

mybatis-mate:
  sharding:
    health: true # 健康检测
    primary: mysql # 默认选择数据源
    datasource:
      mysql: # 数据库组
        - key: node1
          ...
        - key: node2
          cluster: slave # 从库读写分离时候负责 sql 查询操作，主库 master 默认可以不写
          ...
      postgres:
        - key: node1 # 数据节点
          ...

2.7, Distributed transaction log printing

Part of the configuration is as follows:

/**
 * <p>
 * 性能分析拦截器，用于输出每条 SQL 语句及其执行时间
 * </p>
 */
@Slf4j
@Component
@Intercepts({@Signature(type = StatementHandler.class, method = "query", args = {Statement.class, ResultHandler.class}),
        @Signature(type = StatementHandler.class, method = "update", args = {Statement.class}),
        @Signature(type = StatementHandler.class, method = "batch", args = {Statement.class})})
public class PerformanceInterceptor implements Interceptor {
    /**
     * SQL 执行最大时长，超过自动停止运行，有助于发现问题。
     */
    private long maxTime = 0;
    /**
     * SQL 是否格式化
     */
    private boolean format = false;
    /**
     * 是否写入日志文件<br>
     * true 写入日志文件，不阻断程序执行！<br>
     * 超过设定的最大执行时长异常提示！
     */
    private boolean writeInLog = false;

    @Override
    public Object intercept(Invocation invocation) throws Throwable {
        Statement statement;
        Object firstArg = invocation.getArgs()[0];
        if (Proxy.isProxyClass(firstArg.getClass())) {
            statement = (Statement) SystemMetaObject.forObject(firstArg).getValue("h.statement");
        } else {
            statement = (Statement) firstArg;
        }
        MetaObject stmtMetaObj = SystemMetaObject.forObject(statement);
        try {
            statement = (Statement) stmtMetaObj.getValue("stmt.statement");
        } catch (Exception e) {
            // do nothing
        }
        if (stmtMetaObj.hasGetter("delegate")) {//Hikari
            try {
                statement = (Statement) stmtMetaObj.getValue("delegate");
            } catch (Exception e) {

            }
        }

        String originalSql = null;
        if (originalSql == null) {
            originalSql = statement.toString();
        }
        originalSql = originalSql.replaceAll("[\\s]+", " ");
        int index = indexOfSqlStart(originalSql);
        if (index > 0) {
            originalSql = originalSql.substring(index);
        }

        // 计算执行 SQL 耗时
        long start = SystemClock.now();
        Object result = invocation.proceed();
        long timing = SystemClock.now() - start;

        // 格式化 SQL 打印执行结果
        Object target = PluginUtils.realTarget(invocation.getTarget());
        MetaObject metaObject = SystemMetaObject.forObject(target);
        MappedStatement ms = (MappedStatement) metaObject.getValue("delegate.mappedStatement");
        StringBuilder formatSql = new StringBuilder();
        formatSql.append(" Time：").append(timing);
        formatSql.append(" ms - ID：").append(ms.getId());
        formatSql.append("\n Execute SQL：").append(sqlFormat(originalSql, format)).append("\n");
        if (this.isWriteInLog()) {
            if (this.getMaxTime() >= 1 && timing > this.getMaxTime()) {
                log.error(formatSql.toString());
            } else {
                log.debug(formatSql.toString());
            }
        } else {
            System.err.println(formatSql);
            if (this.getMaxTime() >= 1 && timing > this.getMaxTime()) {
                throw new RuntimeException(" The SQL execution time is too large, please optimize ! ");
            }
        }
        return result;
    }

    @Override
    public Object plugin(Object target) {
        if (target instanceof StatementHandler) {
            return Plugin.wrap(target, this);
        }
        return target;
    }

    @Override
    public void setProperties(Properties prop) {
        String maxTime = prop.getProperty("maxTime");
        String format = prop.getProperty("format");
        if (StringUtils.isNotEmpty(maxTime)) {
            this.maxTime = Long.parseLong(maxTime);
        }
        if (StringUtils.isNotEmpty(format)) {
            this.format = Boolean.valueOf(format);
        }
    }

    public long getMaxTime() {
        return maxTime;
    }

    public PerformanceInterceptor setMaxTime(long maxTime) {
        this.maxTime = maxTime;
        return this;
    }

    public boolean isFormat() {
        return format;
    }

    public PerformanceInterceptor setFormat(boolean format) {
        this.format = format;
        return this;
    }

    public boolean isWriteInLog() {
        return writeInLog;
    }

    public PerformanceInterceptor setWriteInLog(boolean writeInLog) {
        this.writeInLog = writeInLog;
        return this;
    }

    public Method getMethodRegular(Class<?> clazz, String methodName) {
        if (Object.class.equals(clazz)) {
            return null;
        }
        for (Method method : clazz.getDeclaredMethods()) {
            if (method.getName().equals(methodName)) {
                return method;
            }
        }
        return getMethodRegular(clazz.getSuperclass(), methodName);
    }

    /**
     * 获取sql语句开头部分
     *
     * @param sql
     * @return
     */
    private int indexOfSqlStart(String sql) {
        String upperCaseSql = sql.toUpperCase();
        Set<Integer> set = new HashSet<>();
        set.add(upperCaseSql.indexOf("SELECT "));
        set.add(upperCaseSql.indexOf("UPDATE "));
        set.add(upperCaseSql.indexOf("INSERT "));
        set.add(upperCaseSql.indexOf("DELETE "));
        set.remove(-1);
        if (CollectionUtils.isEmpty(set)) {
            return -1;
        }
        List<Integer> list = new ArrayList<>(set);
        Collections.sort(list, Integer::compareTo);
        return list.get(0);
    }

    private final static SqlFormatter sqlFormatter = new SqlFormatter();

    /**
     * 格式sql
     *
     * @param boundSql
     * @param format
     * @return
     */
    public static String sqlFormat(String boundSql, boolean format) {
        if (format) {
            try {
                return sqlFormatter.format(boundSql);
            } catch (Exception ignored) {
            }
        }
        return boundSql;
    }
}

use:

@RestController
@AllArgsConstructor
public class TestController {
    private BuyService buyService;

    // 数据库 test 表 t_order 在事务一致情况无法插入数据，能够插入说明多数据源事务无效
    // 测试访问 http://localhost:8080/test
    // 制造事务回滚 http://localhost:8080/test?error=true 也可通过修改表结构制造错误
    // 注释 ShardingConfig 注入 dataSourceProvider 可测试事务无效情况
    @GetMapping("/test")
    public String test(Boolean error) {
        return buyService.buy(null != error && error);
    }
}

2.8. Data permissions

Add annotations to the mapper layer:

// 测试 test 类型数据权限范围，混合分页模式
@DataScope(type = "test", value = {
        // 关联表 user 别名 u 指定部门字段权限
        @DataColumn(alias = "u", name = "department_id"),
        // 关联表 user 别名 u 指定手机号字段（自己判断处理）
        @DataColumn(alias = "u", name = "mobile")
})
@Select("select u.* from user u")
List<User> selectTestList(IPage<User> page, Long id, @Param("name") String username);

Simulate business processing logic:

@Bean
public IDataScopeProvider dataScopeProvider() {
    return new AbstractDataScopeProvider() {
        @Override
        protected void setWhere(PlainSelect plainSelect, Object[] args, DataScopeProperty dataScopeProperty) {
            // args 中包含 mapper 方法的请求参数，需要使用可以自行获取
            /*
                // 测试数据权限，最终执行 SQL 语句
                SELECT u.* FROM user u WHERE (u.department_id IN ('1', '2', '3', '5'))
                AND u.mobile LIKE '%1533%'
             */
            if ("test".equals(dataScopeProperty.getType())) {
                // 业务 test 类型
                List<DataColumnProperty> dataColumns = dataScopeProperty.getColumns();
                for (DataColumnProperty dataColumn : dataColumns) {
                    if ("department_id".equals(dataColumn.getName())) {
                        // 追加部门字段 IN 条件，也可以是 SQL 语句
                        Set<String> deptIds = new HashSet<>();
                        deptIds.add("1");
                        deptIds.add("2");
                        deptIds.add("3");
                        deptIds.add("5");
                        ItemsList itemsList = new ExpressionList(deptIds.stream().map(StringValue::new).collect(Collectors.toList()));
                        InExpression inExpression = new InExpression(new Column(dataColumn.getAliasDotName()), itemsList);
                        if (null == plainSelect.getWhere()) {
                            // 不存在 where 条件
                            plainSelect.setWhere(new Parenthesis(inExpression));
                        } else {
                            // 存在 where 条件 and 处理
                            plainSelect.setWhere(new AndExpression(plainSelect.getWhere(), inExpression));
                        }
                    } else if ("mobile".equals(dataColumn.getName())) {
                        // 支持一个自定义条件
                        LikeExpression likeExpression = new LikeExpression();
                        likeExpression.setLeftExpression(new Column(dataColumn.getAliasDotName()));
                        likeExpression.setRightExpression(new StringValue("%1533%"));
                        plainSelect.setWhere(new AndExpression(plainSelect.getWhere(), likeExpression));
                    }
                }
            }
        }
    };
}

Final execution SQL output:

SELECT u.* FROM user u 
  WHERE (u.department_id IN ('1', '2', '3', '5')) 
  AND u.mobile LIKE '%1533%' LIMIT 1, 10

Currently only the paid version is available. For more examples of using mybatis-mate, see:

mybatis-mate-examples: mybatis-mate is an mp enterprise-level module that supports sub-database and sub-table, data auditing, data-sensitive word filtering (AC algorithm), field encryption, dictionary write-back (data binding), data permissions, table structure Automatically generate SQL maintenance, etc.