sharding-jdbc 4.0 从开始到放弃

因为业务发展,单笔数据量到达千万级别,且每月按照100万递增,mysql单表明显无法满足现有业务发展,打算分表操作,想到了sharding-jdbc,也才开发关注时的1.0发展到现在的Apache的4.0,想想应该可以满足自己的需求了
自己项目的配置,spring boot ,mybatis,druid ,看官网介绍的也比较详细
修改下配置,加入maven依赖

<dependency>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
            <version>${sharding-jdbc.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>sharding-jdbc-core</artifactId>
            <version>${sharding-jdbc.version}</version>
            <exclusions>
                <exclusion>
                    <artifactId>sharding-core-parse-oracle</artifactId>
                    <groupId>org.apache.shardingsphere</groupId>
                </exclusion>
                <exclusion>
                    <artifactId>sharding-core-parse-postgresql</artifactId>
                    <groupId>org.apache.shardingsphere</groupId>
                </exclusion>
                <exclusion>
                    <artifactId>sharding-core-parse-sqlserver</artifactId>
                    <groupId>org.apache.shardingsphere</groupId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.antlr</groupId>
            <artifactId>antlr4-runtime</artifactId>
            <version>4.7.2</version>
        </dependency>

不知道为啥不自动依赖antlr4,明明解析的强依赖,还要报错了自己去找
因为自己只分表不分库,所以sharding的配置根据官网的说明,当然也经历了一番折腾

spring:
 profiles: dev
 shardingsphere:
   datasource:
     name: ds
     ds:
       url: jdbc:mysql://
       username: root
       password: 123456
       filters: log4j,wall,mergeStat
       type: com.alibaba.druid.pool.DruidDataSource
       driver-class-name: com.mysql.jdbc.Driver
  sharding:
     tables:
       account_flow:
         tableStrategy:
           standard:
             shardingColumn: trade_time
             preciseAlgorithmClassName: com.elm.account.config.sharding.AccountFlowTableStrategy
             rangeAlgorithmClassName: com.elm.account.config.sharding.AccountFlowTableStrategy
     bindingTables:
       - account_flow
   props:
     sql.show: true

看介绍都是inline但是支持的好少啊,改为标准策略,自己实现了=,in,and方法,如下

/**
 * 订单流水的分表规则
 * @author by jueyue on 19-6-15.
 */
public class AccountFlowTableStrategy implements PreciseShardingAlgorithm<String>, RangeShardingAlgorithm<String> {

    DateTimeFormatter formFormatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
    DateTimeFormatter toFormatter   = DateTimeFormatter.ofPattern("yyyyMM");

    @Override
    public String doSharding(Collection collection, PreciseShardingValue preciseShardingValue) {
        return preciseShardingValue.getLogicTableName() + "_" + preciseShardingValue.getValue().toString().substring(0, 7).replaceAll("-", "");
    }

    @Override
    public Collection<String> doSharding(Collection collection, RangeShardingValue rangeShardingValue) {
        List<String>  list      = new ArrayList<>();
        Range<String> ranges    = rangeShardingValue.getValueRange();
        LocalDateTime startTime = LocalDateTime.parse(ranges.lowerEndpoint(), formFormatter);
        LocalDateTime endTime   = LocalDateTime.parse(ranges.upperEndpoint(), formFormatter);
        while (startTime.isBefore(endTime)) {
            list.add(rangeShardingValue.getLogicTableName() + "_" + startTime.format(toFormatter));
            startTime = startTime.plusMonths(1);
        }
        return list;
    }
}

启动,报错,说找不到表,原表已经被我删除,改成按月的表的,顺便提一下,jdk8的时间确实好用,发现原来的数据库忘记注释了,去吧原来的数据库注释了,查询成功,
然后问题就来了,因为修改账户信息是用的mysql的几个函数,substring,md5,concat,upper结果好像是有substring是可以被识别的,其他三个都解析失败,找了各种文档也没发现如果解决,没法想到了函数,在mysql把这个sql改成了函数,还是sql 解析失败,又开始找资料,但是貌似就是不支持,没法想到了能否跳过解析这个步骤,执行原始的规则,发现了HintStrategy策略貌似感觉应该可以,但是结果依然JJ

找来找去花费了2天时间还是没有找到如何让我的sql顺利执行,提了个issue,打算放弃了,改用mysql的表分区外加历史表迁移实现把,等官网看看再实现的多点再用吧.

sharding-jdbc 4.0 从开始到放弃

猜你喜欢