Springcloud practical self-developed distributed id generator

1. Background

In daily development, we need to use ID to uniquely represent         various data in the system . For example, the user ID corresponds to and only corresponds to one person, the product ID corresponds to and only corresponds to one product, and the order ID corresponds to and only corresponds to one order. We also have various IDs in real life . For example, an ID card ID corresponds to and only corresponds to one person. Simply put, ID is the unique identification of data. Under normal circumstances, the auto-incrementing primary key of the database will be used as the data ID . However, in the case of large quantities, we often introduce methods such as distribution and sub-database and table sub-database to deal with it. Obviously, after the data is sub-database and sub-table, we still A unique ID is needed to identify a piece of data or message, and the database's self-increasing ID can no longer meet the demand. At this time, a system that can generate a globally unique ID is very necessary. To summarize, what are the requirements for ID numbers in business systems?
        Global uniqueness : Duplicate ID numbers cannot appear. Since it is a unique identifier, this is the most basic requirement.
        Trend increasing , monotonous increasing : ensure that the next ID must be greater than the previous ID .
        Information security : If the ID is continuous, it will be very easy for malicious users to steal the information. Just download the specified URL in order ; if it is an order number, it will be even more dangerous. Competitors can directly know our daily order volume. Therefore, in some application scenarios, IDs may need to be irregular or irregular.
        At the same time, in addition to the requirements for the ID number itself, the business also has extremely high requirements for the availability of the ID number generation system. Imagine that if the ID generation system is unstable and relies heavily on the ID generation system, key actions such as order generation cannot be executed. Therefore, an ID generation system also needs to have the average delay and TP999 delay as low as possible; the availability is 5 9s ; and the QPS is high .

2: Introduction to common methods

        2.1 UUID

        The standard form of UUID (Universally Unique Identifier) ​​contains 32 hexadecimal digits, divided into five segments with hyphens, 36 characters in the form of 8-4-4-4-12, example: 550e8400-e29b-41d4-a716 -446655440000.

        2.1.1 Advantages

                Very high performance: locally generated, no network consumption.

        2.1.2 Disadvantages

                Not easy to store: UUID is too long, 16 bytes and 128 bits, and is usually represented by a 36-bit string, which is not applicable to many scenarios.

                Information insecurity: The algorithm for generating UUID based on MAC address may cause the MAC address to be leaked. This vulnerability was once used to find the location of the creator of the Melissa virus.

                When ID is used as the primary key, there will be some problems in specific environments. For example, in the scenario of DB primary key, UUID is very unsuitable:

                ① MySQL officially recommends that the primary key should be as short as possible [4]. The 36-character UUID does not meet the requirements.

                ② Unfavorable to MySQL index: If used as the primary key of the database, under the InnoDB engine, the disorder of UUID may cause frequent changes in data location, seriously affecting performance. Clustered indexes are used in the MySQL InnoDB engine. Since most RDBMS use B-tree data structures to store index data, we should try to use ordered primary keys to ensure writing performance when selecting primary keys.

                You can directly use the UUID that comes with jdk. The original generated one is with an underscore. If you don't need it, you can remove it yourself. For example, the following code:

public static void main(String[] args) {
        for (int i = 0; i < 5; i++) {
            String rawUUID = UUID.randomUUID().toString();
            //去除“-”
            String uuid = rawUUID.replaceAll("-", "");
            System.out.println(uuid);
        }
    }
        2.2 Snowflake algorithm and its derivatives

                This solution is roughly an algorithm that generates IDs by dividing the namespace (UUIDs also count, because they are relatively common, so they are analyzed separately). Snowflake is Twitter's open source distributed ID generation algorithm. Snowflake divides 64-bit into multiple segments to mark the machine, time, etc. For example, the 64-bit in snowflake is represented as shown in the figure below:

        Bit 0: Sign bit (marking positive and negative), always 0, useless, don't care. Bits 1~41: 41 bits in total, used to represent timestamp, unit is millisecond, can support 2^41 milliseconds (about 69 years) Bits
        42~52: 10 bits in total, generally speaking, the first 5 bits represent the computer room ID, the last 5 digits represent the machine ID (can be adjusted according to the actual situation in the actual project), so that nodes in different clusters/computer rooms can be distinguished, so that 32 IDCs can be represented, and each IDC can have 32 machines.
        Bits 53~64: 12 bits in total, used to represent the serial number. The serial number is an auto-increment value, representing the maximum number of IDs that a single machine can generate per millisecond (2^12 = 4096), which means that a single machine can generate up to 4096 unique IDs per millisecond.

        Theoretically, the QPS of the snowflake solution is about 409.6w/s. This allocation method can ensure that the IDs generated by any machine in any IDC in any millisecond are different.

        Three  distributed ID microservices

        From the above analysis, we can see that each solution has its own advantages and disadvantages. We now refer to the Meituan Leaf solution to implement our own distributed ID.

        3.1 Meituan Leaf solution implementation

        The original MySQL solution had to read and write to the database every time to obtain the ID, which caused great pressure on the database. Change to batch acquisition, and obtain the value of one segment (step determines the size) number segment each time. After use, go to the database to obtain a new number segment, which can greatly reduce the pressure on the database.

       3.1.1 Advantages

        Leaf services can be easily expanded linearly, and their performance can fully support most business scenarios. The ID number is a 64-bit number with an increasing trend of 8 bytes, which meets the primary key requirements of the above database storage.
        High disaster tolerance: The Leaf service has an internal number segment cache. Even if the DB is down, Leaf can still provide services to the outside world normally within a short period of time.
        The size of max_id can be customized, which is very convenient for business migration from the original ID method.


       3.1.2 Disadvantages

        The ID number is not random enough and can leak information about the number of numbers issued, making it unsafe.
        TP999 data fluctuates greatly. When the number segment is used up, there will still be a wait for the I/O of updating the database when obtaining a new number segment. The tg999 data will occasionally have spikes.
        DB outage may cause the entire system to become unavailable.

        3.1.3  Optimization

        The timing for Leaf to retrieve the number segment is when the number segment is consumed, which means that the ID issuance time at the critical point of the number segment depends on the next time the number segment is retrieved from the DB, and the requests that come in during this period The thread may also be blocked because the DB number segment is not retrieved. If the network requesting the DB and the performance of the DB are stable, this situation will have little impact on the system. However, if the network jitters when fetching the DB, or a slow query occurs in the DB, the response time of the entire system will slow down.
        For this reason, we hope that the process of DB retrieving the number segment can be non-blocking, and there is no need to block the request thread when the DB is retrieving the number segment. That is, when the number segment is consumed to a certain point, the next number segment will be asynchronously loaded into the memory. middle. There is no need to wait until the number range is exhausted before updating the number range. Doing this can greatly reduce the TP999 indicator of the system.
        Using the double buffer method, there are two number segment buffer segments inside the Leaf service. When 10% of the current number segment has been issued, if the next number segment has not been updated, another update thread is started to update the next number segment. After all the current number segments are delivered, if the next number segment is ready, the next number segment will be switched to the current segment and then delivered, and the cycle repeats. It is generally recommended that the segment length be set to 600 times the call QPS (10 minutes) during peak service periods, so that even if the DB is down, Leaf can still continue to send calls for 10-20 minutes without being affected. Each time a request comes, the status of the next number segment will be determined and this number segment will be updated, so occasional network jitters will not affect the update of the next number segment.

Four: Distributed ID actual combat

        Database configuration

CREATE DATABASE qiyu_live_common CHARACTER set utf8mb3 
COLLATE=utf8_bin;


CREATE TABLE `t_id_generate_config` (
 `id` int NOT NULL AUTO_INCREMENT COMMENT '主键 id',
 `remark` varchar(255) CHARACTER SET utf8mb4 COLLATE 
utf8mb4_unicode_ci DEFAULT NULL COMMENT '描述',
 `next_threshold` bigint DEFAULT NULL COMMENT '当前 id 所在阶段的阈
值',
 `init_num` bigint DEFAULT NULL COMMENT '初始化值',
 `current_start` bigint DEFAULT NULL COMMENT '当前 id 所在阶段的开始
值',
 `step` int DEFAULT NULL COMMENT 'id 递增区间',
 `is_seq` tinyint DEFAULT NULL COMMENT '是否有序(0 无序,1 有序)',
 `id_prefix` varchar(60) CHARACTER SET utf8mb4 COLLATE 
utf8mb4_unicode_ci DEFAULT NULL COMMENT '业务前缀码,如果没有则返回
时不携带',
 `version` int NOT NULL DEFAULT '0' COMMENT '乐观锁版本号',
 `create_time` datetime DEFAULT CURRENT_TIMESTAMP COMMENT '创建时
间',
 `update_time` datetime DEFAULT CURRENT_TIMESTAMP ON UPDATE 
CURRENT_TIMESTAMP COMMENT '更新时间',
 PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=8 DEFAULT CHARSET=utf8mb4 
COLLATE=utf8mb4_unicode_ci;

        Insert record

INSERT INTO `t_id_generate_config` (`id`, `remark`, 
`next_threshold`, `init_num`, `current_start`, `step`, `is_seq`, 
`id_prefix`, `version`, `create_time`, `update_time`)
VALUES
 (1, '用户 id 生成策略', 10050, 10000, 10000, 50, 0, 
'user_id', 0, '2023-05-23 12:38:21', '2023-05-23 23:31:45');

        Build springboot project and configuration files

        1. Create two mavens and import maven dependencies

        Import maven dependencies

        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-boot-starter</artifactId>
            <version>${mybatis-plus.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.dubbo</groupId>
            <artifactId>dubbo-spring-boot-starter</artifactId>
            <version>${dubbo.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <exclusions>
                <exclusion>
                    <artifactId>log4j-to-slf4j</artifactId>
                    <groupId>org.apache.logging.log4j</groupId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>${qiyu-mysql.version}</version>
        </dependency>
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
        </dependency>
        <dependency>
            <groupId>org.idea</groupId>
            <artifactId>qiyu-live-id-generate-interface</artifactId>
            <version>1.0-SNAPSHOT</version>
            <scope>compile</scope>
        </dependency>

        Configuration file

spring:
  application:
    name: qiyu-live-id-generate-provider
  datasource:
    driver-class-name: com.mysql.cj.jdbc.Driver
    #    访问主库
    url: jdbc:mysql://192.168.1.128:8808/qiyu_live_common?useUnicode=true&characterEncoding=utf8
    username: root
    password: root

        In the following module, the basic configuration policy enumeration and external interface are generated.

       Create an id generation strategy enumeration class

package org.qiyu.live.id.generate.enums;

/**
 * @Author idea
 * @Date: Created in 17:55 2023/6/13
 * @Description
 */
public enum IdTypeEnum {

    USER_ID(1,"用户id生成策略");

    int code;
    String desc;

    IdTypeEnum(int code, String desc) {
        this.code = code;
        this.desc = desc;
    }

    public int getCode() {
        return code;
    }

    public String getDesc() {
        return desc;
    }
}

        Generate external interface method

package org.qiyu.live.id.generate.interfaces;

/**
 * @Author idea
 * @Date: Created in 19:45 2023/5/25
 * @Description
 */
public interface IdGenerateRpc {
    /**
     * 获取有序id
     *
     * @param id
     * @return
     */
    Long getSeqId(Integer id);

    /**
     * 获取无序id
     *
     * @param id
     * @return
     */
    Long getUnSeqId(Integer id);

}

        Next, implement it in the id generation module

        Create a database po class (here is the database id configuration policy table)

package com.laoyang.id.dao.po;

import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;

import java.util.Date;

/**
 * @Author idea
 * @Date: Created in 19:59 2023/5/23
 * @Description
 */
@TableName("t_id_gengrate_config")
public class IdGeneratePO {

    @TableId(type = IdType.AUTO)
    private Integer id;

    /**
     * id备注描述
     */
    private String remark;

    /**
     * 初始化值
     */
    private long initNum;

    /**
     * 步长
     */
    private int step;

    /**
     * 是否是有序的id
     */
    private int isSeq;

    /**
     * 当前id所在阶段的开始值
     */
    private long currentStart;

    /**
     * 当前id所在阶段的阈值
     */
    private long nextThreshold;

    /**
     * 业务代码前缀
     */
    private String idPrefix;

    /**
     * 乐观锁版本号
     */
    private int version;

    private Date createTime;

    private Date updateTime;

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    public String getRemark() {
        return remark;
    }

    public void setRemark(String remark) {
        this.remark = remark;
    }

    public long getInitNum() {
        return initNum;
    }

    public void setInitNum(long initNum) {
        this.initNum = initNum;
    }

    public int getStep() {
        return step;
    }

    public void setStep(int step) {
        this.step = step;
    }

    public long getCurrentStart() {
        return currentStart;
    }

    public void setCurrentStart(long currentStart) {
        this.currentStart = currentStart;
    }

    public long getNextThreshold() {
        return nextThreshold;
    }

    public void setNextThreshold(long nextThreshold) {
        this.nextThreshold = nextThreshold;
    }

    public String getIdPrefix() {
        return idPrefix;
    }

    public void setIdPrefix(String idPrefix) {
        this.idPrefix = idPrefix;
    }

    public int getVersion() {
        return version;
    }

    public void setVersion(int version) {
        this.version = version;
    }

    public Date getCreateTime() {
        return createTime;
    }

    public void setCreateTime(Date createTime) {
        this.createTime = createTime;
    }

    public Date getUpdateTime() {
        return updateTime;
    }

    public void setUpdateTime(Date updateTime) {
        this.updateTime = updateTime;
    }

    public int getIsSeq() {
        return isSeq;
    }

    public void setIsSeq(int isSeq) {
        this.isSeq = isSeq;
    }

    @Override
    public String toString() {
        return "IdGeneratePO{" +
                "id=" + id +
                ", remark='" + remark + '\'' +
                ", initNum=" + initNum +
                ", step=" + step +
                ", isSeq=" + isSeq +
                ", currentStart=" + currentStart +
                ", nextThreshold=" + nextThreshold +
                ", idPrefix='" + idPrefix + '\'' +
                ", version=" + version +
                ", createTime=" + createTime +
                ", updateTime=" + updateTime +
                '}';
    }
}

        Generate mapper mapping class, note that optimistic locking is added during insertion, pay attention to this sql

package com.laoyang.id.dao.mapper;

import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.apache.ibatis.annotations.Mapper;
import org.apache.ibatis.annotations.Param;
import org.apache.ibatis.annotations.Select;
import org.apache.ibatis.annotations.Update;
import com.laoyang.id.dao.po.IdGeneratePO;

import java.util.List;

/**
 * @Author idea
 * @Date: Created in 19:47 2023/5/25
 * @Description
 */
@Mapper
public interface IdGenerateMapper extends BaseMapper<IdGeneratePO> {

    @Update("update t_id_gengrate_config set next_threshold=next_threshold+step," +
            "current_start=current_start+step,version=version+1 where id =#{id} and version=#{version}")
    int updateNewIdCountAndVersion(@Param("id")int id,@Param("version")int version);

    @Select("select * from t_id_gengrate_config")
    List<IdGeneratePO> selectAll();
}

        Create the bo class under service to generate ordered id and unordered id objects

        

package com.laoyang.id.service.bo;

import java.util.concurrent.atomic.AtomicLong;

/**
 * @Author idea
 * @Date: Created in 20:00 2023/5/25
 * @Description 有序id的BO对象
 */
public class LocalSeqIdBO {

    private int id;
    /**
     * 在内存中记录的当前有序id的值
     */
    private AtomicLong currentNum;

    /**
     * 当前id段的开始值
     */
    private Long currentStart;
    /**
     * 当前id段的结束值
     */
    private Long nextThreshold;

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    public AtomicLong getCurrentNum() {
        return currentNum;
    }

    public void setCurrentNum(AtomicLong currentNum) {
        this.currentNum = currentNum;
    }

    public Long getCurrentStart() {
        return currentStart;
    }

    public void setCurrentStart(Long currentStart) {
        this.currentStart = currentStart;
    }

    public Long getNextThreshold() {
        return nextThreshold;
    }

    public void setNextThreshold(Long nextThreshold) {
        this.nextThreshold = nextThreshold;
    }
}
package com.laoyang.id.service.bo;

import java.util.concurrent.ConcurrentLinkedQueue;

/**
 * @Author idea
 * @Date: Created in 20:32 2023/5/26
 * @Description 无序id的BO对象
 */
public class LocalUnSeqIdBO {

    private int id;
    /**
     * 提前将无序的id存放在这条队列中
     */
    private ConcurrentLinkedQueue<Long> idQueue;
    /**
     * 当前id段的开始值
     */
    private Long currentStart;
    /**
     * 当前id段的结束值
     */
    private Long nextThreshold;

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    public ConcurrentLinkedQueue<Long> getIdQueue() {
        return idQueue;
    }

    public void setIdQueue(ConcurrentLinkedQueue<Long> idQueue) {
        this.idQueue = idQueue;
    }

    public Long getCurrentStart() {
        return currentStart;
    }

    public void setCurrentStart(Long currentStart) {
        this.currentStart = currentStart;
    }

    public Long getNextThreshold() {
        return nextThreshold;
    }

    public void setNextThreshold(Long nextThreshold) {
        this.nextThreshold = nextThreshold;
    }
}

        Generate service class to generate ordered ID and unordered ID

package com.laoyang.id.service;

/**
 * @Author idea
 * @Date: Created in 19:58 2023/5/25
 * @Description
 */
public interface IdGenerateService {

    /**
     * 获取有序id
     *
     * @param id
     * @return
     */
    Long getSeqId(Integer id);

    /**
     * 获取无序id
     *
     * @param id
     * @return
     */
    Long getUnSeqId(Integer id);
}

        Implement ordered id and unordered id methods (here is the key, mainly using atomic classes, some synchronization operations, etc., thread pool)

package com.laoyang.id.service.impl;

import jakarta.annotation.Resource;
import com.laoyang.id.dao.mapper.IdGenerateMapper;
import com.laoyang.id.dao.po.IdGeneratePO;
import com.laoyang.id.service.IdGenerateService;
import com.laoyang.id.service.bo.LocalSeqIdBO;
import com.laoyang.id.service.bo.LocalUnSeqIdBO;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.stereotype.Service;

import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicLong;

/**
 * @Author idea
 * @Date: Created in 19:58 2023/5/25
 * @Description
 */
@Service
public class IdGenerateServiceImpl implements IdGenerateService, InitializingBean {

    @Resource
    private IdGenerateMapper idGenerateMapper;

    private static final Logger LOGGER = LoggerFactory.getLogger(IdGenerateServiceImpl.class);
    private static Map<Integer, LocalSeqIdBO> localSeqIdBOMap = new ConcurrentHashMap<>();
    private static Map<Integer, LocalUnSeqIdBO> localUnSeqIdBOMap = new ConcurrentHashMap<>();
    private static ThreadPoolExecutor threadPoolExecutor = new ThreadPoolExecutor(8, 16, 3, TimeUnit.SECONDS, new ArrayBlockingQueue<>(1000),
            new ThreadFactory() {
                @Override
                public Thread newThread(Runnable r) {
                    Thread thread = new Thread(r);
                    thread.setName("id-generate-thread-" + ThreadLocalRandom.current().nextInt(1000));
                    return thread;
                }
            });
    private static final float UPDATE_RATE = 0.50f;
    private static final int SEQ_ID = 1;
    private static Map<Integer, Semaphore> semaphoreMap = new ConcurrentHashMap<>();

    @Override
    public Long getUnSeqId(Integer id) {
        if (id == null) {
            LOGGER.error("[getSeqId] id is error,id is {}", id);
            return null;
        }
        LocalUnSeqIdBO localUnSeqIdBO = localUnSeqIdBOMap.get(id);
        if (localUnSeqIdBO == null) {
            LOGGER.error("[getUnSeqId] localUnSeqIdBO is null,id is {}", id);
            return null;
        }
        Long returnId = localUnSeqIdBO.getIdQueue().poll();
        if (returnId == null) {
            LOGGER.error("[getUnSeqId] returnId is null,id is {}", id);
            return null;
        }
        this.refreshLocalUnSeqId(localUnSeqIdBO);
        return returnId;
    }

    /**
     *
     * @param id 传的是对应的业务id
     * @return
     */
    @Override
    public Long getSeqId(Integer id) {
        if (id == null) {
            LOGGER.error("[getSeqId] id is error,id is {}", id);
            return null;
        }
        LocalSeqIdBO localSeqIdBO = localSeqIdBOMap.get(id);
        if (localSeqIdBO == null) {
            LOGGER.error("[getSeqId] localSeqIdBO is null,id is {}", id);
            return null;
        }
        this.refreshLocalSeqId(localSeqIdBO);
        long returnId = localSeqIdBO.getCurrentNum().incrementAndGet();
        if (returnId > localSeqIdBO.getNextThreshold()) {
            //同步去刷新 可能是高并发下还未更新本地数据
            LOGGER.error("[getSeqId] id is over limit,id is {}", id);
            return null;
        }
        return returnId;
    }

    /**
     * 刷新本地有序id段
     *
     * @param localSeqIdBO
     */
    private void refreshLocalSeqId(LocalSeqIdBO localSeqIdBO) {
        long step = localSeqIdBO.getNextThreshold() - localSeqIdBO.getCurrentStart();
        if (localSeqIdBO.getCurrentNum().get() - localSeqIdBO.getCurrentStart() > step * UPDATE_RATE) {
            Semaphore semaphore = semaphoreMap.get(localSeqIdBO.getId());
            if (semaphore == null) {
                LOGGER.error("semaphore is null,id is {}", localSeqIdBO.getId());
                return;
            }
            boolean acquireStatus = semaphore.tryAcquire();
            if (acquireStatus) {
                LOGGER.info("开始尝试进行本地id段的同步操作");
                //异步进行同步id段操作
                threadPoolExecutor.execute(new Runnable() {
                    @Override
                    public void run() {
                        try {
                            IdGeneratePO idGeneratePO = idGenerateMapper.selectById(localSeqIdBO.getId());
                            tryUpdateMySQLRecord(idGeneratePO);
                        } catch (Exception e) {
                            LOGGER.error("[refreshLocalSeqId] error is ", e);
                        } finally {
                            semaphoreMap.get(localSeqIdBO.getId()).release();
                            LOGGER.info("本地有序id段同步完成,id is {}", localSeqIdBO.getId());
                        }
                    }
                });
            }
        }
    }

    /**
     * 刷新本地无序id段
     *
     * @param localUnSeqIdBO
     */
    private void refreshLocalUnSeqId(LocalUnSeqIdBO localUnSeqIdBO) {
        long begin = localUnSeqIdBO.getCurrentStart();
        long end = localUnSeqIdBO.getNextThreshold();
        long remainSize = localUnSeqIdBO.getIdQueue().size();
        //如果使用剩余空间不足25%,则进行刷新
        if ((end - begin) * 0.35 > remainSize) {
            LOGGER.info("本地无序id段同步开始,id is {}", localUnSeqIdBO.getId());
            Semaphore semaphore = semaphoreMap.get(localUnSeqIdBO.getId());
            if (semaphore == null) {
                LOGGER.error("semaphore is null,id is {}", localUnSeqIdBO.getId());
                return;
            }
            boolean acquireStatus = semaphore.tryAcquire();
            if (acquireStatus) {
                threadPoolExecutor.execute(new Runnable() {
                    @Override
                    public void run() {
                        try {
                            IdGeneratePO idGeneratePO = idGenerateMapper.selectById(localUnSeqIdBO.getId());
                            tryUpdateMySQLRecord(idGeneratePO);
                        } catch (Exception e) {
                            LOGGER.error("[refreshLocalUnSeqId] error is ", e);
                        } finally {
                            semaphoreMap.get(localUnSeqIdBO.getId()).release();
                            LOGGER.info("本地无序id段同步完成,id is {}", localUnSeqIdBO.getId());
                        }
                    }
                });
            }
        }
    }

    //bean初始化的时候会回调到这里
    @Override
    public void afterPropertiesSet() throws Exception {
        List<IdGeneratePO> idGeneratePOList = idGenerateMapper.selectAll();
        for (IdGeneratePO idGeneratePO : idGeneratePOList) {
            LOGGER.info("服务刚启动,抢占新的id段");
            tryUpdateMySQLRecord(idGeneratePO);
            semaphoreMap.put(idGeneratePO.getId(), new Semaphore(1));
        }
    }

    /**
     * 更新mysql里面的分布式id的配置信息,占用相应的id段
     * 同步执行,很多的网络IO,性能较慢
     *
     * @param idGeneratePO
     */
    private void tryUpdateMySQLRecord(IdGeneratePO idGeneratePO) {
        int updateResult = idGenerateMapper.updateNewIdCountAndVersion(idGeneratePO.getId(), idGeneratePO.getVersion());
        if (updateResult > 0) {
            localIdBOHandler(idGeneratePO);
            return;
        }
        //重试进行更新
        for (int i = 0; i < 3; i++) {
            idGeneratePO = idGenerateMapper.selectById(idGeneratePO.getId());
            updateResult = idGenerateMapper.updateNewIdCountAndVersion(idGeneratePO.getId(), idGeneratePO.getVersion());
            if (updateResult > 0) {
                localIdBOHandler(idGeneratePO);
                return;
            }
        }
        throw new RuntimeException("表id段占用失败,竞争过于激烈,id is " + idGeneratePO.getId());
    }

    /**
     * 专门处理如何将本地ID对象放入到Map中,并且进行初始化的
     *
     * @param idGeneratePO
     */
    private void localIdBOHandler(IdGeneratePO idGeneratePO) {
        long currentStart = idGeneratePO.getCurrentStart();
        long nextThreshold = idGeneratePO.getNextThreshold();
        long currentNum = currentStart;
        if (idGeneratePO.getIsSeq() == SEQ_ID) {
            LocalSeqIdBO localSeqIdBO = new LocalSeqIdBO();
            AtomicLong atomicLong = new AtomicLong(currentNum);
            localSeqIdBO.setId(idGeneratePO.getId());
            localSeqIdBO.setCurrentNum(atomicLong);
            localSeqIdBO.setCurrentStart(currentStart);
            localSeqIdBO.setNextThreshold(nextThreshold);
            localSeqIdBOMap.put(localSeqIdBO.getId(), localSeqIdBO);
        } else {
            LocalUnSeqIdBO localUnSeqIdBO = new LocalUnSeqIdBO();
            localUnSeqIdBO.setCurrentStart(currentStart);
            localUnSeqIdBO.setNextThreshold(nextThreshold);
            localUnSeqIdBO.setId(idGeneratePO.getId());
            long begin = localUnSeqIdBO.getCurrentStart();
            long end = localUnSeqIdBO.getNextThreshold();
            List<Long> idList = new ArrayList<>();
            for (long i = begin; i < end; i++) {
                idList.add(i);
            }
            //将本地id段提前打乱,然后放入到队列中
            Collections.shuffle(idList);
            ConcurrentLinkedQueue<Long> idQueue = new ConcurrentLinkedQueue<>();
            idQueue.addAll(idList);
            localUnSeqIdBO.setIdQueue(idQueue);
            localUnSeqIdBOMap.put(localUnSeqIdBO.getId(), localUnSeqIdBO);
        }
    }
}

        Finally create the startup class

package com.laoyang.id;

import jakarta.annotation.Resource;
import org.apache.dubbo.config.spring.context.annotation.EnableDubbo;
import com.laoyang.id.service.IdGenerateService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.WebApplicationType;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;

import java.util.HashSet;

/**
 * @Author idea
 * @Date: Created in 19:45 2023/5/25
 * @Description
 */
@SpringBootApplication
public class IdGenerateApplication implements CommandLineRunner {

    private static final Logger LOGGER = LoggerFactory.getLogger(IdGenerateApplication.class);

    @Resource
    private IdGenerateService idGenerateService;

    public static void main(String[] args) {
        SpringApplication springApplication = new SpringApplication(IdGenerateApplication.class);
        springApplication.setWebApplicationType(WebApplicationType.NONE);
        springApplication.run(args);
    }

    @Override
    public void run(String... args) throws Exception {
        HashSet<Long> idSet = new HashSet<>();
        for (int i = 0; i < 1500; i++) {
            Long id = idGenerateService.getSeqId(1);
            System.out.println(id);
            idSet.add(id);
        }
        System.out.println(idSet.size());
    }
}

        Eventually the output will be printed on the console!

Guess you like

Origin blog.csdn.net/qq_67801847/article/details/133254215