In Internet business systems, various IDs are involved, such as payment IDs and refund IDs in payment systems. What are the general solutions for generating IDs? Especially in complex distributed system business scenarios, it is very important which solution we should adopt that suits us. Let's list them one by one below, not all of them are suitable. These solutions are for reference only, and may be useful to you.

1. Distributed ID

1. What is distributed ID

In daily development, we need to use unique IDs for various data in the system. For example, a user ID corresponds to and only corresponds to one person, a product ID corresponds to and only corresponds to a product, and an order ID corresponds to and only corresponds to an order.

Take the MySQL database as an example:
When the amount of business data is not large, a single database and single table can fully support the existing business, and even a larger data can be handled by a MySQL master-slave synchronization read and write separation .
However, as the data grows day by day, the master-slave synchronization can no longer be supported, so the database needs to be divided into databases and tables , but after the database is divided into tables, a unique ID is required to identify a piece of data. The self-incrementing ID of the database obviously cannot meet the requirements. Requirements; special ones such as orders and coupons also need to be identified by a unique ID. At this time, a system that can generate a globally unique ID is very necessary. Then this globally unique ID is called a distributed ID.

2. Characteristics of Distributed ID

Uniqueness: Ensure that the generated ID is unique across the network.
Orderly increment: ensure that the generated ID is sequentially incremented by a certain number for a certain user or business.
High Availability: Ensure that IDs can be generated correctly at any time.
With time: The ID contains the time, and you can know which day the transaction is at a glance.

2. Distributed ID generation scheme

1. UUID

The core idea of the algorithm is to combine the machine's network card, local time, and a random number to generate a UUID.

advantage:

Simple code implementation

Locally generated, no performance issues, no risk of high availability

Unique in the world, easy data migration

shortcoming:

The length is too long, the storage is redundant, and it is unordered and unreadable, and the query efficiency is low

The IDs generated each time are out of order and do not satisfy the trend of increasing

UUID is a string, and it is relatively long, takes up a lot of space, and the query efficiency is low

ID has no meaning, poor readability

2. Database self-increment ID

Use the database's id auto-increment strategy, such as MySQL's auto_increment. And you can use the two databases to set different step lengths and generate non-duplicate ID strategies to achieve high availability.

Advantages: The IDs generated by the database are absolutely ordered, and the implementation of high availability is simple

Disadvantages: Independent deployment of database instances is required, which is costly and has performance bottlenecks

3. Generate IDs in batches

Multiple IDs are generated in batches on demand at a time. Each generation needs to access the database, modify the database to the maximum ID value, and record the current value and the maximum value in memory.

Advantages: Avoid accessing the database every time an ID is generated and bring pressure, improve performance

Disadvantages: It belongs to the local generation strategy, there is a single point of failure , and the ID is not continuous due to service restart

4. Redis generates ID

All Redis command operations are single-threaded, and it provides self-incrementing atomic commands such as incr and increby, so it can ensure that the generated ID must be unique and ordered.

Advantages: orderly increment, strong readability, high performance . It does not depend on the database, it is flexible and convenient, and its performance is better than that of the database; the digital ID is naturally sorted, which is very helpful for pagination or results that need to be sorted.

Disadvantages: Occupies bandwidth and relies on Redis : If there is no Redis in the system, new components need to be introduced to increase the complexity of the system ; the workload of coding and configuration is relatively large.
Considering the performance bottleneck of a single node, a Redis cluster can be used to obtain higher throughput. Suppose there are 5 Redis in a cluster. The values that can be initialized for each Redis are 1, 2, 3, 4, 5 respectively, and then the step size is 5. The ID generated by each Redis is:

A：1, 6, 11, 16, 21
B：2, 7, 12, 17, 22
C：3, 8, 13, 18, 23
D：4, 9, 14, 19, 24
E：5, 10, 15, 20, 25

It is good to decide which machine to load to, and it will be difficult to modify it in the future. The step size and initial value must be determined in advance. Using Redis cluster can also prevent the problem of single point of failure. In addition, it is more suitable to use Redis to generate serial numbers starting from 0 every day. For example, order number = date + self-increment number of the day. A Key can be generated in Redis every day and accumulated using INCR.

5. Twitter's snowflake algorithm

Twitter's open source snowflake is composed of time stamp + machine + increment sequence, the basic trend is increasing, and the performance is very high.

Snowflake generates a Long type value, and the Long type data occupies 8 bytes, which is 64 bits. SnowFlake splits 64, and each part has a different meaning. Of course, the number of digits of the machine code and serial number can be customized.

Divide the 64-bit into multiple segments, and mark the machine, time, etc. separately. For example, the 64-bit in snowflake are respectively represented as shown in the following figure (picture from the Internet):

Sign bit (1bit): The reserved sign bit, which is always zero. (Because the long type is signed in java, the highest bit is the sign bit, the positive number is 0, the negative number is 1, and the IDs used in the actual system are generally positive numbers, so the highest bit is 0)
Timestamp bit (41bit): The number of milliseconds that a 41-bit timestamp can hold is 2 to the 41st power, and the number of milliseconds used in a year is: 365 * 24 * 60 * 60 * 1000. Through calculation, it can be seen that Math.pow(2, 41) / (365 * 24 * 60 * 60 * 1000L);the result is approximately equal to 69.73 years

The time epoch of ShardingSphere's snowflake algorithm starts at midnight on November 1, 2016, and can be used until 2086, which is believed to meet the requirements of most systems.

Worker process bit (10bit): This flag is unique within the Java process. If it is a distributed application deployment, it should be ensured that the id of each worker process is different. The value defaults to 0 and can be set through properties. 10-bit machines can represent 1024 machines respectively. If we have a need for IDC division, we can also divide 10-bit into 5-bit for IDC and 5-bit for working machines. In this way, 32 IDCs can be represented, and each IDC can have 32 machines, which can be defined according to their own needs.
Sequence number bit (12bit): This sequence is used to generate different IDs within the same millisecond. If the number generated in this millisecond exceeds 4096 (2 to the power of 12), the generator will wait until the next millisecond to continue generating. This 12-bit count supports each node to generate up to 1 << 12 = 4096 IDs per millisecond (same machine, same moment)

Advantages: Locally generated, does not depend on middleware. The generated distributed id is small enough, only 8 bytes, and it is incremented

It can meet the requirement that IDs are not repeated in a high-concurrency distributed system environment

High generation efficiency

Based on the timestamp, the basic orderly increment can be guaranteed

Does not depend on third-party libraries or middleware

Generated idwith timing and uniqueness

Disadvantages: clock callback problem , strongly dependent on the server time, if time callback occurs, duplicate ids may appear

6. Baidu UidGenerator

For details, please refer to the official website description: https://github.com/baidu/uid-generator/blob/master/README.zh_cn.md

7. Meituan Leaf

Leaf is an open source distributed ID generator of Meituan, which can guarantee global uniqueness, increasing trend, monotonically increasing, and information security. It also mentions the comparison of several distributed schemes, but it also needs middleware such as relational database and ZooKeeper .

https://github.com/Meituan-Dianping/Leaf/tree/master/leaf-core

For details, please refer to the official website: https://tech.meituan.com/2017/04/21/mt-leaf.html

8. Tinyid

Tinyid is developed by Didi, Github address: https://github.com/didi/tinyid.
Tinyid is implemented based on the principle of number segment mode. It is exactly the same as Leaf. Each service obtains a number segment (1000,2000], (2000,3000], (3000,4000]

insert image description here

Having said so much, today we will mainly talk about Twitter's open source snowflake

Three, snowflake

1. Process

2. java code implementation

In the following example, 41bit is for timestamp, 5bit is for IDC, 5bit is for working machine, and 12bit is for serial number. The code is hard-coded. If some bits need to be dynamically adjusted, they can be defined in member properties. The calculation process requires some fundamentals of bit manipulation.

import java.util.Date;

/**
 * @Author allen
 * @Description TODO
 * @Date 2023-07-26 9:51
 * @Version 1.0
 */
public class SnowFlakeUtil {

    private static SnowFlakeUtil snowFlakeUtil;
    static {
        snowFlakeUtil = new SnowFlakeUtil();
    }

    // 初始时间戳(纪年)，可用雪花算法服务上线时间戳的值
    // 1650789964886：2022-04-24 16:45:59
    private static final long INIT_EPOCH = 1650789964886L;

    // 时间位取&
    private static final long TIME_BIT = 0b1111111111111111111111111111111111111111110000000000000000000000L;

    // 记录最后使用的毫秒时间戳，主要用于判断是否同一毫秒，以及用于服务器时钟回拨判断
    private long lastTimeMillis = -1L;

    // dataCenterId占用的位数
    private static final long DATA_CENTER_ID_BITS = 5L;

    // dataCenterId占用5个比特位，最大值31
    // 0000000000000000000000000000000000000000000000000000000000011111
    private static final long MAX_DATA_CENTER_ID = ~(-1L << DATA_CENTER_ID_BITS);

    // dataCenterId
    private long dataCenterId;

    // workId占用的位数
    private static final long WORKER_ID_BITS = 5L;

    // workId占用5个比特位，最大值31
    // 0000000000000000000000000000000000000000000000000000000000011111
    private static final long MAX_WORKER_ID = ~(-1L << WORKER_ID_BITS);

    // workId
    private long workerId;

    // 最后12位，代表每毫秒内可产生最大序列号，即 2^12 - 1 = 4095
    private static final long SEQUENCE_BITS = 12L;

    // 掩码（最低12位为1，高位都为0），主要用于与自增后的序列号进行位与，如果值为0，则代表自增后的序列号超过了4095
    // 0000000000000000000000000000000000000000000000000000111111111111
    private static final long SEQUENCE_MASK = ~(-1L << SEQUENCE_BITS);

    // 同一毫秒内的最新序号，最大值可为 2^12 - 1 = 4095
    private long sequence;

    // workId位需要左移的位数 12
    private static final long WORK_ID_SHIFT = SEQUENCE_BITS;

    // dataCenterId位需要左移的位数 12+5
    private static final long DATA_CENTER_ID_SHIFT = SEQUENCE_BITS + WORKER_ID_BITS;

    // 时间戳需要左移的位数 12+5+5
    private static final long TIMESTAMP_SHIFT = SEQUENCE_BITS + WORKER_ID_BITS + DATA_CENTER_ID_BITS;

    /**
     * 无参构造
     */
    public SnowFlakeUtil() {
        this(1, 1);
    }

    /**
     * 有参构造
     * @param dataCenterId
     * @param workerId
     */
    public SnowFlakeUtil(long dataCenterId, long workerId) {
        // 检查dataCenterId的合法值
        if (dataCenterId < 0 || dataCenterId > MAX_DATA_CENTER_ID) {
            throw new IllegalArgumentException(
                    String.format("dataCenterId 值必须大于 0 并且小于 %d", MAX_DATA_CENTER_ID));
        }
        // 检查workId的合法值
        if (workerId < 0 || workerId > MAX_WORKER_ID) {
            throw new IllegalArgumentException(String.format("workId 值必须大于 0 并且小于 %d", MAX_WORKER_ID));
        }
        this.workerId = workerId;
        this.dataCenterId = dataCenterId;
    }

    /**
     * 获取唯一ID
     * @return
     */
    public static Long getSnowFlakeId() {
        return snowFlakeUtil.nextId();
    }

    /**
     * 通过雪花算法生成下一个id，注意这里使用synchronized同步
     * @return 唯一id
     */
    public synchronized long nextId() {
        long currentTimeMillis = System.currentTimeMillis();
        //System.out.println(currentTimeMillis);
        // 当前时间小于上一次生成id使用的时间，可能出现服务器时钟回拨问题
        if (currentTimeMillis < lastTimeMillis) {
            throw new RuntimeException(
                    String.format("可能出现服务器时钟回拨问题，请检查服务器时间。当前服务器时间戳：%d，上一次使用时间戳：%d", currentTimeMillis,
                            lastTimeMillis));
        }
        if (currentTimeMillis == lastTimeMillis) {
            // 还是在同一毫秒内，则将序列号递增1，序列号最大值为4095
            // 序列号的最大值是4095，使用掩码（最低12位为1，高位都为0）进行位与运行后如果值为0，则自增后的序列号超过了4095
            // 那么就使用新的时间戳
            sequence = (sequence + 1) & SEQUENCE_MASK;
            if (sequence == 0) {
                currentTimeMillis = getNextMillis(lastTimeMillis);
            }
        } else { // 不在同一毫秒内，则序列号重新从0开始，序列号最大值为4095
            sequence = 0;
        }
        // 记录最后一次使用的毫秒时间戳
        lastTimeMillis = currentTimeMillis;
        // 核心算法，将不同部分的数值移动到指定的位置，然后进行或运行
        // <<：左移运算符, 1 << 2 即将二进制的 1 扩大 2^2 倍
        // |：位或运算符, 是把某两个数中, 只要其中一个的某一位为1, 则结果的该位就为1
        // 优先级：<< > |
        return
                // 时间戳部分
                ((currentTimeMillis - INIT_EPOCH) << TIMESTAMP_SHIFT)
                        // 数据中心部分
                        | (dataCenterId << DATA_CENTER_ID_SHIFT)
                        // 机器表示部分
                        | (workerId << WORK_ID_SHIFT)
                        // 序列号部分
                        | sequence;
    }

    /**
     * 获取指定时间戳的接下来的时间戳，也可以说是下一毫秒
     * @param lastTimeMillis 指定毫秒时间戳
     * @return 时间戳
     */
    private long getNextMillis(long lastTimeMillis) {
        long currentTimeMillis = System.currentTimeMillis();
        while (currentTimeMillis <= lastTimeMillis) {
            currentTimeMillis = System.currentTimeMillis();
        }
        return currentTimeMillis;
    }

    /**
     * 获取随机字符串,length=13
     * @return
     */
    public static String getRandomStr() {
        return Long.toString(getSnowFlakeId(), Character.MAX_RADIX);
    }

    /**
     * 从ID中获取时间
     * @param id 由此类生成的ID
     * @return
     */
    public static Date getTimeBySnowFlakeId(long id) {
        return new Date(((TIME_BIT & id) >> 22) + INIT_EPOCH);
    }

    public static void main(String[] args) {
        SnowFlakeUtil snowFlakeUtil = new SnowFlakeUtil();
        long id = snowFlakeUtil.nextId();
        System.out.println(id);
        Date date = SnowFlakeUtil.getTimeBySnowFlakeId(id);
        System.out.println(date);
        long time = date.getTime();
        System.out.println(time);
        System.out.println(getRandomStr());

/*        System.out.println("============================");
        long startTime = System.currentTimeMillis();
        for (int i = 0; i < 10000000; i++) {
            long id2 = snowFlakeUtil.nextId();
            System.out.println(id2);
        }
        System.out.println(System.currentTimeMillis() - startTime);*/


    }

}

Mainly this: long id = snowFlakeUtil.nextId();

Clock synchronization problem solution:

Snowflake Algorithm-Java Implementation-A Method to Solve Clock Callback_Snowflake Algorithm Clock Callback_fierys' Blog-CSDN Blog

This code is based on another blogger's solution, you can learn from it, it is for reference only

import java.io.IOException;
import java.time.LocalDateTime;
import java.time.ZoneOffset;
import java.time.format.DateTimeFormatter;

/**相较于标准算法,加入了时钟回拨解决方法,仅单机研究,仅个人思考,仅供参考
 */
public class SnowFlow {
    //因为二进制里第一个 bit 为如果是 1，那么都是负数，但是我们生成的 id 都是正数，所以第一个 bit 统一都是 0。

    //机器ID  2进制5位  32位减掉1位 31个
    private long workerId;
    //机房ID 2进制5位  32位减掉1位 31个
    private long datacenterId;
    //代表一毫秒内生成的多个id的最新序号  12位 4096 -1 = 4095 个
    private long sequence;
    //设置一个时间初始值    2^41 - 1   差不多可以用69年
    private long twepoch = 1420041600000L;
    //5位的机器id
    private long workerIdBits = 5L;
    //5位的机房id；。‘
    private long datacenterIdBits = 5L;
    //每毫秒内产生的id数 2 的 12次方
    private long sequenceBits = 12L;
    // 这个是二进制运算，就是5 bit最多只能有31个数字，也就是说机器id最多只能是32以内
    private long maxWorkerId = -1L ^ (-1L << workerIdBits);
    // 这个是一个意思，就是5 bit最多只能有31个数字，机房id最多只能是32以内
    private long maxDatacenterId = -1L ^ (-1L << datacenterIdBits);

    private long workerIdShift = sequenceBits;
    private long datacenterIdShift = sequenceBits + workerIdBits;
    private long timestampLeftShift = sequenceBits + workerIdBits + datacenterIdBits;

    // -1L 二进制就是1111 1111  为什么？
    // -1 左移12位就是 1111  1111 0000 0000 0000 0000
    // 异或  相同为0 ，不同为1
    // 1111  1111  0000  0000  0000  0000
    // ^
    // 1111  1111  1111  1111  1111  1111
    // 0000 0000 1111 1111 1111 1111 换算成10进制就是4095
    private long sequenceMask = -1L ^ (-1L << sequenceBits);
    //记录产生时间毫秒数，判断是否是同1毫秒
    private long lastTimestamp = -1L;
    public long getWorkerId(){
        return workerId;
    }
    public long getDatacenterId() {
        return datacenterId;
    }
    public long getTimestamp() {
        return System.currentTimeMillis();
    }

    //是否发生了时钟回拨
    private boolean isBackwordsFlag = false;
    //是否是第一次发生时钟回拨, 用于设置时钟回拨时间点
    private boolean isFirstBackwordsFlag = true;
    //记录时钟回拨发生时间点, 用于判断回拨后的时间达到回拨时间点时, 跳过 已经用过的 时钟回拨发生时间点 之后的时间 到 未来时间的当前时间点
    private long backBaseTimestamp = -1L;

    public SnowFlow() {
    }

    public SnowFlow(long workerId, long datacenterId, long sequence) {

        // 检查机房id和机器id是否超过31 不能小于0
        if (workerId > maxWorkerId || workerId < 0) {
            throw new IllegalArgumentException(
                    String.format("worker Id can't be greater than %d or less than 0",maxWorkerId));
        }

        if (datacenterId > maxDatacenterId || datacenterId < 0) {

            throw new IllegalArgumentException(
                    String.format("datacenter Id can't be greater than %d or less than 0",maxDatacenterId));
        }
        this.workerId = workerId;
        this.datacenterId = datacenterId;
        this.sequence = sequence;
    }

    // 这个是核心方法，通过调用nextId()方法，
    // 让当前这台机器上的snowflake算法程序生成一个全局唯一的id
    public synchronized long nextId() {
        // 这儿就是获取当前时间戳，单位是毫秒
        long timestamp = timeGen();

        //--20220813--1---------------------------------------
        if (isBackwordsFlag) {
            //当回拨时间再次叨叨回拨时间点时, 跳过回拨这段时间里已经使用了的未来时间
            if (timestamp >= backBaseTimestamp && timestamp < lastTimestamp) {
                //直接将当前时间设置为最新的未来时间
                timestamp = lastTimestamp;
            } else if(timestamp > lastTimestamp) {
                //当前时间已经大于上次时间, 重置时钟回拨标志
                isBackwordsFlag = false;
                isFirstBackwordsFlag = true;
                System.out.println("时间已恢复正常-->" + timestamp);
            } else {
                // timestamp == lastTimestamp 等于的情况在后面
            }
        }
        //--20220813--1----------------------------------------

        // 判断是否小于上次时间戳，如果小于的话，就抛出异常
        if (timestamp < lastTimestamp) {

            System.err.printf("lastTimestamp=%d, timestamp=%d, l-t=%d \n", lastTimestamp, timestamp, lastTimestamp - timestamp);
//            throw new RuntimeException(
//                    String.format("Clock moved backwards. Refusing to generate id for %d milliseconds",
//                            lastTimestamp - timestamp));

            //--20220813--2---------------------------------------
            //这里不再抛出异常, 改为记录时钟回拨发生时间点

            //发生时钟回拨后, 当前时间 timestamp 就变成了 过去的时间
            //此时将 timestamp 设置为 上一次时间, 即相对于当前时间的未来时间
            timestamp = lastTimestamp;
            isBackwordsFlag = true;

            //记录时钟回拨发生的时间点, 后续需要跳过已经使用的未来时间段
            if (isFirstBackwordsFlag) {
                backBaseTimestamp = timestamp;
                isFirstBackwordsFlag = false;
                System.out.println("时钟回拨已发生-->" + backBaseTimestamp);
            }
            //--20220813--2---------------------------------------
        }

        // 下面是说假设在同一个毫秒内，又发送了一个请求生成一个id
        // 这个时候就得把seqence序号给递增1，最多就是4096
        if (timestamp == lastTimestamp) {

            // 这个意思是说一个毫秒内最多只能有4096个数字，无论你传递多少进来，
            //这个位运算保证始终就是在4096这个范围内，避免你自己传递个sequence超过了4096这个范围
            sequence = (sequence + 1) & sequenceMask;
            //当某一毫秒的时间，产生的id数 超过4095，系统会进入等待，直到下一毫秒，系统继续产生ID
            if (sequence == 0) {
                //timestamp = tilNextMillis(lastTimestamp);

                //--20220813--3---------------------------------------
                //这里也不能阻塞了, 因为阻塞方法中需要用到当前时间, 改为将此时代表未来的时间 加 1
                if (isBackwordsFlag) {
                    lastTimestamp++;

                    //根据博友评论反馈, 这里可能需要重新赋值, 如果有人看到这个, 可以验证
                    //timestamp = lastTimestamp++;

                } else {
                    timestamp = tilNextMillis(lastTimestamp);
                }
                //--20220813--3---------------------------------------
            }

        } else {
            //sequence = 0;
            //每毫秒的序列号都从0开始的话，会导致没有竞争情况返回的都是偶数。解决方法是用时间戳&1，这样就会随机得到1或者0。
            sequence = timestamp & 1;
        }
        // 这儿记录一下最近一次生成id的时间戳，单位是毫秒
        //lastTimestamp = timestamp;

        //--20220813--4---------------------------------------
        if(isBackwordsFlag) {
            //什么都不做
        } else {
            lastTimestamp = timestamp;
        }
        //--20220813--4---------------------------------------

        // 这儿就是最核心的二进制位运算操作，生成一个64bit的id
        // 先将当前时间戳左移，放到41 bit那儿；将机房id左移放到5 bit那儿；将机器id左移放到5 bit那儿；将序号放最后12 bit
        // 最后拼接起来成一个64 bit的二进制数字，转换成10进制就是个long型
        long sn = ((timestamp - twepoch) << timestampLeftShift) |
                (datacenterId << datacenterIdShift) |
                (workerId << workerIdShift) | sequence;

        if (isBackwordsFlag) {
            System.out.printf("sn=%d\n", sn);
        }
        return sn;
    }

    /**
     * 当某一毫秒的时间，产生的id数 超过4095，系统会进入等待，直到下一毫秒，系统继续产生ID
     * @param lastTimestamp
     * @return
     */
    private long tilNextMillis(long lastTimestamp) {

        long timestamp = timeGen();

        while (timestamp <= lastTimestamp) {
            timestamp = timeGen();
        }
        return timestamp;
    }
    //获取当前时间戳
    private long timeGen(){
        return System.currentTimeMillis();
    }

    /**
     *  main 测试类
     * @param args
     */
    public static void main(String[] args) throws IOException, InterruptedException {

        SnowFlow snowFlow = new SnowFlow(1, 1, 1);
        int count = 10000000;
        //int count = 100;
        for (int i = 0; i < count; i++) {
            //实际测试发现遍历太快输出日志过多导致卡顿, 增加睡眠时间, 或输出到文件
            snowFlow.nextId();

            //Thread.sleep(100);

//            System.out.println(snowFlow.nextId());

//            if (i == 1000) {
            //不具有管理员权限的用户, 修改不成功
            //testClockMvoedBackwords(30);
//            }
            //改为 手动修改,  右键cmd,以管理员权限打开后,使用time命令即可, time 16:15:00
        }


        System.out.println(System.currentTimeMillis());
        /**
         * 这里为什么特意输出一个开始时间呢, 其实就是一个运行了两年的程序突然有一天出bug了,导致了严重的生产事件!
         * 那么时间初始化影响什么呢, 答案是 序列的长度
         * 有人就说了, 这个一般是作为 主键用的, 长度貌似影响不大, 确实是这样的
         * 这次的bug不是雪花算法本身的问题, 而是程序里面有个功能是严格长度截取的, 并且只考虑了长度不够的情况, 没有考虑到变长的情况
         * 最根本的原因是 本人截取的时候 序列的长度一直是18位, 然后截取9位的代码是这么写的 substring(9);
         * 当未来的某一天序列长度增加到了19位,那么这个截取就会返回10位长度, 最终导致一个大范围的交易失败......
         * 锅当然是本人背, 这里提出这种情况, 供大家参考.
         * 经过仔细研究所谓的序列可以使用69年, 序列的长度变化是这样的, 假设以当前时间为初始化值
         * 12 13 14 15 16 17 18(约7年) 19(约58年)
         * 长度随时间增加, 长度越长, 保持相同长度的时间越长
         */
        DateTimeFormatter dtf2 = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
        String dateString = "2015-01-01 00:00:00";
        LocalDateTime localDateTime = LocalDateTime.parse(dateString,dtf2);
        System.out.println(localDateTime.toInstant(ZoneOffset.ofHours(8)).toEpochMilli());

    }

    //windows os 模拟时钟回拨, 将当前时间减去几秒
    private static void testClockMvoedBackwords(long seconds) throws IOException {
        System.out.println(LocalDateTime.now().format(DateTimeFormatter.ofPattern("HH:mm:ss")));

        LocalDateTime localDateTime = LocalDateTime.now();
        String backTime = localDateTime.minusSeconds(seconds).format(DateTimeFormatter.ofPattern("HH:mm:ss"));
        System.out.println(backTime);

        if (System.getProperty("os.name").contains("Windows")) {
            String cmd = "cmd /c start time 15:41:56";// + backTime;
            //不具有管理员权限的用户, 修改不生效, 提示 客户端无所需特权
            Runtime.getRuntime().exec(cmd);
//            Runtime.getRuntime().exec("cmd /c notepad");
            System.out.println(LocalDateTime.now().format(DateTimeFormatter.ofPattern("HH:mm:ss")));
        }
    }
}

3. The solution to the clock callback problem of Snowflake production plan

The first method is to turn off clock synchronization to avoid clock synchronization problems, but this is not realistic, because systems that are strongly dependent on time generally have to do clock synchronization to avoid serious time errors, deploy some things on virtual machines, play After the virtual machine is hibernated and resumed again, the time in the virtual machine and the time of the host are often out of sync, causing some distributed systems with large data to crash, and the communication between nodes will rely on time stamps for comparison and heartbeat Slow, it will cause the node to hang

The second method is to record the time when the ID was last generated. If it is found that the time stamp is smaller than the last time stamp when the ID is generated this time, it means that the clock has been dialed back. At this time, ID generation is not allowed within this time. Wait, wait until the current time catches up with the last generation time. The problem is, what if the callback time is too much? It may take a long time, which affects the availability of the system, so it is not a particularly good way to store the timestamp of the last generated unique ID in the memory. When the clock is dialed back, the current timestamp will be dialed back to before the last timestamp. Request comes, to generate a unique ID, you do not directly return an ID to him, you do a comparison first, if you find that the current timestamp is compared with the timestamp of the last generated unique ID, it is smaller than him, and the clock is called back , as long as you generate an ID, it is possible that if the repeated availability of the ID is so poor, if someone’s business service wants to generate a billing data at this time and apply for an ID, at this time you finally waited for hundreds of milliseconds, you Also tell him that you have an internal exception and cannot obtain the unique ID. Repeated retries will affect the operation of his business services.

The third method is for the optimization of the second method. If you find that the clock callback is too harsh, such as exceeding 1 minute, you will call the police directly at this time, and at the same time no longer provide external services, and remove yourself from the cluster, such as If you register based on the microservice registration center, you have to take the initiative to go offline. When you find that the current timestamp is smaller than the timestamp of the last generated ID, and you find that the clock has been dialed back, judge how many milliseconds have been dialed back. For example, if the callback time is within 500ms, you can hang the request at this time and wait for 500ms. After 500ms, the current timestamp is greater than the timestamp of the last generated ID. At this time, you can normally
generate a unique ID and return it to the business side. , for the business side, only in the case of a few clock callbacks, the request usually only takes 50ms, 500ms, which is still within the acceptable range, so it is still possible, but the request is slower. If
you It is found that your current timestamp is compared with the timestamp of the last generated unique ID. When you compare it, you will find that it exceeds 500ms, but within 5s, you can return an abnormal state + abnormal continuation Give the client time, don't say there is a problem, you can notify him to retry the retry
mechanism by himself, it is best not to let the business side do it yourself, you can completely encapsulate a client of your unique ID generation service, based on RPC request Your interface, but you encapsulate an automatic retry mechanism in your own client. Once he finds that a server returns a response saying that he cannot provide the service in a short time, he will automatically request the service on other machines to obtain Unique ID
If you want to solve the clock callback, the second and third methods are generally used together, but passive waiting or even active offline will always affect the availability of the system, and it is not particularly good for clock callback detection on the server
side Mechanism + The client encapsulates itself
within 1s: the blocking request waits, and the timeout period of the client should also be 1s, which exposes the largest serial number of the unique ID generated every millisecond within 1s, and locates the previously generated ID according to the milliseconds of the current timestamp The maximum ID sequence number of this millisecond, continue to generate ID at this time, and directly increment on the basis of the maximum ID sequence number of this millisecond generated before. After optimization, it can be guaranteed that there is no need to block and wait
Between 1s and 10s: return the exception code and the duration of the exception, the client does not request this machine within the specified time for
more than 10s: return the fault code, request the service registration center to let itself go offline, after the client receives the fault code, it will Just remove this machine from the list of service machines, and don’t request him anymore. Afterwards, when the ID service deployed on that machine, he finds that his time may have passed for a few seconds. After slowing down, recovering, and being available, it can be done. Register the service again. When your client refreshes the service registration list, you will find him. At this time, you can request him again.

The fourth method is to maintain the ID value generated in the last few seconds in the memory. Generally, the clock callback is tens of milliseconds to hundreds of milliseconds, and rarely exceeds seconds, so it is enough to save the last few seconds, and then If the clock callback occurs, check which millisecond the callback is at this time, because the timestamp is at the millisecond level, and then just look at that millisecond and
continue to generate the ID serial number that was produced in that millisecond. Every subsequent millisecond is followed by analogy, so that the problem of repetition can be perfectly avoided, and there is no need to wait, but there
is also a bottom-up mechanism here, that is, if you keep the ID generated every millisecond in the last 10s, then in case the clock callback happens to What about more than 10s? At this time, the probability is very low. You can combine the two or three schemes and set several thresholds. For example, if you keep the ID of the last 10s, you can ensure that there will be no repetitions and no pauses within 10s of the callback; if it exceeds 10s, Within 60s, there can be a waiting process to let him advance to the range of 10s you reserved before; if the call back exceeds 60s, the timestamp of the last unique ID generated directly
offline The maximum ID serial number per millisecond is gone. After restarting, the time callback occurs, and the time callback problem cannot be found. Secondly, there is no way to continue to generate unique IDs that are not repeated according to the previous thinking.

4. Clock callback optimization

1. We generally need to turn on the clock synchronization function, so that the ID can be maximized to ensure that it is ordered according to time, but after the clock synchronization is turned on, the clock may be dialed back. If the clock is dialed back, the generated ID will be repeated. , so we generally turn on the clock synchronization and turn off the clock callback function;
2. The number of digits in the serial number is limited, and the number of IDs that can be represented is limited. When the clock is synchronized, if a server is much faster, even if the clock is turned off Call back, but before the time catches up, the ID may have been used up. When the self-incrementing serial number is used up, we can do the following work: stop the ID generation service and give an alarm, wait if the clock callback is less than a certain threshold, such as If it is greater than a certain threshold, a third-party component such as ZK will be used to regenerate a workerid or borrow the ID of the next timestamp from the self-incrementing timestamp; 3. After the
service is restarted, the ID may be repeated. For this reason, we generally need to save the timestamp regularly and restart it. The final timestamp must be greater than the saved timestamp + several times the saving interval (such as 3 times), why several times, mainly considering the data loss, but if it is saved to the local hard disk and fsync every time, At this time, 1 times is enough. After restarting, if it is smaller than the second point, it can be handled similarly to the second point;
4. If the QPS of the requested ID is not high, such as one per millisecond, then the tail number of the ID obtained each time is 0, and then the database and table are divided based on the ID. Maybe the data distribution will be uneven. At this time, we can increase the time interval of the timestamp or the serial number will increase automatically from a random value each time.

It is mainly the method of generating id. I will optimize this again. This is further optimized according to Meituan leaf. Please refer to

    /**
     * 通过雪花算法生成下一个id，注意这里使用synchronized同步
     * @return 唯一id
     */
    public synchronized long nextId() {
        long currentTimeMillis = System.currentTimeMillis();
        //System.out.println(currentTimeMillis);
        // 当前时间小于上一次生成id使用的时间，可能出现服务器时钟回拨问题
        //long timestamp = timeGen();
        if (currentTimeMillis < lastTimeMillis) {
            long offset = lastTimeMillis - currentTimeMillis;
            if (offset <= 5) {
                try {
                    wait(offset << 1);
                    currentTimeMillis = timeGen();
                    if (currentTimeMillis < lastTimeMillis) {
                        throw new RuntimeException("id生成失败");
                    }
                } catch (InterruptedException e) {
                    throw new RuntimeException("生成id时出现错误");
                }
            } else {
                throw new RuntimeException(
                        String.format("可能出现服务器时钟回拨问题，请检查服务器时间。当前服务器时间戳：%d，上一次使用时间戳：%d", currentTimeMillis,
                                lastTimeMillis));
            }
        }


        /*if (currentTimeMillis < lastTimeMillis) {
            throw new RuntimeException(
                    String.format("可能出现服务器时钟回拨问题，请检查服务器时间。当前服务器时间戳：%d，上一次使用时间戳：%d", currentTimeMillis,
                            lastTimeMillis));
        }*/
        if (currentTimeMillis == lastTimeMillis) {
            // 还是在同一毫秒内，则将序列号递增1，序列号最大值为4095
            // 序列号的最大值是4095，使用掩码（最低12位为1，高位都为0）进行位与运行后如果值为0，则自增后的序列号超过了4095
            // 那么就使用新的时间戳
            sequence = (sequence + 1) & SEQUENCE_MASK;
            if (sequence == 0) {
                currentTimeMillis = getNextMillis(lastTimeMillis);
            }
        } else { // 不在同一毫秒内，则序列号重新从0开始，序列号最大值为4095
            sequence = 0;
        }
        // 记录最后一次使用的毫秒时间戳
        lastTimeMillis = currentTimeMillis;
        // 核心算法，将不同部分的数值移动到指定的位置，然后进行或运行
        // <<：左移运算符, 1 << 2 即将二进制的 1 扩大 2^2 倍
        // |：位或运算符, 是把某两个数中, 只要其中一个的某一位为1, 则结果的该位就为1
        // 优先级：<< > |
        return
                // 时间戳部分
                ((currentTimeMillis - INIT_EPOCH) << TIMESTAMP_SHIFT)
                        // 数据中心部分
                        | (dataCenterId << DATA_CENTER_ID_SHIFT)
                        // 机器表示部分
                        | (workerId << WORK_ID_SHIFT)
                        // 序列号部分
                        | sequence;
    }