Overview
In distributed systems, there are some scenarios in which globally unique IDs are required. In this case, 36-bit UUIDs can be used to prevent ID conflicts. However, UUIDs have some disadvantages. First, they are relatively long, and UUIDs are generally disordered.
Sometimes we want to use a simpler ID, and hope that the ID can be generated in time order.
Twitter's snowflake solves this need. Initially, Twitter migrated the storage system from MySQL to Cassandra. Because Cassandra does not have a sequential ID generation mechanism, it developed such a globally unique ID generation service.
structure
The structure of snowflake is as follows (each part is separated by -):
0 - 0000000000 0000000000 0000000000 0000000000 0 - 00000 - 00000 - 000000000000
The first bit is unused, the next 41 bits are millisecond time (the length of 41 bits can be used for 69 years), then 5 bits of datacenterId and 5 bits of workerId (the length of 10 bits supports deployment of up to 1024 nodes), and finally 12 bits is the count in milliseconds (12-bit count sequence number supports each node to generate 4096 ID sequence numbers per millisecond)
It adds up to exactly 64 bits, which is a Long type. (Length up to 19 after converting to string)
The IDs generated by snowflake are sorted by time as a whole, and there is no ID collision (distinguished by datacenter and workerId) in the entire distributed system, and the efficiency is high. After testing, snowflake can generate 260,000 IDs per second.
/**
* Twitter_Snowflake<br>
* The structure of SnowFlake is as follows (each part is separated by -):<br>
* 0 - 0000000000 0000000000 0000000000 0000000000 0 - 00000 - 00000 - 000000000000 <br>
* 1-bit identification, since the long basic type is signed in Java, the highest bit is the sign bit, positive numbers are 0, and negative numbers are 1, so id is generally positive, and the highest bit is 0<br>
* 41-bit time cutoff (millisecond level), note that the 41-bit time cutoff is not the time cutoff that stores the current time, but the difference between the time cutoffs (current time cutoff - start time cutoff)
* The value obtained), the start time here is generally the time when our id generator starts to be used, which is specified by our program (the startTime property of the IdWorker class in the following program). 41-bit time cutoff, can use 69 years, year T = (1L << 41) / (1000L * 60 * 60 * 24 * 365) = 69<br>
* 10 data machine bits, which can be deployed on 1024 nodes, including 5 datacenterId and 5 workerId<br>
* 12-bit sequence, counting in milliseconds, 12-bit counting sequence number supports each node to generate 4096 ID numbers every millisecond (same machine, same time interval)<br>
* Add up to exactly 64 bits, which is a Long type. <br>
* The advantage of SnowFlake is that it is sorted by time as a whole, and there is no ID collision in the entire distributed system (distinguished by the data center ID and machine ID), and the efficiency is high. After testing, SnowFlake can generate About 260,000 IDs.
*/
public class SnowflakeIdWorker {
// ==============================Fields===========================================
/** Start time cutoff (2015-01-01) */
private final long twepoch = 1420041600000L;
/** The number of digits occupied by the machine id */
private final long workerIdBits = 5L;
/** The number of bits occupied by the data ID id */
private final long datacenterIdBits = 5L;
/** The maximum supported machine id, the result is 31 (this shift algorithm can quickly calculate the maximum decimal number that a few binary numbers can represent) */
private final long maxWorkerId = -1L ^ (-1L << workerIdBits);
/** The maximum supported data identifier id, the result is 31 */
private final long maxDatacenterId = -1L ^ (-1L << datacenterIdBits);
/** The number of digits the sequence occupies in the id */
private final long sequenceBits = 12L;
/** Shift the machine ID 12 bits to the left */
private final long workerIdShift = sequenceBits;
/** The data identification id is shifted to the left by 17 bits (12+5) */
private final long datacenterIdShift = sequenceBits + workerIdBits;
/** Shift the time cut to the left by 22 bits (5+5+12) */
private final long timestampLeftShift = sequenceBits + workerIdBits + datacenterIdBits;
/** Generate the mask of the sequence, here is 4095 (0b111111111111=0xfff=4095) */
private final long sequenceMask = -1L ^ (-1L << sequenceBits);
/** Working machine ID (0~31) */
private long workerId;
/** Data center ID (0~31) */
private long datacenterId;
/** Sequence in milliseconds (0~4095) */
private long sequence = 0L;
/** The last time the ID was generated */
private long lastTimestamp = -1L;
//==============================Constructors=====================================
/**
* Constructor
* @param workerId job ID (0~31)
* @param datacenterId data center ID (0~31)
*/
public SnowflakeIdWorker(long workerId, long datacenterId) {
if (workerId > maxWorkerId || workerId < 0) {
throw new IllegalArgumentException(String.format("worker Id can't be greater than %d or less than 0", maxWorkerId));
}
if (datacenterId > maxDatacenterId || datacenterId < 0) {
throw new IllegalArgumentException(String.format("datacenter Id can't be greater than %d or less than 0", maxDatacenterId));
}
this.workerId = workerId;
this.datacenterId = datacenterId;
}
// ==============================Methods==========================================
/**
* Get the next ID (this method is thread safe)
* @return SnowflakeId
*/
public synchronized long nextId() {
long timestamp = timeGen();
//If the current time is less than the timestamp generated by the last ID, it means that an exception should be thrown when the system clock rolls back
if (timestamp < lastTimestamp) {
throw new RuntimeException(
String.format("Clock moved backwards. Refusing to generate id for %d milliseconds", lastTimestamp - timestamp));
}
//If it is generated at the same time, perform the sequence within milliseconds
if (lastTimestamp == timestamp) {
sequence = (sequence + 1) & sequenceMask;
//Sequence overflow in milliseconds
if (sequence == 0) {
//Block until the next millisecond to get a new timestamp
timestamp = tilNextMillis(lastTimestamp);
}
}
//Timestamp changed, sequence reset in milliseconds
else {
sequence = 0L;
}
//The last time the ID was generated
lastTimestamp = timestamp;
//Shift and join together by OR operation to form a 64-bit ID
return ((timestamp - twepoch) << timestampLeftShift) //
| (datacenterId << datacenterIdShift) //
| (workerId << workerIdShift) //
| sequence;
}
/**
* Block until the next millisecond until a new timestamp is obtained
* @param lastTimestamp The last time the ID was generated
* @return current timestamp
*/
protected long tilNextMillis(long lastTimestamp) {
long timestamp = timeGen();
while (timestamp <= lastTimestamp) {
timestamp = timeGen();
}
return timestamp;
}
/**
* Return the current time in milliseconds
* @return current time (milliseconds)
*/
protected long timeGen() {
return System.currentTimeMillis();
}
//==============================Test=============================================
/** test*/
public static void main(String[] args) {
SnowflakeIdWorker idWorker = new SnowflakeIdWorker(0, 0);
for (int i = 0; i < 1000; i++) {
long id = idWorker.nextId();
System.out.println(Long.toBinaryString(id));
System.out.println(id);
}
}
}