Table of contents
1) Why should the ID be synchronized
2) What are the synchronization ID generation methods?
3) ID requirements and code implementation in the game
1. Why do you need to synchronize the ID
In the game server, the id is often our unique identifier. It may be the key we store in the map container, or it may be the unique index we store in the database, or even worse, it is also unique in the database That is, we hope that the data falling into the database will always be the only record, even if it is a different database. We know that game servers are often divided into many groups, and the data between different groups is often separated. Sometimes we need to update the database Merging (or merging servers), then the uniqueness of d is very important when merging, and this is a very error-prone project. The IDs of account numbers, props, tasks, etc., the IDs of these key data may involve N data tables, and it is very cumbersome to modify the database data one by one. Sometimes we want players to obtain items that can be traded across the server. Without a synchronized id across the server, this kind of transaction is not so easy to achieve.
2. What are the methods of synchronous id generation:
1. Database feature implementation ID
2. ID generation center
3, Java only uuid
4. Snowflake algorithm and variants
1) Implementation of database features
Realized through the characteristics of the database, the database can set the field to self-increment, and use the database characteristics. This implementation method requires the cooperation of the database and the return of the successful self-increment. Of course, it is also possible to implement it through the distributed lock of redis. Regardless of the above two situations, their average speed depends on the concurrency capability of the database itself
2) ID generation center
The ID generation center determines the uniqueness through a separate process, and the ID is generated by a separate process. This generation method has relatively high performance, and the efficiency of assigning IDs depends on the network delay. Of course, batch allocation can be solved to reduce the interaction with the ID center. At the beginning, an available ID range is allocated for a process that needs an ID, and when the process that needs an ID is about to run out of IDs, it is allocated once, and so on. The ID center will drop the ID data for each allocation into the database or a persistent cache, so that even if the process is down, there will be no ID conflicts
3) java uuid generation
The Java native API provides a UUID generation method, a globally unique identifier, and the UUID generation is 16 bytes. The Java UUID code is as follows:
package java.util;
import java.io.Serializable;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import jdk.internal.misc.JavaLangAccess;
import jdk.internal.misc.SharedSecrets;
public final class UUID implements Serializable, Comparable<UUID> {
private static final long serialVersionUID = -4856846361193249489L;
private final long mostSigBits;
private final long leastSigBits;
private static final JavaLangAccess jla = SharedSecrets.getJavaLangAccess();
private UUID(byte[] data) {
long msb = 0L;
long lsb = 0L;
assert data.length == 16 : "data must be 16 bytes in length";
int i;
for(i = 0; i < 8; ++i) {
msb = msb << 8 | (long)(data[i] & 255);
}
for(i = 8; i < 16; ++i) {
lsb = lsb << 8 | (long)(data[i] & 255);
}
this.mostSigBits = msb;
this.leastSigBits = lsb;
}
public UUID(long mostSigBits, long leastSigBits) {
this.mostSigBits = mostSigBits;
this.leastSigBits = leastSigBits;
}
public static UUID randomUUID() {
SecureRandom ng = UUID.Holder.numberGenerator;
byte[] randomBytes = new byte[16];
ng.nextBytes(randomBytes);
randomBytes[6] = (byte)(randomBytes[6] & 15);
randomBytes[6] = (byte)(randomBytes[6] | 64);
randomBytes[8] = (byte)(randomBytes[8] & 63);
randomBytes[8] = (byte)(randomBytes[8] | 128);
return new UUID(randomBytes);
}
public static UUID nameUUIDFromBytes(byte[] name) {
MessageDigest md;
try {
md = MessageDigest.getInstance("MD5");
} catch (NoSuchAlgorithmException var3) {
throw new InternalError("MD5 not supported", var3);
}
byte[] md5Bytes = md.digest(name);
md5Bytes[6] = (byte)(md5Bytes[6] & 15);
md5Bytes[6] = (byte)(md5Bytes[6] | 48);
md5Bytes[8] = (byte)(md5Bytes[8] & 63);
md5Bytes[8] = (byte)(md5Bytes[8] | 128);
return new UUID(md5Bytes);
}
public static UUID fromString(String name) {
int len = name.length();
if (len > 36) {
throw new IllegalArgumentException("UUID string too large");
} else {
int dash1 = name.indexOf(45, 0);
int dash2 = name.indexOf(45, dash1 + 1);
int dash3 = name.indexOf(45, dash2 + 1);
int dash4 = name.indexOf(45, dash3 + 1);
int dash5 = name.indexOf(45, dash4 + 1);
if (dash4 >= 0 && dash5 < 0) {
long mostSigBits = Long.parseLong(name, 0, dash1, 16) & 4294967295L;
mostSigBits <<= 16;
mostSigBits |= Long.parseLong(name, dash1 + 1, dash2, 16) & 65535L;
mostSigBits <<= 16;
mostSigBits |= Long.parseLong(name, dash2 + 1, dash3, 16) & 65535L;
long leastSigBits = Long.parseLong(name, dash3 + 1, dash4, 16) & 65535L;
leastSigBits <<= 48;
leastSigBits |= Long.parseLong(name, dash4 + 1, len, 16) & 281474976710655L;
return new UUID(mostSigBits, leastSigBits);
} else {
throw new IllegalArgumentException("Invalid UUID string: " + name);
}
}
}
public long getLeastSignificantBits() {
return this.leastSigBits;
}
public long getMostSignificantBits() {
return this.mostSigBits;
}
public int version() {
return (int)(this.mostSigBits >> 12 & 15L);
}
public int variant() {
return (int)(this.leastSigBits >>> (int)(64L - (this.leastSigBits >>> 62)) & this.leastSigBits >> 63);
}
public long timestamp() {
if (this.version() != 1) {
throw new UnsupportedOperationException("Not a time-based UUID");
} else {
return (this.mostSigBits & 4095L) << 48 | (this.mostSigBits >> 16 & 65535L) << 32 | this.mostSigBits >>> 32;
}
}
public int clockSequence() {
if (this.version() != 1) {
throw new UnsupportedOperationException("Not a time-based UUID");
} else {
return (int)((this.leastSigBits & 4611404543450677248L) >>> 48);
}
}
public long node() {
if (this.version() != 1) {
throw new UnsupportedOperationException("Not a time-based UUID");
} else {
return this.leastSigBits & 281474976710655L;
}
}
public String toString() {
return jla.fastUUID(this.leastSigBits, this.mostSigBits);
}
public int hashCode() {
long hilo = this.mostSigBits ^ this.leastSigBits;
return (int)(hilo >> 32) ^ (int)hilo;
}
public boolean equals(Object obj) {
if (null != obj && obj.getClass() == UUID.class) {
UUID id = (UUID)obj;
return this.mostSigBits == id.mostSigBits && this.leastSigBits == id.leastSigBits;
} else {
return false;
}
}
public int compareTo(UUID val) {
return this.mostSigBits < val.mostSigBits ? -1 : (this.mostSigBits > val.mostSigBits ? 1 : (this.leastSigBits < val.leastSigBits ? -1 : (this.leastSigBits > val.leastSigBits ? 1 : 0)));
}
private static class Holder {
static final SecureRandom numberGenerator = new SecureRandom();
private Holder() {
}
}
}
You can see from the code that UUID.randomUUID() has obtained a UUID object, which has two long type member variables
private final long mostSigBits;
private final long leastSigBits;We can obtain its int type value as a unique identifier through hashcode, but we found that there is a small probability of collision,
public String toString() {
return jla.fastUUID(this.leastSigBits, this.mostSigBits);
}
Any collision is not allowed; or by using UUID.randomUUID().toString() as a unique character, this can ensure uniqueness, but the relatively long string identifier generated is not suitable for the actual development of game server business. It is not friendly in use.
Through the above code, we can see that the UUID obtains two long types, or generates a string. The maximum type of the basic type that the computer can express is 8 bytes (long type), and the two longs do not have a matching basic data type. , the string is unique, but it is very inconvenient to use. So is there a unique id for the entire server that can be expressed with a long type? The answer must be yes, the snowflake algorithm was born
snowflake algorithm
The snowflake algorithm is an algorithm invented by Twitter, whose main purpose is to solve the problem of how to generate IDs in a distributed environment
1. Rigid requirements for distributed ID generation rules: globally unique: no duplicate ID numbers can appear, since it is a unique identifier, this is the most basic requirement
In the snowflake algorithm, a 41-bit timestamp and 10-bit process ID are used to generate a serial number of 12 bits, and a high bit of 1 bit is combined to form a 64long type. From the above figure, we can see that a process with 4.096 million ids can be generated in one second. In actual use It meets our business needs far and wide. However, careful friends will find that the process ID is very important. If the process ID is the same as other processes, there may be a small probability of collision, so in actual use, the process ID must ensure its uniqueness.
Based on this snowflake algorithm, it provides me with a good idea to obtain the general idea of using a Long type to obtain a globally unique id, that is, to store information such as timestamp and process id into a Long. The advantage of the snowflake algorithm is that it does not use third-party tools to generate IDs locally, ensuring that the generated IDs are completely synchronized globally. At the same time, a long integer can be used to express the ID we need, which is convenient for us to operate on the data. This is in line with needs in game development.
3. In game development, we have the following requirements
1) Hope to have a unique id in thread safe
Because we have a large number of containers that need a large number of containers to meet our business needs, the uniqueness of the key value is critical
It contains:
(38-bit time+25-bit seq+1 highest mark bit) = 64-bit, thread synchronization, but not process synchronization
2) It is hoped that the id is synchronized between processes (like snowflake algorithm)
Item ID generation, character ID, guild ID, etc. all need to have a unique identifier for easy storage and query.
It contains:
(Inter-process unique ID 14-bit processId+bit+29-bit time+20-bit seq+1-bit highest mark)=64
3) account id
The game may have multiple server groups. It is hoped that each id also contains group (groupId) information, and it is unique globally. It is required in special application scenarios, such as when logging in, when the SDK login openId is uniquely bound to the account. , its generation may not be as efficient as the process synchronization ID, but it is also synchronized between processes.
It contains:
( Inter-process unique ID 14-bit processId+bit+29-bit time+12-bit groupId+8-bit seq+1-bit highest mark)=64
These three IDs all have a common feature. The timestamp, serial number and time spin are used to generate our ID. The thread ID is the most efficient, followed by the process ID, and the account ID is the lowest.
The two needs of 1 and 2 are easy to meet, and the need of 3 needs special treatment
The implementation code is as follows:
create id interface
public interface IIdGenerator {
long nextID();
}Create object class base class
import java.util.Calendar;
public abstract class AbstractIdGenerator implements IIdGenerator{
//进程ID 最大1024*16-1
protected static final int PROCESS_BITS = 14;
private volatile long seq = 0L;
protected volatile long lastTime = 0L;
public AbstractIdGenerator()
{
this.lastTime = getNowWorldTimeSec();
}
/**
* 通过自旋获得下一个序号，平滑处理序号分配，如果上次时间打段还没有分配玩，时间戳就不改变
* 这样就可以减少ID浪费。因为并不是每秒钟ID都能耗尽，在ID峰值的时候降低概率因为时间自旋而引起
* 阻塞
* @param currentTime 当前时间（单位秒）
* @return
*/
protected long nextSeq(long currentTime)
{
if (lastTime>currentTime) {
this.lastTime = this.enterNextSec();
this.seq = 1L;
}
else{
++this.seq;
//如果当前秒ID没有耗尽，那么就不跳秒表
if (this.seq >= getMaxSeq()) {
lastTime++;
if(lastTime>currentTime) {
lastTime = this.enterNextSec();
}
seq = 1;
}
}
return lastTime;
}
private long enterNextSec() {
long sec = getNowWorldTimeSec();
while (sec <= this.lastTime) {
sec = getNowWorldTimeSec();
}
return sec;
}
abstract long getMaxSeq();
protected long getSeq()
{
return seq;
}
private static volatile long wordTimeSec = 0;
public static long getDefaultWordTimeSec() {
if(wordTimeSec ==0)
{
synchronized ( AbstractIdGenerator.class) {
Calendar instance = Calendar.getInstance();
instance.set(Calendar.YEAR, 2022);
instance.set(Calendar.MONTH, 1);
instance.set(Calendar.DAY_OF_MONTH, 1);
instance.set(Calendar.HOUR_OF_DAY, 0);
instance.set(Calendar.MINUTE, 0);
instance.set(Calendar.SECOND, 0);
instance.set(Calendar.MILLISECOND, 0);
wordTimeSec = instance.getTimeInMillis() / 1000L;
}
}
return wordTimeSec;
}
public static long currentTimeMillis()
{
return System.currentTimeMillis();
}
public static long currentTimeSeconds()
{
return System.currentTimeMillis()/1000L;
}
/**
* @return 当前时间秒速-系统时间与2022/1/1-1970/1/1时间
*/
public static long getNowWorldTimeSec()
{
return (currentTimeSeconds()-getDefaultWordTimeSec());
}
}
ObjectIDGenerator (38-bit time+25-bit seq+1 highest mark bit) thread synchronization, but not process synchronization
public class ObjectIDGenerator extends AbstractIdGenerator{
private static int SEQ_BITS = 25;
public ObjectIDGenerator()
{
super();
}
@Override
public synchronized long nextID() {
long nowTime = nextSeq(getNowWorldTimeSec());
return (((nowTime<<SEQ_BITS) | getSeq()) & 0x7FFFFFFFFFFFFFFFL);
}
@Override
protected long getMaxSeq()
{
return (1<<SEQ_BITS)-1;
}
}SyncIdGenerator (inter-process unique ID 14-bit processId+bit+29-bit time+20-bit seq+1-bit highest mark)=64
process security
public class SyncIdGenerator extends AbstractIdGenerator{
private volatile long processID = 0L;
private static int SEQ_BITS = 20;
public SyncIdGenerator(int processID) {
super();
this.processID = processID;
if (this.processID >= ((1<<PROCESS_BITS)-1)) {
int value = (1<<(PROCESS_BITS-1));
throw new IllegalArgumentException("invalid processID must less than = " +value+" processID = "+processID);
}
}
@Override
public synchronized long nextID()
{
long nowTime = nextSeq(getNowWorldTimeSec());
long id = 0L;
id = (this.processID <<(63-PROCESS_BITS));
id |= (nowTime <<SEQ_BITS);
id |= getSeq();
return (id & 0x7FFFFFFFFFFFFFFFL);
}
@Override
protected long getMaxSeq()
{
return ((1<<SEQ_BITS)-1);
}
}UserIdGenerator (inter-process unique ID 14-bit processId+bit+29-bit time+12-bit groupId+8-bit seq+1-bit highest mark)=64
/**
* 账号ID生成器，账号登录往往生成并不需很频繁，但是由于ID中添加groupId所以并发量受到限制
* 这里对ID做了预借100秒100*256个
*/
public class UserIdGenerator extends AbstractIdGenerator{
private static int GROUP_ID_BITS = 12;
private static int TIME_BITS = 29;
private static int SEQ_BITS = 8;
private long processID;
private long groupID;
public UserIdGenerator(int processID,int groupID)
{
super();
if(groupID>getMaxGroupID())
{
int value = getMaxGroupID();
throw new IllegalArgumentException("invalid groupID must less than MaxGroupID(= " +value+" groupID = "+groupID);
}
if(processID>getMaxProcessID())
{
int value = getMaxProcessID();
throw new IllegalArgumentException("invalid processID must less than ProcessID= " +value+" processID = "+processID);
}
//先预借100*256个ID
lastTime = getNowWorldTimeSec()-100;
this.processID = processID;
this.groupID = groupID;
}
@Override
public synchronized long nextID() {
long nowTime = nextSeq(getNowWorldTimeSec());
long id = 0L;
id = (groupID <<(63-GROUP_ID_BITS));
id |= (processID <<(63-PROCESS_BITS-GROUP_ID_BITS));
id |= (nowTime<<SEQ_BITS);
id |= getSeq();
return (id & 0x7FFFFFFFFFFFFFFFL);
}
private int getMaxGroupID()
{
return (1<<GROUP_ID_BITS)-1;
}
private int getMaxProcessID()
{
return (1<<PROCESS_BITS)-1;
}
@Override
protected long getMaxSeq()
{
return (1<<SEQ_BITS)-1;
}
public static int getGroupID(long userID)
{
//将前面数据清空
long a = (0x7FFFFFFFFFFFFFFFL & userID);
a = (a>>(63-GROUP_ID_BITS));
return (int)a;
}
public static int getProcessID(long userID)
{
//将前面数据清空
long a = (0x7FFFFFFFFFFFFFFFL & userID);
a = a<<GROUP_ID_BITS;
a = (a>>(63-PROCESS_BITS));
return (int)a;
}
}
Special processing is done for SyncIdGenerator and UserIdGenerator ID, so that ID generation can be smooth and the probability of ID generation bottlenecks can be reduced as much as possible
/**
* 通过自旋获得下一个序号，平滑处理序号分配，如果上次时间打段还没有分配玩，时间戳就不改变
* 这样就可以减少ID浪费。因为并不是每秒钟ID都能耗尽，在ID峰值的时候降低概率因为时间自旋而引起
* 阻塞
* @param currentTime 当前时间（单位秒）
* @return
*/
protected long nextSeq(long currentTime)
{
if (lastTime>currentTime) {
this.lastTime = this.enterNextSec();
this.seq = 1L;
}
else{
++this.seq;
//如果当前秒ID没有耗尽，那么就不跳秒表
if (this.seq >= getMaxSeq()) {
lastTime++;
if(lastTime>currentTime) {
lastTime = this.enterNextSec();
}
seq = 1;
}
}
return lastTime;
}