Distributed ID Generator - Tinyid

Get into the habit of writing together! This is the 11th day of my participation in the "Nuggets Daily New Plan · April Update Challenge", click to view the details of the event .

1. Tinyid

inyid is a distributed id generation system developed in Java, based on the database number segment algorithm (developed by Didi). This algorithm is similar to Meituan's Leaf distributed Leaf-segment database solution. No implementation of the Snowflake algorithm is provided. Features of Tinyid:

The provided ID is a long type, which is the same as the ID generated by Meituan's Leaf and Snowflake algorithms.
Provides Http access, and also provides Java-client. This client method can be generated locally even when Http is not available. (Availability depends on step size). Java-client greatly improves the performance of generating data.
Not applicable to scenarios with similar orders. This is the same as Meituan's Leaf. Easy to predict order volume.

2.Tinyid Java implementation

2.1 Tinyid Service

clone code:

git clone https://github.com/didi/tinyid.git
复制代码

Execute the script:

CREATE TABLE `tiny_id_info` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT '自增主键',
  `biz_type` varchar(63) NOT NULL DEFAULT '' COMMENT '业务类型，唯一',
  `begin_id` bigint(20) NOT NULL DEFAULT '0' COMMENT '开始id，仅记录初始值，无其他含义。初始化时begin_id和max_id应相同',
  `max_id` bigint(20) NOT NULL DEFAULT '0' COMMENT '当前最大id',
  `step` int(11) DEFAULT '0' COMMENT '步长',
  `delta` int(11) NOT NULL DEFAULT '1' COMMENT '每次id增量',
  `remainder` int(11) NOT NULL DEFAULT '0' COMMENT '余数',
  `create_time` timestamp NOT NULL DEFAULT '2010-01-01 00:00:00' COMMENT '创建时间',
  `update_time` timestamp NOT NULL DEFAULT '2010-01-01 00:00:00' COMMENT '更新时间',
  `version` bigint(20) NOT NULL DEFAULT '0' COMMENT '版本号',
  PRIMARY KEY (`id`),
  UNIQUE KEY `uniq_biz_type` (`biz_type`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4 COMMENT 'id信息表';

CREATE TABLE `tiny_id_token` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
  `token` varchar(255) NOT NULL DEFAULT '' COMMENT 'token',
  `biz_type` varchar(63) NOT NULL DEFAULT '' COMMENT '此token可访问的业务类型标识',
  `remark` varchar(255) NOT NULL DEFAULT '' COMMENT '备注',
  `create_time` timestamp NOT NULL DEFAULT '2010-01-01 00:00:00' COMMENT '创建时间',
  `update_time` timestamp NOT NULL DEFAULT '2010-01-01 00:00:00' COMMENT '更新时间',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4 COMMENT 'token信息表';

INSERT INTO `tiny_id_info` (`id`, `biz_type`, `begin_id`, `max_id`, `step`, `delta`, `remainder`, `create_time`, `update_time`, `version`)
VALUES
	(1, 'test', 1, 1, 100000, 1, 0, '2018-07-21 23:52:58', '2018-07-22 23:19:27', 1);

INSERT INTO `tiny_id_info` (`id`, `biz_type`, `begin_id`, `max_id`, `step`, `delta`, `remainder`, `create_time`, `update_time`, `version`)
VALUES
	(2, 'test_odd', 1, 1, 100000, 2, 1, '2018-07-21 23:52:58', '2018-07-23 00:39:24', 3);


INSERT INTO `tiny_id_token` (`id`, `token`, `biz_type`, `remark`, `create_time`, `update_time`)
VALUES
	(1, '0f673adf80504e2eaa552f5d791b644c', 'test', '1', '2017-12-14 16:36:46', '2017-12-14 16:36:48');

INSERT INTO `tiny_id_token` (`id`, `token`, `biz_type`, `remark`, `create_time`, `update_time`)
VALUES
	(2, '0f673adf80504e2eaa552f5d791b644c', 'test_odd', '1', '2017-12-14 16:36:46', '2017-12-14 16:36:48');
复制代码

Tips: The data has been modified by CHARSET, and utf8 has become utf8mb4

Change setting:

修改tinyid-server/src/main/resources/offline/application.properties

datasource.tinyid.type=org.apache.tomcat.jdbc.pool.DataSource

datasource.tinyid.primary.driver-class-name=com.mysql.jdbc.Driver
datasource.tinyid.primary.url=jdbc:mysql://192.168.43.129:3306/leaf?useUnicode=true&characterEncoding=utf-8
datasource.tinyid.primary.username=root
datasource.tinyid.primary.password=sys123456
复制代码

I am using mysql8, so I modified mysql-connector-javathe to 8.0.28.

Start the service:

cd tinyid-server/
sh build.sh offline
java -jar output/tinyid-server-0.1.0-SNAPSHOT.jar
复制代码

After startup:

This startup is much simpler and easier than Meituan's Leaf.

2.2 REST API

nextId:
curl 'http://localhost:9999/tinyid/id/nextId?bizType=test&token=0f673adf80504e2eaa552f5d791b644c'
response:{"data":[2],"code":200,"message":""}

nextId Simple:
curl 'http://localhost:9999/tinyid/id/nextIdSimple?bizType=test&token=0f673adf80504e2eaa552f5d791b644c'
response: 3

with batchSize:
curl 'http://localhost:9999/tinyid/id/nextIdSimple?bizType=test&token=0f673adf80504e2eaa552f5d791b644c&batchSize=10'
response: 4,5,6,7,8,9,10,11,12,13

Get nextId like 1,3,5,7,9...
bizType=test_odd : delta is 2 and remainder is 1
curl 'http://localhost:9999/tinyid/id/nextIdSimple?bizType=test_odd&batchSize=10&token=0f673adf80504e2eaa552f5d791b644c'
response: 3,5,7,9,11,13,15,17,19,21
复制代码

tinyid provides single ID production as well as batch generation.

Tips: Batch generation improves efficiency, but there will be waste if the caller uses the generated batch IDs that are not used up.

2.3 Java-client usage

We can use the test class of the tinyid project's java-client:

Modify tinyid_client.properties :

tinyid.server=172.27.53.158:9999
tinyid.token=0f673adf80504e2eaa552f5d791b644c
复制代码

3. Summary

The idea of tinyid is to achieve the same segmentation mode as Meituan Leaf through segmentation. Improvements have been made to Meituan's segment mode. At the same time, the client mode is provided. The client mode increases the efficiency of ID generation and improves the fault tolerance rate. If HTTP is not available but the local client is available for a while. The length of time used also depends on the length of the step and the rate at which the ID is consumed.

I am an ant carrying an elephant. The article is helpful to you. Like and follow me. If the article is incorrect, please leave a comment~ Thank you

References: