Testing process
foreword
During the interview, I was asked which one is faster for inserting data between snowflake id and auto-increment id? I personally think that the database self-increases the fastest, and there is no explanation for how much faster it is. Do a test to see how fast it is. The test results are for reference only.
2. Use steps
1. Introduce the snowflake id generation tool hutool
The code is as follows (example):
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>5.1.0</version>
</dependency>
2. Create the table used in the experiment
Use the normal auto-increment id table:
CREATE TABLE `a` (
`id` int(11) NOT NULL AUTO_INCREMENT COMMENT '主键',
`name` varchar(255) DEFAULT NULL,
`sex` int(255) DEFAULT NULL,
`age` int(11) DEFAULT NULL,
`insertTime` datetime DEFAULT NULL,
UNIQUE KEY `id` (`id`),
KEY `nameindex` (`name`,`sex`,`age`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=20001 DEFAULT CHARSET=utf8;
snowflake id table
CREATE TABLE `a1` (
`id` bigint(20) DEFAULT NULL,
`name` varchar(255) DEFAULT NULL,
`sex` int(255) DEFAULT NULL,
`age` int(11) DEFAULT NULL,
`insertTime` datetime DEFAULT NULL,
UNIQUE KEY `id` (`id`),
KEY `nameindex` (`name`,`sex`,`age`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
3. Test code
The control layer is as follows:
@RequestMapping(value = "/hello2",method = RequestMethod.GET)
public String index3() {
userService.insertUser1();
//helloService.sayHello();
// redisTemplate.opsForValue().set("aaa","xxx");
return "xxxx";
}
@RequestMapping(value = "/hello3",method = RequestMethod.GET)
public String index4() {
userService.insertUser2();
//helloService.sayHello();
// redisTemplate.opsForValue().set("aaa","xxx");
return "xxxx";
}
The service class is as follows
: insertUser1 is for self-incrementing id insertion, and insertUser2 is for snowflake id insertion.
@Override
public void insertUser1() {
long starTime = System.currentTimeMillis();
for (int j = 0; j < 10; j++) {
List<User> list = new ArrayList<>();
for(int i = 0; i < 2000; i++) {
User user = new User();
Date date = new Date();
user.setName("小红"+date.getTime());
user.setAge(12);
user.setSex(1);
user.setInsertTime(date);
list.add(user);
}
userDao.insertUser1(list);
}
System.out.println("插入总耗时:{}"+ (System.currentTimeMillis()-starTime)+"ms");
}
@Override
public void insertUser2() {
long starTime = System.currentTimeMillis();
Snowflake snowflake = IdUtil.getSnowflake(31,31);
for (int j = 0; j < 10; j++) {
List<User1> list = new ArrayList<>();
for(int i = 0; i < 2000; i++) {
User1 user = new User1();
Date date = new Date();
user.setId(snowflake.nextId());
user.setName("小红"+date.getTime());
user.setAge(12);
user.setSex(1);
user.setInsertTime(date);
list.add(user);
}
userDao.insertUser2(list);
}
System.out.println("插入总耗时:{}"+ (System.currentTimeMillis()-starTime)+"ms");
}
IdUtil.getSnowflake(31,31) The two parameters here are workerid and datacenterid, and the two parameters form the machine id.
4.xml statement
<insert id="insertUser1" parameterType="com.example.springdemo.demo.model.User">
insert into a
(name,sex,age,insertTime) values
<foreach collection="list" item="item" index="index" separator=",">
(
#{
item.name},
#{
item.sex},
#{
item.age},
#{
item.insertTime}
)
</foreach>
</insert>
<insert id="insertUser2" parameterType="com.example.springdemo.demo.model.User1">
insert into a1
(id,name,sex,age,insertTime) values
<foreach collection="list" item="item" index="index" separator=",">
(
#{
item.id},
#{
item.name},
#{
item.sex},
#{
item.age},
#{
item.insertTime}
)
</foreach>
5. Start the test
1. First test that both insert 20,000 records.
The result of first calling the auto-increment id
is as follows:
2023-02-20 11:26:35.003 INFO 10680 --- [nio-8008-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2023-02-20 11:26:35.003 INFO 10680 --- [nio-8008-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2023-02-20 11:26:35.005 INFO 10680 --- [nio-8008-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 2 ms
插入总耗时:{
}7710ms
The result of calling Snowflake ID again
is as follows;
插入总耗时:{
}7880ms
The self-increment is 170ms faster than the snowflake id
2. Test both tables by inserting 200,000 records.
Clear both tables and test again.
truncate table a;
truncate table a1;
auto-increment id test
2023-02-20 11:39:11.374 INFO 14136 --- [nio-8008-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2023-02-20 11:39:11.374 INFO 14136 --- [nio-8008-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2023-02-20 11:39:11.375 INFO 14136 --- [nio-8008-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 1 ms
插入总耗时:{
}14814ms
Snowflake id test
test results
插入总耗时:{
}17030ms
The auto-increment is 2216ms faster than the snowflake id.
I tested it many times before and after, and the result is that the auto-increment id is faster.
Summarize
1. The database type of snowflake id should use bigint, and the java type should use long type. If the database type is not bigint, an error will be reported as follows
### Error updating database. Cause: com.mysql.cj.jdbc.exceptions.MysqlDataTruncation: Data truncation: Out of range value for column 'id' at row 1
### The error may exist in file [D:\javaTools\ideaCode\springbootdemo\target\classes\com\example\springdemo\demo\dao\UserDao.xml]
### The error may involve com.example.springdemo.demo.dao.UserDao.insertUser2-Inline
### The error occurred while setting parameters
2. Experiments have proved that the auto-increment id is indeed faster than the snowflake id, but the current system is basically a distributed system. The reason for using the snowflake id is
2.1 to avoid id duplication, and the snowflake id is fixed at the time stamp + machine id case in order. Basically meet the data requirements of the index.
2.2 It is necessary to generate id in advance to do other business operations, instead of inserting data and returning id.
2.3 Data security requirements, self-incrementing id allows users to directly see the id and guess the amount of data.
3. The snowflake id affects the performance of data insertion, but no better solution has been found so far, and it is acceptable to sacrifice some performance.