Use MySQL stored procedures or delete test data volume generated

1. Objectives

Sometimes you need to insert a large amount of data in the database for testing applications running in the large amount of data is normal, you can use MySQL stored procedures, or delete the volume generated test data.

2. The stored procedure script

In the following script, each insert / delete 5000 data submitted once. Insert 100,000 data takes about 3.7 seconds to delete it takes about 1.2 seconds.

When performing bulk insert / delete operations, may be inserted according to the modified total number of records required, the number of records each transaction inserted, the specific operation of insertion, the number of deleted records each transaction, specific deletion operation.

2.1. Batch generate test data

DROP PROCEDURE IF EXISTS insert_procedure;

# 将语句的结束符号从分号 ; 临时改为两个//
delimiter //

CREATE PROCEDURE insert_procedure ()
exit_label:
BEGIN
   -- 插入总记录数（根据需要修改）
   DECLARE total        int DEFAULT 100000;
   -- 每次事务插入的记录数（根据需要修改）
   DECLARE page         int DEFAULT 5000;
   -- 每次插入实际记录数
   DECLARE insert_num   int DEFAULT 0;
   -- 提交插入的总次数
   DECLARE num          int DEFAULT 0;
   -- 外层循环下标
   DECLARE n1           int DEFAULT 1;
   -- 内层循环下标
   DECLARE n2           int DEFAULT 1;
   -- 当前循环的起始数字
   DECLARE start        int DEFAULT 0;
   -- 每次插入的流水号字段
   DECLARE seq          VARCHAR(32);

   -- 除法只有整数部分，没有小数部分
   SET num = total / page;

   IF (num = 0)
   THEN
      -- 退出
      LEAVE exit_label;
   END IF;

   -- 判断模是否为0，非0时需要将总次数加1
   IF (total % page != 0)
   THEN
      SET num = num + 1;
   END IF;

   -- 外层循环
   WHILE n1 <= num
   DO
      -- 开始事务
      START TRANSACTION;

      IF (n1 < num)
      THEN
         -- 非最后一次插入
         SET insert_num = page;
      ELSE
         -- 最后一次插入
         SET insert_num = total - page * (num - 1);
      END IF;

      -- 内层循环
      SET start = page * (n1 - 1);

      WHILE n2 <= insert_num
      DO
         -- 执行插入（根据需要修改）
         SET seq = concat("testtime", unix_timestamp(now()), "num", start + n2);

         INSERT INTO test_table(id,flag,create_time,update_time)
         VALUES (seq,seq,now() - INTERVAL '5' DAY,now() - INTERVAL '5' DAY);

         SET n2 = n2 + 1;
      END WHILE;

      -- 提交事务
      COMMIT;

      -- 外层循环下标加1
      SET n1 = n1 + 1;
      -- 内层循环下标置1
      SET n2 = 1;
   END WHILE;
END
//
# 将语句的结束符号恢复为分号 ;
delimiter ;

# 执行存储过程
CALL insert_procedure;


SHOW PROCEDURE STATUS LIKE '%insert_procedure%';

SHOW PROCESSLIST;

2.2. Batch delete the test data

DROP PROCEDURE IF EXISTS delete_procedure;

# 将语句的结束符号从分号 ; 临时改为两个//
delimiter //

CREATE PROCEDURE delete_procedure ()
exit_label:
BEGIN
   -- 每次事务删除记录数（根据需要修改）
   DECLARE page         int DEFAULT 5000;
   -- 记录每次删除返回记录数
   DECLARE delete_num   int DEFAULT 0;

   WHILE 1 = 1
   DO
      -- 开启事务
      START TRANSACTION;

      -- 执行删除操作（根据需要修改）
      DELETE FROM test_table
      WHERE id LIKE 'testtime%'
      LIMIT page;

      -- 获取被删除的行数，需要在提交事务之前执行
      SET delete_num = ROW_COUNT();

      -- 提交事务
      COMMIT;

      IF (delete_num = 0)
      THEN
         -- 退出
         LEAVE exit_label;
      END IF;
   END WHILE;
END
//
# 将语句的结束符号恢复为分号 ;
delimiter ;

# 执行存储过程
CALL delete_procedure;

3. Other commands

3.1. Check the stored procedure status

Performing "SHOW PROCEDURE STATUS" command to view the stored procedure, the command as follows:

SHOW PROCEDURE STATUS LIKE '%insert_procedure%';

Example output:

Db	Name	Type	Definer	Modified	Created	Security_type	Comment	character_set_client	collation_connection	Database Collation
testdb	insert_procedure	PROCEDURE	test@localhost	2020/1/9 21:31:00	2020/1/9 21:31:00	DEFINER		latin1	latin1_swedish_ci	latin1_swedish_ci

3.2. View the currently executing stored procedure

Performing "SHOW PROCESSLIST;" can view the currently executing stored procedure, the following exemplary output.

Id	User	Host	db	Command	Time	State	Info	Progress
218919458	test	localhost:64027	testdb	Query	0	call	my_procedure	0

3.3. End the stored procedure execution

When the need to end the stored procedure is being performed, perform "KILL" command value Id plus the above query results, such as "KILL 218919458;".