Mysql tens of millions of levels of sub-table optimization

Demand: As the amount of data increases, a single table can no longer support the business well, and the query of tens of millions of data is slow

Mysql data optimization scheme:

1. Solution 1 : Use myisam to optimize the level table
2. Option 2 : Use mysql partition optimization

One: Myisam horizontal partition

1. Create a level table user_1:

--  创建水平分表
create table user_1(
	id varchar(50) PRIMARY key COMMENT '主键id',
	user_name varchar(50) DEFAULT null COMMENT '用户名称',
	creator varchar(50) DEFAULT null COMMENT '创建者',
	create_date datetime DEFAULT null COMMENT '创建时间',
	updater varchar(50) DEFAULT null COMMENT '更新者',
	update_date datetime DEFAULT null COMMENT '更新时间'
)ENGINE=MyISAM DEFAULT CHARSET=utf8 COMMENT='用户1表';

2. Create a level table user_2:

create table user_2(
	id varchar(50) PRIMARY key COMMENT '主键id',
	user_name varchar(50) DEFAULT null COMMENT '用户名称',
	creator varchar(50) DEFAULT null COMMENT '创建者',
	create_date datetime DEFAULT null COMMENT '创建时间',
	updater varchar(50) DEFAULT null COMMENT '更新者',
	update_date datetime DEFAULT null COMMENT '更新时间'
)ENGINE=MyISAM DEFAULT CHARSET=utf8 COMMENT='用户2表';

3. Create a horizontal table user

 --  创建总表
create table user(
	id varchar(50) PRIMARY key COMMENT '主键id',
	user_name varchar(50) DEFAULT null COMMENT '用户名称',
	creator varchar(50) DEFAULT null COMMENT '创建者',
	create_date datetime DEFAULT null COMMENT '创建时间',
	updater varchar(50) DEFAULT null COMMENT '更新者',
	update_date datetime DEFAULT null COMMENT '更新时间'
)ENGINE=MRG_MyISAM DEFAULT CHARSET=utf8 INSERT_METHOD=LAST UNION=(
`user_1`,`user_2`);

4. Newly added partition user_3

create table user_3(
	id varchar(50) PRIMARY key COMMENT '主键id',
	user_name varchar(50) DEFAULT null COMMENT '用户名称',
	creator varchar(50) DEFAULT null COMMENT '创建者',
	create_date datetime DEFAULT null COMMENT '创建时间',
	updater varchar(50) DEFAULT null COMMENT '更新者',
	update_date datetime DEFAULT null COMMENT '更新时间'
)ENGINE=MyISAM DEFAULT CHARSET=utf8 COMMENT='用户3表';

Initialization data 

--  插入数据
insert into user_1(`id`,`user_name`,`create_date`,`update_date`) values ('1','张三',now(),now());
insert into user_1(`id`,`user_name`,`create_date`,`update_date`) values ('2','李四',now(),now());
insert into user_2(`id`,`user_name`,`create_date`,`update_date`) values ('1','王五',now(),now());
insert into user_3(`id`,`user_name`,`create_date`,`update_date`) values ('3','赵六3',now(),now());
insert into user(`id`,`user_name`,`create_date`,`update_date`) values ('4','老赵1',now(),now());

Verify data results: pass

 

5. Re-add partition

alter table user UNION=(`user_1`,`user_2`,`user_3`);

Parameter Description 

1：分表ENGINE必须为：MyISAM，总表必须为MRG_MyISAM
2：INSERT_METHOD：向总表插入数据默认到哪个位置取值：
       0：不允许插入
       1：允许插入
       FIRST：merge的第一张表
       LAST：merge的最后一张表

An abnormal error occurred

1168 - Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist

 identify the problem:

•  The sub-table MERGE engine is only suitable for MYISAM tables
• Union has a table that does not exist or is not in the same database
• The structure of the table is inconsistent

Official website address: https://dev.mysql.com/doc/refman/8.0/en/merge-storage-engine.html