Detailed mysql partition table and points

concept

Do not be content in a number of jobs to flicker, the partition is a function mysql support mechanism, while the sub-table is a data storage engineers use mysql basic functions to achieve.

Partition table is divided with two similar but different concepts.

In fact, there is no sub-mysql table this concept, sub-table is an engineer on behalf of human treatment, namely a table of data is too much (the amount of data will produce one million a day), it might Engineer consider sub-table to operate (e.g., in time, for example one day create a table or a table created in January), thus created out of the table, the table structure thereof is the same, but different data stores. Broadly speaking, can be considered a table of contents, but each table is a separate entity, each table has its own table metadata files (.frm) and table space file (.ibd) (example illustrates the use of default innodb engine, properly speaking, unless you have special requirements specific to the use of myisam, it is recommended to use innodb).

The zoning is a mechanism inside mysql to achieve, is partitioned operating table referred to as the partition table, partition is performed by keyword partition by. The partition table is a table of contents, after dividing partition according to some specified rule, it can be stored together with the associated content, so that the operation of the data table can be more efficient. Mysql partition table is an internal implementation, it is logically a table (only a .frm file), but in a different partitioning rule may have a plurality of tablespace files (.ibd) used to store different data partition.

Partition table principle

The partition table is an independent logical table, but a plurality of physical sub underlying tables (the data stored in the physical sub-table).

Call the partition table in the underlying implementation package has been achieved, so for SQL is concerned, it is transparent, you are operating this table, but after the optimization process and through a variety of underlying operating table for this will become the a partition (physical child table).

Mysql embodiment is implemented by encapsulating the partition table of the underlying table, so the index is in accordance with the child partition definition table, without global index.

When you create a table, use the "partition by" clause to define the data stored in each partition. And when performing a query, mysql optimizer will not have to exclude partition data according to the definition of partition, so that the query needs to scan only a limited number of the partition that contains the data needed to narrow the scope of the inquiry scan.

Application and advantages of the partition table

• Table is very large that they can not all be placed in memory, or hot data only in the last part of the table, the other is the historical data;
• Data partition tables easier to maintain (want to batch delete data can be used to clear the entire partition of the way, can also be optimized, inspection, repair and other operations on a separate partition);
• Partition table data may be distributed on different physical devices;
• Partition table may be used to avoid certain specific bottlenecks, such as the mutually exclusive access innodb single index, the inode lock contention ext3 file system, and the like;
• You can back up and restore a separate partition.

Limit the partition table

• A table can be up to 1024 partitions;
• Partition expression must be an integer, or the expression returns an integer, of course, also possible to directly use the column partition;
• Partition field must be unique or primary key indexes;
• Partition table can not use foreign key constraints.

Operating logic of the partition table

select query

When a query partition table, partition layer to open and lock all the underlying table, the optimizer determines whether the bid partition filtered, and then call the corresponding interface to access data stored engine each partition.

innodb will release the corresponding table locks in the partition layer, the locking and unlocking process is similar to a general inquiry.

insert operation

When a record is written, the partition layer to open and lock all the underlying table, and then determine which partition receives this record, and then written into the corresponding underlying record table.

delete operation

When a record is deleted, the partition layer to open and lock all the underlying table, and then determine the data corresponding to the partition, and finally the corresponding underlying table delete operation.

update operation

When updating a record, a partition layer to open and lock all the underlying table to determine which records need to be updated in the partition, and then remove and update the data, and then determines which partition the updated data should be placed, and finally the underlying table write operation, and the underlying raw data table where the delete operation.

Supported partition types

mysql support range, hash, key, list, composite partition various ways, there are two commonly used, namely, range and range division manner hash hashing division manner.

Partition expression can be a column, it can be an expression that contains a column. Partition clause can be used in a variety of functions, but the value of the expression returns the partition needs to be a certain integer, and can not be a constant .

Create a partition table

Use the date range to be divided to create a partition table test_partition

create table test_partition( 
    id int not null auto_increment, 
    name varchar(16) default '', 
    birth datetime, 
    primary key(id,birth) 
)engine=innodb default charset=utf8 partition by range(year(birth))( 
    partition p_2016 values less than (2016), 
    partition p_2017 values less than (2017), 
    partition p_2018 values less than (2018), 
    partition p_catchall values less than maxvalue);

We can look at the file table after table of test_partition create success is how:

# ls -l
total 1180
-rw-r----- 1 mysql mysql    65 Nov  2 11:37 db.opt
-rw-r----- 1 mysql mysql  8618 Nov 21 10:20 test_partition.frm
-rw-r----- 1 mysql mysql 98304 Nov 21 10:20 test_partition#P#p_2016.ibd
-rw-r----- 1 mysql mysql 98304 Nov 21 10:24 test_partition#P#p_2017.ibd
-rw-r----- 1 mysql mysql 98304 Nov 21 10:20 test_partition#P#p_2018.ibd
-rw-r----- 1 mysql mysql 98304 Nov 21 10:20 test_partition#P#p_catchall.ibd

Is, just as described earlier, the partition table has a total gauge metadata file (.frm), a plurality of tablespace files (.ibd, were separated by using identification #).

Child partitions

Partition mysql partition table also supports the sub-partition mode, i.e., the partition table partition partition.

We use test_partition to create a partition that contains sub-partition table test_subpartition

create table test_subpartition( 
    id int not null auto_increment, 
    name varchar(16) default '', 
    birth datetime, 
    primary key(id,birth) 
)engine=innodb default charset=utf8 partition by rangenge(year(birth)) subpartition by hash(id) subpartitions 2 ( 
    partition p_2016 values less than (2016), 
    partition p_2017 values less than (2017), 
    partition p_2018 values less than (2018), 
    partition p_catchall values less than maxvalue);

We look at this contains sub-partition table partition table test_subpartition file is like:

# ls -l
total 1180
-rw-r----- 1 mysql mysql    65 Nov  2 11:37 db.opt
-rw-r----- 1 mysql mysql  8618 Nov 21 14:27 test_subpartition.frm
-rw-r----- 1 mysql mysql 98304 Nov 21 14:27 test_subpartition#P#p_2016#SP#p_2016sp0.ibd
-rw-r----- 1 mysql mysql 98304 Nov 21 14:27 test_subpartition#P#p_2016#SP#p_2016sp1.ibd
-rw-r----- 1 mysql mysql 98304 Nov 21 14:27 test_subpartition#P#p_2017#SP#p_2017sp0.ibd
-rw-r----- 1 mysql mysql 98304 Nov 21 14:27 test_subpartition#P#p_2017#SP#p_2017sp1.ibd
-rw-r----- 1 mysql mysql 98304 Nov 21 14:27 test_subpartition#P#p_2018#SP#p_2018sp0.ibd
-rw-r----- 1 mysql mysql 98304 Nov 21 14:27 test_subpartition#P#p_2018#SP#p_2018sp1.ibd
-rw-r----- 1 mysql mysql 98304 Nov 21 14:27 test_subpartition#P#p_catchall#SP#p_catchallsp0.ibd
-rw-r----- 1 mysql mysql 98304 Nov 21 14:27 test_subpartition#P#p_catchall#SP#p_catchallsp1.ibd

Yes, each partition (p_2016, p_2017, p_2018 and p_catchall) table space file (.idb) and is divided into two named sp0, sp1 sub-table space file.

Special case

Suppose there is a table increment primary key column id, according to the desired time to the nearest hotspot data stored centrally. So depending on the time partition must contain a time stamp in the primary key for the job and significance of these primary keys and contradicts itself.

In this case can be used in this way (id div 1000000), div divisible operation is performed, so that the data into a partition 1,000,000, both to achieve the purpose of partitioning, but also avoids the use of time after exceeding a threshold required partitioning new issues partition.

Query optimization partition table

For partitioned tables, brought in where conditions partitioning column, the optimizer can filter out the partition without having to visit, so sometimes even seemingly superfluous and should bring partitioning column , otherwise you may access the storage engine table of all partition, if the table is very large, the speed will be very slow.

Use "explain partition" can be observed optimizer whether the partition filtering.

Note that mysql can only be filtered partition when compared to the use of the partition function of the column itself, and can not go to filter partition based on the value of the expression, even if this expression is the partition function does not work.

So even if you can use an expression when you create a partition, but in the query to filter Shique only partitions based on column.