Partition table principle
Partition table is implemented by multiple related underlying table, this table is identified by a handle to the underlying object, we can access each partition directly. Storage engine management partition table and the bottom of each table as managing general (all underlying tables must use the same storage engine), knowledge index partition table on each respective underlying tables plus an identical index. From the perspective of the storage engine point of view, the underlying table and ordinary table no different storage engines also need to know this is part of a regular table or a partitioned table.
Partition table operation is performed according to the following operating logic:
select query
When a query when a partition table, partition layer to open and lock all the underlying table, the optimizer determines whether the filtering portion to the partition, and then call the corresponding interface to access data stored in each engine partition
insert operation
When a record is written, the partition layer to open and lock all the underlying table, and then determine which partition to accept this record, and then written into the corresponding underlying record table
delete operation
When a record is deleted, the partition layer to open and lock all the underlying table, and then determine the data corresponding to the partition, and finally the corresponding underlying table delete operation
update operation
When updating a record, a partition layer to open and lock all the underlying tables, mysql first record to be updated and then determining which partition, and then remove and update the data, and then determine the updated data which should be re-partition, and finally underlying table write operation, and the underlying data source table where the delete operation
Some operational support filtering, for example, when you delete a record, MySQL will need to find this record, if the conditions are exactly where and partition expression matching, you can partition does not contain all of these records are filtered out, which update equally effective. If insert operation, itself hit only one partition, the other partition will be filtered out. mysql first determine which records belong to which partition, and then write the corresponding record was once the partition table, do not operate on any other partition
While each operation will be "the first to open and lock all the underlying table," but that does not mean that the partition table is locked full table in the process, if the storage engine to achieve their row-level locking, for example innodb, the correspondence table lock release layer partition.
Partition table type
Range partitioning
The column values within a given range of the rows are assigned to partition
the partition table is range-partitioned manner: Each partition contains data line and expression in a given partition the scope of the partition should be continuous and can not overlap You may be used values less than the operator defined.
1, create a common table
CREATE TABLE employees (
id INT NOT NULL,
fname VARCHAR(30),
lname VARCHAR(30),
hired DATE NOT NULL DEFAULT '1970-01-01',
separated DATE NOT NULL DEFAULT '9999-12-31',
job_code INT NOT NULL,
store_id INT NOT NULL
);2, create the table with partitions, the statement is built in accordance with the table below store_id to partition, specify the partition four
CREATE TABLE employees (
id INT NOT NULL,
fname VARCHAR(30),
lname VARCHAR(30),
hired DATE NOT NULL DEFAULT '1970-01-01',
separated DATE NOT NULL DEFAULT '9999-12-31',
job_code INT NOT NULL,
store_id INT NOT NULL
)
PARTITION BY RANGE (store_id) (
PARTITION p0 VALUES LESS THAN (6),
PARTITION p1 VALUES LESS THAN (11),
PARTITION p2 VALUES LESS THAN (16),
PARTITION p3 VALUES LESS THAN (21)
);
--在当前的建表语句中可以看到，store_id的值在1-5的在p0分区，6-10的在p1分区，11-15的在p3分区，16-20的在p4分区，但是如果插入超过20的值就会报错，因为mysql不知道将数据放在哪个分区3, you can use less than maxvalue to avoid this
CREATE TABLE employees (
id INT NOT NULL,
fname VARCHAR(30),
lname VARCHAR(30),
hired DATE NOT NULL DEFAULT '1970-01-01',
separated DATE NOT NULL DEFAULT '9999-12-31',
job_code INT NOT NULL,
store_id INT NOT NULL
)
PARTITION BY RANGE (store_id) (
PARTITION p0 VALUES LESS THAN (6),
PARTITION p1 VALUES LESS THAN (11),
PARTITION p2 VALUES LESS THAN (16),
PARTITION p3 VALUES LESS THAN MAXVALUE
);
--maxvalue表示始终大于等于最大可能整数值的整数值List Partitioning
Similarly as in the range partitioning, except that the list is based on a partition column value matches a set of discrete values to be selected
CREATE TABLE employees (
id INT NOT NULL,
fname VARCHAR(30),
lname VARCHAR(30),
hired DATE NOT NULL DEFAULT '1970-01-01',
separated DATE NOT NULL DEFAULT '9999-12-31',
job_code INT,
store_id INT
)
PARTITION BY LIST(store_id) (
PARTITION pNorth VALUES IN (3,5,6,9,17),
PARTITION pEast VALUES IN (1,2,10,11,19,20),
PARTITION pWest VALUES IN (4,12,13,14,18),
PARTITION pCentral VALUES IN (7,8,15,16)
);Column partition
From the start to support mysql 5.5 column partition, i can think of is the range and list an upgraded version after 5.5, you can use an alternative range and list partitioning column, but the column partition only accept ordinary columns do not accept the expression
CREATE TABLE `list_c` (
`c1` int(11) DEFAULT NULL,
`c2` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1
/*!50500 PARTITION BY RANGE COLUMNS(c1)
(PARTITION p0 VALUES LESS THAN (5) ENGINE = InnoDB,
PARTITION p1 VALUES LESS THAN (10) ENGINE = InnoDB) */
CREATE TABLE `list_c` (
`c1` int(11) DEFAULT NULL,
`c2` int(11) DEFAULT NULL,
`c3` char(20) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1
/*!50500 PARTITION BY RANGE COLUMNS(c1,c3)
(PARTITION p0 VALUES LESS THAN (5,'aaa') ENGINE = InnoDB,
PARTITION p1 VALUES LESS THAN (10,'bbb') ENGINE = InnoDB) */
CREATE TABLE `list_c` (
`c1` int(11) DEFAULT NULL,
`c2` int(11) DEFAULT NULL,
`c3` char(20) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1
/*!50500 PARTITION BY LIST COLUMNS(c3)
(PARTITION p0 VALUES IN ('aaa') ENGINE = InnoDB,
PARTITION p1 VALUES IN ('bbb') ENGINE = InnoDB) */hash partitioning
Partition be selected based on the return value of a user-defined expression, the expression is used to be inserted into the table column values of these lines are calculated. This function can include any expression myql valid, generating a non-negative integer values
CREATE TABLE employees (
id INT NOT NULL,
fname VARCHAR(30),
lname VARCHAR(30),
hired DATE NOT NULL DEFAULT '1970-01-01',
separated DATE NOT NULL DEFAULT '9999-12-31',
job_code INT,
store_id INT
)
PARTITION BY HASH(store_id)
PARTITIONS 4;
CREATE TABLE employees (
id INT NOT NULL,
fname VARCHAR(30),
lname VARCHAR(30),
hired DATE NOT NULL DEFAULT '1970-01-01',
separated DATE NOT NULL DEFAULT '9999-12-31',
job_code INT,
store_id INT
)
PARTITION BY LINEAR HASH(YEAR(hired))
PARTITIONS 4;key partition
Hash partitioning is similar, except that only supports the partition key one or more columns, and provides its own mysql server hash function, there must be one or more columns comprise integer values
CREATE TABLE tk (
col1 INT NOT NULL,
col2 CHAR(5),
col3 DATE
)
PARTITION BY LINEAR KEY (col1)
PARTITIONS 3;Child partitions
On the basis of the partition on, and then after partition storage
CREATE TABLE `t_partition_by_subpart`
(
`id` INT AUTO_INCREMENT,
`sName` VARCHAR(10) NOT NULL,
`sAge` INT(2) UNSIGNED ZEROFILL NOT NULL,
`sAddr` VARCHAR(20) DEFAULT NULL,
`sGrade` INT(2) NOT NULL,
`sStuId` INT(8) DEFAULT NULL,
`sSex` INT(1) UNSIGNED DEFAULT NULL,
PRIMARY KEY (`id`, `sGrade`)
) ENGINE = INNODB
PARTITION BY RANGE(id)
SUBPARTITION BY HASH(sGrade) SUBPARTITIONS 2
(
PARTITION p0 VALUES LESS THAN(5),
PARTITION p1 VALUES LESS THAN(10),
PARTITION p2 VALUES LESS THAN(15)
);Partition table limit
1. A table can have a maximum of 1024 partitions, in version 5.7 when you can support 8196 partition
2. In the early mysql, the partition must be an integer expression or an expression that returns an integer, in mysql5.5 in some scenarios can be directly used to partition the column
3. If there is a unique primary key or index partition field, then all primary key columns and columns must contain a unique index to come
4. partition table foreign key constraints can not be used
how to use
Big Data Split (even using the index, you will find will generate a lot of debris, but also generate a lot of random IO)
simple 1. Use partitioning to store tables, not to any index, according to zoning rules generally positioned until the data required by where conditions require the use of data is limited to a small number of partitions, this policy applies to the normal way to access large amounts of data
2. If the data has obvious hot spots, and in addition to this part of the data, other data is rarely accessed, you can this part of the hot data on a separate partition, the partition so that the data can have the opportunity to have cached in memory, so that the query can access only a small partition table, can use the index, it is possible to effectively use the cache
Scenarios
1. The table is very large that they can not all be placed in memory, or hot data only in the last part of the table, are other historical data
2. Data partition table is easier to maintain. Batch delete large amounts of data can be used to clear the entire partition of the way; on a separate partition to optimize inspection, repair and other operations.
3. The partition table data may be distributed on different physical devices, thereby efficiently utilize a plurality of hardware devices
4. The partition table can be used to prevent certain special bottleneck. exclusive access to a single index of innodb; inode lock ext3 file system competition.
The backup and restore can be separate partition
Attention to the problem
null value will filter invalid partition
partitioning columns and index columns do not match, the query will not lead to partition the filter
selected partitions may be costly (a partition will store a file, it will take up a lot of storage space)
to open and lock all the underlying cost table may be high
maintenance costs can be high partition (partition change you want to delete the old partitions, create new partitions)