percona-toolkit in pt-online-schema-change tools to install and use
pt-online-schema-change Introduction
Usage scenarios: large online modify table structure
In the maintenance of the database, there is always involved in the case to modify the table structure of the production environment, modify some small tables has little effect, but when you modify a large table, often affect normal operation of the business, such as table data exceeds 500W, 1000W, and even when billions of dollars
Online modifications may affect large table
(1) online modify table structure of a large table execution time is often unpredictable, usually longer
(2) due to changes in the structure of the table is a table-level locking, so when you modify table structure, influence tables write in operation
(3) If a long table structure changes, modification failure way, since a transaction is modified table structure, and therefore is restored after the failure of the table structure, the process table is locked unwritable
(4) extensive modification easily lead to the database table structure CPU, IO consumption and other properties, so that performance degradation MySQL server
(5) to modify line table structure easily lead to large delay from the master, thus affecting the read operations
Tools introduces
pt-online-schema-change is a tool percona developed, this feature can be found in percona-toolkit package inside, it can modify table structure online
Principle:
(1) First, it will create a new document exactly the same table, the table name suffix is generally _new
(2) and then make changes in this new field operating table
(3) and then add three trigger on the original table, DELETE / UPDATE / INSERT, the new table is also performed the original sentence to be executed in the table
(4) Finally, the original copy of the data table to the new table, and then replace the original table
This process there are two problems that require attention:
1. trigger
because the whole process is online, in order to change the table during the update to the original table while updating to the new table, creates a corresponding trigger whenever occur for additions and deletions to the original operating table, it will trigger the appropriate action for the new table. So you can not have other triggers on the original list, that is, if there trigger on the original table, OSC will strike.
2. foreign key
foreign key to the table to change the operation becomes more complicated if there is a foreign key, then the original table, the automatic operation of the original table and rename the new table can not be smoothly carried out, it must be completed after foreign key to update the data in the copy the new table, the tool has two ways to support this operation, specific parameter (--alter-foreign-keys-method ) implemented.
--alter-foreign-keys-method
must be continuously linked before and after the foreign key table to change the correct table, when the tool and rename the original table with the new table to replace the original tables, foreign key must correctly updated to the new table, and original table the foreign key is no longer to take effect
Foreign keys make operation of the tool is complicated and introduces additional risks. When a foreign key references the table, the original table rename atomic operations and the new table will not function properly. After the table structure is modified, the tool must be modified to point to the new foreign key table. The tool supports two ways to achieve this, specific reference --alter-foreign-keys-method documentation. Foreign key can also cause some side effects. The final table will be the same as the original table and the foreign key index (unless you specified in the ALTER statement respectively), but the name of the object may be slightly modified in order to avoid conflicts object name in MySQL and InnoDB.
For security reasons, the tool does not actually modify the table, unless you specify --execute option, but this option is not enabled by default. The tool supports a variety of other measures to prevent undesired load or other problems, including automatic detection from the node, connected to them, and using the following security checks:
(1) in most cases, unless the table has PRIMARY KEY or UNIQUE INDEX, the tool will operate. See --alter options.
(2) If it is detected that there is replication filter, the tool will reject operation. See - [no] check-replication- filters option.
(3) The tool data copying operation stops, if it is observed from the delay value greater than the master --max-lag option defaults to 1s.
(4) The tool will stop or abort the operation if it detects that cause too much load on the server. See --max-load and --critical-load option.
(5) This tool set innodb_lock_wait_timeout = 1 and (for MySQL 5.5 and newer versions) lock_wait_timeout = 60, so it would be more likely to become victims of lock contention, and less damage to other matters. These values may be modified by specifying --set-vars.
(6) The tool will refuse to modify the table, if there are foreign key constraints referencing it, unless you specify --alter-foreign-keys-method.
benefit:
Reduce the risk of delay from the master
When can the speed limit, limited resources, avoid excessive load MySQL operation
Suggest:
Doing business low peak, will minimize the impact
percona-toolkit installation
1. go to the official website to download the corresponding version of the official website Download: https://www.percona.com/downloads/percona-toolkit/LATEST/
wget https://www.percona.com/downloads/percona-toolkit/3.1/binary/redhat/7/x86_64/percona-toolkit-3.1-1.el7.x86_64.rpm
2. Installation depends
yum -y install perl-DBI perl-DBD-MySQL perl-Digest-MD5 perl-IO-Socket-SSL perl-TermReadKey
3. Install
rpm -ivh percona-toolkit-3.1-1.el7.x86_64.rpm
Yum mounted directly or
with yum automatically install dependencies, relatively simple
yum install -y percona-toolkit-3.0.12-1.el6.x86_64.rpm
verification
pt-online-schema-change --help
pt-online-schema-change using
Modified table structure without blocking read and write operations
./bin/pt-online-schema-change --help can view the parameters
DNS options are commonly used:
--user = username connecting the mysql --password = mysql connection password --host = mysql connection address P = 3306 connection port number mysql D = library name is connected mysql T = mysql connection the table name - the ALTER tABLE statement to modify the structure - the execute execution modify table structure --charset = utf8 use utf8 encoding, to avoid Chinese garbled --no-version-check does not check and update percona toolkit version, does not check mysql version Wait.
Case:
Field of the SQL statement to add: the ALTER TABLE test.liu_test the COLUMN Liu the ADD int (10) the DEFAULT NULL;
pt-online-schema-change --user=root --password=xxx --host=172.16.xx.xx P=3306,D=test,t=liu_test --charset=utf8 --no-version-check --execute --alter "ADD COLUMN liu int(10) DEFAULT NULL"
Reference official website: https: //www.percona.com/doc/percona-toolkit/LATEST/pt-online-schema-change.html