For the convenience of user experience, PolarDB-X provides a free experimental environment, where you can experience the installation and deployment of PolarDB-X and various kernel features. In addition to free experiments, PolarDB-X also provides free video courses to teach you how to use PolarDB-X distributed database.
This experiment will guide you how to use PolarDB-X to import and export data
Free experimental address for this issue
Address of this teaching video
Preparation
Assuming that PolarDB-X has been built and deployed according to the content of the previous lecture , PolarDB-X is installed using PolarDB-X Operator, and the PolarDB-X database can be successfully linked.
Summary of PolarDB-X import and export methods
The common data export methods of PolarDB-X are:
- mysql -e command line to export data
- mysqldump tool to export data
- Select into outfile statement to export data (closed by default)
- Batch Tool tool to export data (the import and export tool supporting PolarDB-X)
The common data import methods of PolarDB-X are:
- The source statement imports data
- MySQL command to import data
- program import data
- The load data statement imports data
- Batch Tool tool imports data (the import and export tool supporting PolarDB-X)
Initial data preparation
1. It is recommended to split the screen and operate with two terminals.
Click on the upper right
The split-screen icon creates two terminals, which are convenient for logging in to the database and executing commands separately.
illustrate:
a. The operation of k8s, that is, the kubectl command, needs to be performed under the galaxykube user;
b. Login to the database and other execution commands can be operated under the root or galaxykube user, just pay attention to the file path.
2. Execute the following command to install sysbench.
curl -s https://packagecloud.io/install/repositories/akopytov/sysbench/script.rpm.sh | sudo bash sudo yum -y install sysbenchNote: You can check whether the installation is successful through sysbench --help.
3. Execute the following command to log in to the database.
Use connection mysql -hip -Pport -uuser -ppassword -Ac to log in to PolarDB-X.
mysql -h127.0.0.1 -P3306 -upolardbx_root -p123456 -Ac4. Execute the following command to create a database.
create database sysbench_int;5. Import initial data through sysbench.
5.1 First execute exit to log out of the database.
5.2 Execute the following command to import initial data through sysbench.
sysbench oltp_insert --mysql-host=127.0.0.1 --mysql-port=3306 --mysql-user=polardbx_root --mysql-password=123456 --mysql-db=sysbench_int --db-driver=mysql --tables=1 --table-size=100000 --report-interval=1 prepare6. Change the single-database single-table built by sysbench by default to the sub-database and sub-table mode.
6.1 Execute the following command to log in to PolarDB-X using the connection mysql -hip -Pport -uuser -ppassword -Ac.
mysql -h127.0.0.1 -P3306 -upolardbx_root -p123456 -Ac6.2 Execute the following command, first rename the original table.
use sysbench_int;
ALTER TABLE sbtest1 RENAME TO sbtest1_single;6.3 Execute the following command to manually create sub-database and sub-table.
CREATE TABLE `sbtest1` (
`id` int(11) NOT NULL AUTO_INCREMENT BY GROUP,
`k` int(11) NOT NULL DEFAULT '0',
`c` char(120) NOT NULL DEFAULT '',
`pad` char(60) NOT NULL DEFAULT '',
PRIMARY KEY (`id`)
) dbpartition by hash(`id`) tbpartition by hash(`id`) tbpartitions 2;6.4 Execute the following command and execute it into the new table through insert select.
INSERT INTO sbtest1 SELECT * FROM sbtest1_single;6.5 Execute the following command to view the initial data information.
show create table sbtest1;
select min(id),max(id),count(id) from sbtest1;
select * from sbtest1 limit 10;Import and export in pure data format
1. Execute exit to log out of the database first.
2. Export the data through the mysql -e command.
2.1 Execute the following command to export data from PolarDB-X through the mysql -e command.
time mysql -h127.0.0.1 -P3306 -upolardbx_root -p123456 sysbench_int -N -e "SELECT id,k,c,pad FROM sbtest1;" > data_10w.txt
time mysql -h127.0.0.1 -P3306 -upolardbx_root -p123456 sysbench_int -N -e "SELECT id,k,c,pad FROM sbtest1;" | sed 's/\t/,/g' >data_10w.csv2.2 Execute the following command to view the file size.
ll -h wc -l data_10w.txt2.3 Execute the following command to view the data format.
head -10 data_10w.txt3. Import data through the LoadData statement
3.1 Execute the following command to log in to PolarDB-X.
Note: The LoadData statement needs to be added with the -local-infile parameter.
mysql -h127.0.0.1 -P3306 -upolardbx_root -p123456 -Ac --local-infile3.2 Execute the following command to create the target database table.
create database test_one;
use test_one;
create table test1 like sysbench_int.sbtest1;
show create table test1;3.3 Execute the following command to import data into the target database table.
-- 文件路径需要对齐
LOAD DATA LOCAL INFILE 'data_10w.txt' INTO TABLE test1;3.4 Execute the following command to view the imported data.
select min(id),max(id),count(id) from test1;
select * from test1 limit 10;SQL statement format for import and export
1. Execute exit to log out of the database first.
2. Export data through mysqldump.
2.1 Execute the following command to export data through mysqldump.
time mysqldump -h127.0.0.1 -P3306 -upolardbx_root -p123456 --default-character-set=utf8mb4 --net_buffer_length=10240 --no-tablespaces --no-create-db --no-create-info --skip-add-locks --skip-lock-tables --skip-tz-utc --set-charset --hex-blob sysbench_int sbtest1 > dump_10w.sqlv2.2 Execute the following command to view the data in sql statement format.
head -30 dump_10w.sql3. Import through the source statement.
3.1 Execute the following command to log in to PolarDB-X.
mysql -h127.0.0.1 -P3306 -upolardbx_root -p123456 -Ac --local-infile3.2 Execute the following command to create the target database table.
create database test_two; use test_two; create table sbtest1 like sysbench_int.sbtest1;3.3 Execute the following command to import data to the target table.
-- 注意文件路径
source /root/dump_10w.sql;3.4 Execute the following command to check the table data.
-- 可检查表数据
select min(id),max(id),count(id) from sbtest1;4. Import through the mysql command
4.1 Execute the following command to log in to PolarDB-X.
mysql -h127.0.0.1 -P3306 -upolardbx_root -p123456 --local-infile4.2 Execute the following command to clear the table data.
truncate table test_two.sbtest1;4.3 Execute exit to exit the database, and then execute the following import command.
mysql -h127.0.0.1 -P3306 -upolardbx_root -p123456 --default-character-set=utf8mb4 test_two < dump_10w.sql4.4 Execute the following command to log in to the database again to check the table data.
mysql -h127.0.0.1 -P3306 -upolardbx_root -p123456 --local-infile use test_two; select min(id),max(id),count(id) from sbtest1;Batch-Tool tool for import and export
1. Install the batch-tool tool (introduction to open source tools: https://github.com/ApsaraDB/galaxysql-tools/tree/main/batch-tool )
Run the following command to download batch-tool.
## github下载, 国内使用github下载不稳定时，请您使用下面的oss下载
wget https://github.com/ApsaraDB/galaxysql-tools/releases/download/batch-tool-v1.2.0/batch-tool.jar
## oss下载
wget https://labfileapp.oss-cn-hangzhou.aliyuncs.com/PolarDB-X/batch-tool.jar2. Execute the following command to install the java environment.
yum install -y java-1.8.0-openjdk.x86_643. Export through the batch-tool tool.
3.1 Execute the following command to export through the batch-tool tool.
java -jar batch-tool.jar -h127.0.0.1 -P3306 -upolardbx_root -p123456 -D sysbench_int -o export -t sbtest1 -s , -F 13.2 Execute the following command to view the file status.
head -30 sbtest1_04. Import through the batch-tool tool.
4.1 Execute the following command to log in to PolarDB-X.
mysql -h127.0.0.1 -P3306 -upolardbx_root -p123456 -Ac --local-infile4.2 Execute the following command to clear the data in the table test_one.test1
truncate table test_one.test1;4.3 Execute exit to log out of the database.
4.4 Execute the following command to import data.
java -jar batch-tool.jar -h127.0.0.1 -P3306 -upolardbx_root -p123456 -D test_one -o import -t test1 -s , -f "sbtest1_0" -maxConn 8 -minConn 44.5 Execute the following command to log in to PolarDB-X again to check the insertion status of the table.
mysql -h127.0.0.1 -P3306 -upolardbx_root -p123456 -Ac --local-infile
use test_one;
-- 检查表数据
select min(id),max(id),count(id) from test1;Click to try cloud products for free now to start the practical journey on the cloud!
This article is the original content of Alibaba Cloud and may not be reproduced without permission.