Use the Sqoop tool to import records from MySQL database to HDFS, import records from MySQL database to Hive, and export data from HDFS to MySQL database

1. Import records from MySQL database to HDFS

1. Mysqls database to create a database sqoop01

mysql> create database sqoop01;

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| hive               |
| mysql              |
| sqoop01            |
| test               |
+--------------------+
5 rows in set (0.00 sec)

2. Create a table

mysql> use sqoop01;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed

create table t_user(id int, name varchar(20), age int);
insert into t_user values(1, 'David', 21)
insert into t_user values(2, 'Tom', 20)
insert into t_user values(3, 'Amy', 19)

mysql> select * from t_user;
+------+-------+------+
| id   | name  | age  |
+------+-------+------+
|    1 | David |   21 |
|    2 | Tom   |   20 |
|    3 | Amy   |   19 |
+------+-------+------+
3 rows in set (0.02 sec)

3. Execute the following commands

sqoop import --connect jdbc:mysql://hadoop/sqoop01 --username root --password root --table t_user --target-dir hdfs://hadoop:8020/sqoop1 -m 1

The execution result is shown in the figure

HDFS is shown in the figure:

2. Import records from MySQL database to Hive

Execute the following command

sqoop import --connect jdbc:mysql://hadoop/sqoop01 --username root --password root --table t_user --target-dir hdfs://hadoop:8020/sqoop2 --hive-import --hive-overwrite --create-hive-table --hive-table t_user -m 1

The execution result is shown in the figure:

3. Export data from HDFS to MySQL database

1. The table export_user under the database needs to be created in advance

create table export_user(id int, name varchar(20), age int);

2. Execute the command

sqoop export --connect "jdbc:mysql://hadoop/sqoop01" --username root --password root --table export_user -m 1 --export-dir /sqoop1

3. The result is shown in the figure