sqoop installation: it can be installed on one node.
1. Upload sqoop
2. Installation and configuration
After adding sqoop to environment variable
Copy the database connection driver to $SQOOP_HOME/lib
3. Use
The first category: data in the database is imported to HDFS
sqoop import --connect jdbc:mysql://192.168.1.10:3306/itcast --username root --password 123 --table trade_detail --columns 'id, account, income, expenses'
Specify output path, specify data separator
sqoop import --connect jdbc:mysql://192.168.1.10:3306/itcast --username root --password 123 --table trade_detail --target-dir '/sqoop/td' --fields-terminated-by '\t'
Specify the number of Maps -m
sqoop import --connect jdbc:mysql://192.168.1.10:3306/itcast --username root --password 123 --table trade_detail --target-dir '/sqoop/td1' --fields-terminated-by '\t' -m 2
Add where condition, note: conditions must be enclosed in quotation marks
sqoop import --connect jdbc:mysql://192.168.1.10:3306/itcast --username root --password 123 --table trade_detail --where 'id>3' --target-dir '/sqoop/td2'
Add query statement (use \ to wrap the statement)
sqoop import --connect jdbc:mysql://192.168.1.10:3306/itcast --username root --password 123 \
--query 'SELECT * FROM trade_detail where id > 2 AND $CONDITIONS' --split-by trade_detail.id --target-dir '/sqoop/td3'
Note: If you use the --query command, you need to pay attention to the parameters after where, and the parameter AND $CONDITIONS must be added
And there is a difference between single quotes and double quotes. If double quotes are used after --query, you need to add \ before $CONDITIONS, that is, \$CONDITIONS
If you set the number of maps to 1, that is -m 1, you don't need to add --split-by ${tablename.column}, otherwise you need to add
The second category: export the data on HDFS to the database (don't forget to specify the delimiter)
sqoop export --connect jdbc:mysql://192.168.8.120:3306/itcast --username root --password 123 --export-dir '/td3' --table td_bak -m 1 --fields-terminated-by ','
4. Configure mysql remote connection
GRANT ALL PRIVILEGES ON itcast.* TO 'root'@'192.168.1.201' IDENTIFIED BY '123' WITH GRANT OPTION;
FLUSH PRIVILEGES;
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123' WITH GRANT OPTION;
FLUSH PRIVILEGES