Sqoop从Mysql导入数据到Hive
- 连接bigdatacar数据库展示数据库中所有的表
./sqoop list-tables --connect jdbc:mysql://192.168.1.38:3306/bigdatacar --username root --password 1234
- 将关系型数据的表结构导入到hive中
./sqoop import "-Dorg.apache.sqoop.splitter.allow_text_splitter=true" --connect jdbc:mysql://192.168.1.38:3306/bigdatacar --username root --password 1234 --table t_user --hive-import --hive-table hive_sqoop_user
- 需要修改环境变量
vi /etc/profile
export HADOOP_HOME=/version/hadoop/hadoop-3.1.3
export HIVE_HOME=/version/hive/apache-hive-3.1.2-bin
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HBASE_HOME/bin:$HIVE_HOME/bin
export HIVE_CONF_DIR=$HIVE_HOME/conf
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HIVE_HOME/lib/*
- 导入成功结果如下
- Hive查看导入进来的数据
Time taken: 0.176 seconds, Fetched: 2 row(s)
hive> select * from hive_sqoop_user;
OK
0 admin 21232f297a57a5a743894a0e4a801fc3 1 admin
1 test 21232f297a57a5a743894a0e4a801fc3 2 test
10 admin1 21232f297a57a5a743894a0e4a801fc3 1 admin1
11 test1 21232f297a57a5a743894a0e4a801fc3 2 test
12 test1 1213 2 test
2 admin1 21232f297a57a5a743894a0e4a801fc3 1 admin1
3 test1 21232f297a57a5a743894a0e4a801fc3 2 test
4 admin1 21232f297a57a5a743894a0e4a801fc3 1 admin1
5 test1 21232f297a57a5a743894a0e4a801fc3 2 test
6 admin1 21232f297a57a5a743894a0e4a801fc3 1 admin1
7 test1 21232f297a57a5a743894a0e4a801fc3 2 test
8 admin1 21232f297a57a5a743894a0e4a801fc3 1 admin1
9 test1 21232f297a57a5a743894a0e4a801fc3 2 test
Time taken: 1.064 seconds, Fetched: 13 row(s
e taken: 1.064 seconds, Fetched: 13 row(s