Sqoop是一款开源的工具,主要用于在Hadoop(Hive)与传统的数据库(mysql、postgresql...)间进行数据的传递,可以将一个关系型数据库(例如 : MySQL ,Oracle ,Postgres等)中的数据导进到Hadoop的HDFS中,也可以将HDFS的数据导进到关系型数据库中。
前置环境:jdk,Hadoop,mysql,hbase,hive,zookeeper
安装:上传,解压(tar -zvxf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz)
/home/hadoop/sqoop-1.4.7/lib
将mysql的jar文件(mysql-connector-java-5.1.38.jar)放入其中
sudo vi /etc/profile
PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HADOOP_HOME/etc/hadoop:$MYSQL_HOME/bin:$HIVE_HOME/bin:$ZOOKEEPER
_HOME/bin:$KAFKA_HOME/bin:/home/hadoop/sqoop-1.4.7/bin:$PATH(不是引用变量不用加$)
source /etc/profile
cd /home/hadoop/sqoop-1.4.7/conf
cp sqoop-env-template.sh sqoop-env.sh
cp sqoop-site-template.xml sqoop-site.xml
vi sqoop-env.sh→export HADOOP_COMMON_HOME=/home/hadoop/hadoop-2.9.0
export HADOOP_MAPRED_HOME=/home/hadoop/hadoop-2.9.0
export HBASE_HOME=/home/hadoop/hbase-1.2.6
export HIVE_HOME=/home/hadoop/hive-2.3.3
export ZOOCFGDIR=/home/hadoop/zookeeper/conf
[hadoop@sure ~]$ sqoop
Warning: /home/hadoop/sqoop-1.4.7/bin/../../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/hadoop/sqoop-1.4.7/bin/../../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.9.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hbase-1.2.6/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Try 'sqoop help' for usage.
开启sqoop成功