Sqoop安装和简单的使用

版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/u014527619/article/details/87966951

下载

官网dowload http://sqoop.apache.org/

安装

解压到自己规定的目录

tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz 

配置环境变量:

#sqoop
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HIVE_HOME/lib/*:$HIVE_HOME/conf
export SQOOP_HOME=/mapbar/app/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0
export HIVE_CONF_DIR=/mapbar/app/hadoop/hive-1.1.0-cdh5.13.0/conf
export PATH=$PATH:$SQOOP_HOME/bin

修改sqoop-env.sh

export HADOOP_COMMON_HOME=/home/xuyao/下载/hadoop-2.6.5 
export HADOOP_MAPRED_HOME=/home/xuyao/下载/hadoop-2.6.5 

修改bin/configure-sqoop:注释掉HCAT_HOME、ACCUMULO_HOME、ZOOKEEPER_HOME的检查。

## Moved to be a runtime check in sqoop.
#if [ ! -d "${HCAT_HOME}" ]; then
#  echo "Warning: $HCAT_HOME does not exist! HCatalog jobs will fail."
#  echo 'Please set $HCAT_HOME to the root of your HCatalog installation.'
#fi

#if [ ! -d "${ACCUMULO_HOME}" ]; then
#  echo "Warning: $ACCUMULO_HOME does not exist! Accumulo imports will fail."
#  echo 'Please set $ACCUMULO_HOME to the root of your Accumulo installation.'
#fi
#if [ ! -d "${ZOOKEEPER_HOME}" ]; then
#  echo "Warning: $ZOOKEEPER_HOME does not exist! Accumulo imports will fail."
#  echo 'Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.'
#fi

简单适用

hive导入到mysql

mysql的表需要提前建立,字段跟hive 保持一致。

sqoop export \
--connect jdbc:mysql://static-node0:3307/fuwj-test  \
--username fuwj \
--password 123456 \
--table b_ctp_behavior_trip \
--export-dir /user/hive/cp.db/b_ctp_behavior_trip/day=20190217 \
--input-fields-terminated-by '|'

猜你喜欢

转载自blog.csdn.net/u014527619/article/details/87966951