HDP 2.5 integrates Sqoop2

foreword

HDP 2.5 only has Sqoop 1 by default, and sqoop 1 has only a client, no server, and cannot be operated through Java Api, so it is necessary to integrate Sqoop2. In fact, we only need to start the server of Sqoop2 here.

environment

• CentOS 7.x
• HDP 2.5
• JDK 1.8
• Sqoop1.99.7

Note: sqoop 1.99.7 is actually Sqoop2

configure

The configuration is very simple. After downloading sqoop2, decompress it to a path, such as: /usr/lib/
enter the decompressed bin directory, edit the sqoop.sh file:
find function sqoop_server_classpath_setthe function, and just click the environment variables in it, as follows:

function sqoop_server_classpath_set {

HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-${HADOOP_HOME}/share/hadoop/common}
HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-${HADOOP_HOME}/share/hadoop/hdfs}
HADOOP_MAPRED_HOME=${HADOOP_MAPRED_HOME:-${HADOOP_HOME}/share/hadoop/mapreduce}
HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-${HADOOP_HOME}/share/hadoop/yarn}

Comment out these environment variables and change them to the following:

function sqoop_server_classpath_set {

 HDP=/usr/hdp/2.5.0.0-1245
 HADOOP_COMMON_HOME=$HDP/hadoop
 HADOOP_HDFS_HOME=$HDP/hadoop-hdfs
 HADOOP_MAPRED_HOME=$HDP/hadoop-mapreduce
 HADOOP_YARN_HOME=$HDP/hadoop-yarn

start server

Enter the bin directory of sqoop and execute the following command to start the server:

# ./sqoop.sh server start

查看是否启动成功
# jps
20611 SqoopJettyServer

If the SqoopJettyServer process appears, it means that it has been started successfully, and then we can perform the corresponding data import and export operations through the Java API interface.