1. Unzip the installation package

Network disk resources are provided here

Link: https://pan.baidu.com/s/1QkFqVnlvuOJ_aB2bjn-OKg?pwd=ucsy Extraction code: ucsy

There are two compressed packages here. sqoop-1.4.7.tar.gz is the installation package of sqoop, and the other is some jar packages we need inside.

Unzip the installation package

tar -zxf ./sqoop-1.4.7.tar.gz -C /opt/soft

Change the name to a simple name to facilitate subsequent operations.

mv sqoop-1.4.7/ sqoop147

2. Modify the configuration file

Go to the sqoop/conf directory and copy the sqoop-env-template.sh file to this directory and name it sqoop-env.sh

 cp sqoop-env-template.sh sqoop-env.sh

Modify sqoop-env.sh

vim sqoop-env.sh

#Set path to where bin/hadoop is available
export HADOOP_COMMON_HOME=/opt/soft/hadoop313


#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME=/opt/soft/hadoop313

#set the path to where bin/hbase is available
export HBASE_HOME=/opt/soft/hbase235

#Set the path to where bin/hive is available
export HIVE_HOME=/opt/soft/hive312
export HIVE_CONF_DIR=/opt/soft/hive312/conf

#Set the path for where zookeper config dir is
export ZOOCFGDIR=/opt/soft/zk345/conf

Three copies of the jar package

(1) Complete the jar package in the sqoop147 directory

You need sqoop-1.4.7.jar in the sqoop-1.4.7.bin__hadoop-2.6.0 directory, copy a copy to the sqoop147 directory

cp /opt/install/sqoop-1.4.7.bin__hadoop-2.6.0/sqoop-1.4.7.jar /opt/soft/sqoop147/

(2) Complete the jar package in the lib directory

This command is executed in /opt/soft/sqoop147/lib

cd /opt/soft/sqoop147/lib

 cp /opt/soft/hive312/lib/hive-common-3.1.2.jar ./

This mysql driver package was copied when installing hive, so I have it directly in this directory. If the driver package is not available in this directory, you can go to the bin directory of hive to find it.

cp /opt/install/mysql-connector-java-8.0.29.jar ./

cp /opt/install/sqoop-1.4.7.bin__hadoop-2.6.0/lib/avro-1.8.1.jar ./

4. Modify environment variables

vim /etc/profile

#SQOOP
export SQOOP_HOME=/opt/soft/sqoop147
export PATH=$SQOOP_HOME/bin:$PATH

After modifying the source,

source /etc/profile

5. Check the sqoop version and test the connection

Check sqoop version

sqoop version

[root@kb46 conf]# sqoop version
Warning: /opt/soft/sqoop147/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /opt/soft/sqoop147/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
2023-09-15 11:21:14,751 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
Sqoop 1.4.7
git commit id 2328971411f57f0cb683dfb79d19d4d19d185dd8
Compiled by maugli on Thu Dec 21 15:59:58 STD 2017

View sqoop commands

sqoop help

Connect to mysql to view all MySQL databases

The IP and mysql password here should be replaced with your own.

There is another little trick here. When typing the sqoop command on the command line, \ represents the connector, \ followed by Enter and then write, which is a command.

 sqoop list-databases --connect jdbc:mysql://172.25.**.**:3306/ --username root --password password

Result example:

[root@kb46 conf]# sqoop list-databases --connect jdbc:mysql://172.25.38.169:3306/ --username root --password 100860
Warning: /opt/soft/sqoop147/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /opt/soft/sqoop147/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
2023-09-15 11:25:35,880 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
2023-09-15 11:25:35,910 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
2023-09-15 11:25:35,996 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
mysql
information_schema
performance_schema
sys
kb46

linux installation sqoop