Sqoop2 Install
1. install server
download the tarball form the official website
#tar -xzvf sqoop-1.99.3-bin-hadoop200.tar.gz
Assume that the server and client will install in the same host:192.168.122.1
configure server related configuration files in dir
/path/to/sqoop-1.99.3-bin-hadoop200/server/conf
a. catalina.properties
add hadoop related jars to common.load property
b. sqoop.properties
the most import item is
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/path/to/hadoop-conf-dir
To avoid access privileges, we should copy all hadoop configuration files to another dir. The reason is that the hadoop namode and yarn resource manager damons run in the same host with sqoop server, these conf files will be locked by hadoop, which lead to access denied to sqoop server.
2. install client
I installed the sqoop server and client in the same host due to resource limilation, which I used one PC installed ubuntu 12.04 with 3 centos VM.
3. add env
add sqoop bin dir to path in ~/.bashrc and using source to make it effect immidately.
4. start server
# sqoop.sh server start
5. start client
#sqoop.sh client
or you can run sqoop script in batch mode
#sqoop.sh client /path/to/script.sqoop
Note:Commands create, update and clone are not currently supported in batch mode
so, you can't create/update/clone some connection and job in you scripts. Also, you can't
use sqoop1's commands in script,such as import.
sqoop:000> set server --host your.host.com --port 12000 --webapp sqoop
sqoop:000> show version --all
sqoop:000> show connector --all
sqoop:000> create connection --cid 1
sqoop:000> show connector --all
sqoop:000> create job --xid 1 --type import
sqoop:000> start job -j 1
sqoop:000> status job -j 1
sqoop:000> update job -j 1
sqoop:000> clone job -j 1
sqoop:000> delete job -j 1
the usage of sqoop CLI refer to the official documents.
Warning:Sqoop2 is totally different from Sqoop1. So the commands belong to sqoop1 could not be used
in Sqoop2 CLI or script.
Sqoop2 wiki:
https://cwiki.apache.org/confluence/display/SQOOP/Home
https://cwiki.apache.org/confluence/display/SQOOP/Sqoop2+Quickstart
Sqoop1 Usage: https://sqoop.apache.org/docs/1.4.2/SqoopUserGuide.html