Sqoop1.99.7
1. HDFSà MYSQL
1. 启动服务: sqoop2-serverstart
[root@slave2 bin]# sqoop2-server start
Setting conf dir:/opt/hadoop/packages/sqoop-1.99.7/bin/../conf
Sqoop home directory:/opt/hadoop/packages/sqoop-1.99.7
Starting the Sqoop2 server...
Sqoop2 server started.
2. 启动客户端： sqoop2-shell
[root@slave2 bin]# sqoop2-shell
Setting conf dir:/opt/hadoop/packages/sqoop-1.99.7/bin/../conf
Sqoop home directory:/opt/hadoop/packages/sqoop-1.99.7
Loading resource file .sqoop2rc
sqoop:000> set server --host slave2.server--port 12000 --webapp sqoop
Server is set successfully
===> OK
sqoop:000> set option --name verbose--value true
Verbose option was changed to true
===> OK
Resource file loaded.
Sqoop Shell: Type 'help' or '\h' for help.
之前直接写了配置文件./.sqoop2rc 启动时会读取该文件
3. 测试连接，显示版本说明连接成功
sqoop:000> show version --all
client version:
Sqoop 1.99.7 source revision435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb
Compiled by abefine on Tue Jul 19 16:08:27PDT 2016
0 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... usingbuiltin-java classes where applicable
server version:
Sqoop 1.99.7 source revision435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb
Compiled by abefine on Tue Jul 19 16:08:27PDT 2016
API versions:
[v1]
sqoop:000>
4. 创建HDFS的link对象
sqoop:000>createlink --connector hdfs-connector
Creating link for connector with namehdfs-connector
Please fill following values to create newlink object
Name: HDFS //要创建的link名称
HDFS cluster
URI: hdfs://192.168.111.71:9000 //hadoop的fs.default的值
Conf directory:/opt/hadoop/packages/hadoop-2.6.0/etc/hadoop //hadoop配置文件的目录
Additional configs::
There are currently 0 values in the map:
entry#
New link was successfully created withvalidation status OK and name HDFS
5. 创建Mysql的link的对象
sqoop:000> createlink --connector generic-jdbc-connector
Creating link for connector with namegeneric-jdbc-connector
Please fill following values to create newlink object
Name: MYSQL //要创建的link名称
Database connection
Driver class: com.mysql.jdbc.Driver //驱动
Connection String:jdbc:mysql://192.168.111.73:3306/test //数据库连接字符串
Username: root //用户名
Password: ****** //密码
Fetch Size:
Connection Properties:
There are currently 0 values in the map:
entry#
SQL Dialect
Identifier enclose: //此处是空格
New link was successfully created withvalidation status OK and name MYSQL
Identifierenclose:
指定SQL中标识符的定界符，也就是说，有的SQL标示符是一个引号：select* from "table_name"，这种定界符在MySQL中是会报错的。这个属性默认值就是双引号，所以不能使用回车，必须将之覆盖，我使用空格覆盖了这个值
查看创建好的link
sqoop:000> showlink
+-------+------------------------+---------+----------------------
| Name | Connector Name | Enabled |
+-------+------------------------+---------------------------------
| HDFS | hdfs-connector | true |
| MYSQL | generic-jdbc-connector | true |
+-------+------------------------+---------+-----------------------
6. 创建job
sqoop:000>create job --from HDFS --to MYSQL
Creating job for links with from name HDFSand to name MYSQL
Please fill following values to create newjob object
Name: HDFStoMYSQL //job名称
Input configuration
Input directory: /connect //数据来源于HDFS上面的哪个目录
Override null value:
Null value:
Incremental import
Incremental type:
0 :NONE
1 :NEW_FILES
Choose:
Last imported date:
Database target
Schema name: test //导入数据库的那个databases
Table name: people //导入哪个表
Column names:
There are currently 0 values in the list:
element#
Staging table:
Clear stage table:
Throttling resources
Extractors:
Loaders:
Classpath configuration
Extra mapper jars:
There are currently 0 values in the list:
element#
New job was successfully created withvalidation status OK and nameHDFStoMYSQL
说明：表名和表SQL语句是互斥的。如果提供了表名，则不应提供表SQL语句。如果提供表SQL语句，则不应提供表名称。
仅当提供表名时，才应提供表列名称。
7. 准备工作
在mysql中创建表
mysql> CREATE TABLE people (
-> name varchar(255),
-> sex varchar(255),
-> id int(2)
-> );
Query OK, 0 rows affected (0.16 sec)
8. 启动job
查看job
sqoop:000> show job
+----+-------------+--------------------------------+--------------------------------+---------+-------------------------
| Id | Name | From Connector | To Connector | Enabled |
+----+-------------+--------------------------------+--------------------------------+---------+--------------------------
| 8 | HDFStoMYSQL | HDFS (hdfs-connector) | MYSQL (generic-jdbc-connector) | true |
+----+-------------+--------------------------------+--------------------------------+---------+-------------------------
启动job
sqoop:000> startjob --name HDFStoMYSQL
Submission details
Job Name: HDFStoMYSQL
Server URL:http://slave2.server:12000/sqoop/
Created by: root
Creation date: 2017-04-11 07:17:33 CST
Lastly updated by: root
External ID: job_1491832257951_0006
http://master.server:8088/proxy/application_1491832257951_0006/
Target Connector schema: Schema{name= test. people ,columns=[
Text{name=name,nullable=true,type=TEXT,charSize=null},
Text{name=sex,nullable=true,type=TEXT,charSize=null},
FixedPoint{name=id,nullable=true,type=FIXED_POINT,byteSize=4,signed=true}]}
2017-04-11 07:17:33 CST: BOOTING - Progress is not available
可以在http://192.168.111.71:8088 查看进度
9. 验证
mysql> select * from people;
+----------+-------+------+-----------
| name | sex | id |
+----------+-------+------+-----------
| xiaoming | man | 1|
| xiaohong | woman | 2 |
+----------+-------+------+-----------
2 rows in set (0.13 sec)