sqoop导orale 数据到HDFS 问题汇总

环境:
数据源:oracle 11.2G
hadoop:ambari 2.6.1 hdp2.6.4
sqoop :1.4.6

出现的问题:
问题1:
ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver

解决方法:将ojdbc6.jar拷贝到sqoop/lib目录下

问题2:
ERROR manager.SqlManager: Error executing statement: java.sql.SQLException: Listener refused the connection with the following error:
ORA-12505, TNS:listener does not currently know of SID given in connect descriptor

解决方法:
将--connect jdbc:oracle:thin:@<host>:1521:orcl
改为--connect jdbc:oracle:thin:@//<host>:1521/orcl 


问题3:
ERROR tool.ImportTool: Error during import: No primary key could be found for table UPRO_LNT.T_LNT_CARD_INFO. Please specify one with --split-by or perform a sequential import with '-m 1'.

解决方法:
数据表没有主键,3种解决方法:
1.修改源表,建立主键。
2.将你的map个数设置为1(Sqoop默认是4)
3.使用--split-by,后面跟上表的最后一列名字。从而能够对数据进行分行 ,命令如下:
sqoop import --connect jdbc:oracle:thin:@//10.240.32.21:1521/clearing   --username LNT_BIGDATA  --password ClR#snowBB057  --table UPRO_LNT.T_LNT_CARD_INFO   --target-dir /source/db/cardinfo/20180507 --m 5  --columns "LCN,FCN,RN,BAL,IS_BATCH,IS_TIME,IS_TYPE,IS_VERSION,IS_AREA,DEPOSTIT,VALID_TIME,CARD_TYPE,PURSE_TYPE,CARD_MARK,TRAN_TIM,IS_MD,TRAN_NUM,IS_NUMBER,ULIMIT_BAL" --split-by FCN

问题4:
ERROR tool.ImportTool: Encountered IOException running import job: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/user/root/.staging":hdfs:hdfs:drwxr-xr-x

解决方法:
2种方法:
1.切换到hdfs用户 su hdfs
    sqoop ............
2.sudo -u hdfs sqoop .......


问题5:
 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Generating splits for a textual index column allowed only in case of "-Dorg.apache.sqoop.splitter.allow_text_splitter=true" property passed as a parameter
解决方法:
sudo -u hdfs sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true ......

猜你喜欢

转载自blog.csdn.net/qq_39160721/article/details/80223034
今日推荐