sqoop 利用azkaban导出hive数据到mysql的错误

ERROR tool.ExportTool: Error during export:  because

Export job failed!
    at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:439)
    at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931)
    at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
    at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.

sqoop import export其实也是mapreduce ,然后问题出现在map阶段执行完,export 到mysql中

大概原因是:hive表结构和mysql表结构不一致,主要表现在

字段数量,字段类型,非空约束等比如说hive表结构中有null,而mysql却有非空约束,显然会错,

还有sqoop导出到mysql需要指定export -dir 为hdfs上的路径,确定hdfs路径是否有问题,hdfs路径下是否有多个文件,

如果sqoop被azkaban执行的话,还需要在sqoop任务中添加jdk路径,jdk路径是否正确

export JAVA_HOME=/usr/local/jdk1.8
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$JAVA_HOME/bin:$PATH

猜你喜欢

转载自blog.csdn.net/qq_38250124/article/details/84402162