解决hive+hbase整合的时候遇到问题记录

版本:hive: 0.8.1 hbase:0.92.0 hadoop:1.0.0

错误信息类似如下:

Exception in thread "Thread-169" java.lang.RuntimeException: Error while reading from task log url 

        at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)

        at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:211) 

        at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:81) 

        at java.lang.Thread.run(Thread.java:662) 

Caused by: java.io.IOException: Server returned HTTP response code: 400 for URL: http://slaver1:50060/tasklog?taskid=attempt_201112211741_0005_m_000000_1&start=-8193 

        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436) 

        at java.net.URL.openStream(URL.java:1010) 

        at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120)

        ... 3 more 

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask 

解决过程如下:

看错误日志,url是:

http://slaver1:50060/tasklog?taskid=attempt_201112211741_0005_m_000000_1&start=-8193 

再看看HADOOP的源码:TaskLogServlet 

    String attemptIdStr = request.getParameter("attemptid");

 if (attemptIdStr == null) {

      response.sendError(HttpServletResponse.SC_BAD_REQUEST, 

                         "Argument attemptid is required");

      return;

    }

所以taskid应该是attemptid ,应该是hadoop的版本不一致导致的。

然后在浏览器里访问:http://slaver1:50060/tasklog?attemptid=attempt_201112211741_0005_m_000000_1&start=-8193 

获得真正的错误信息,class not found 导致的,然后就好办了,把对应的jar包扔到hadoop的lib .

stop-mapred.sh 

start-mapred.sh

搞定。

猜你喜欢

转载自attend.iteye.com/blog/1458712
今日推荐