异常问题一:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/mapreduce/TableInputFormatBase
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1.doLoadClass(IsolatedClientLoader.scala:221)
at org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1.loadClass(IsolatedClientLoader.scala:210)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getInputFormatClass(HBaseStorageHandler.java:296)
at org.apache.hadoop.hive.ql.metadata.Table.getInputFormatClass(Table.java:305)
at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$getTableOption$1$$anonfun$apply$11.apply(HiveClientImpl.scala:407)
at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$getTableOption$1$$anonfun$apply$11.apply(HiveClientImpl.scala:361)
at scala.Option.map(Option.scala:146)
解决办法:Maven中引入hbase-server-1.2.0-cdh5.12.2.jar
异常二:Failed to connect to /192.186.0.36 for block, add to deadNodes and continue. java.net.NoRouteToHostException:
服务器是双网段192和10开头,而办公网段是:10.96开头的IP,本地设置的host文件也是连接的10,96的hostName映射,脚本运行时自动连接192.168的网段,导致一直无法连接到集群。
解决办法:
<property> <name>dfs.client.use.datanode.hostname</name> <value>true</value> </property>