1.java.lang.NoClassDefFoundError: org/apache/spark/Logging
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/Logging
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)I
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.spark.streaming.twitter.TwitterUtils$.createStream(TwitterUtils.scala:44)
at TwitterStreamingApp$.main(TwitterStreamingApp.scala:42)
at TwitterStreamingApp.main(TwitterStreamingApp.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:729)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.Logging
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 23 more
spark-streaming-kafka替换依赖
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka-0-8_2.11</artifactId>
<version>2.1.0</version>
</dependency>
spark-defaults.conf 里面再加上spark.yarn.historyServer.address=http://xxx:18080 否则 YARN 中点击“History”按钮是不会跳转到history-server的，看“Environment”中没有spark.yarn.historyServer.address这个属性，都不能跳转，有了都能跳转。自己测试的结论。
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2.spark is not a valid external type for schema of string
createDataFrame时候java的string类型，匹配不了StringType这个类型属性scala.
解决：统一用java
createDataFrame(rows: java.util.List[Row], schema: StructType)
No output operations registered, so nothing to execute
Illegalargumentexception(传递不合法):requirement failed: No output operations registered, so nothing
to execute
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
3.没有触发DStream需要的action
使用一下：
print()
foreachRDD()
saveAsObjectFiles()
saveAsTextFiles()
saveAsHadoopFiles()
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------