[Maven + Scala + Spark]

1. 新建Scala Project
2. 右键项目 -> Configure -> convert to Maven project
3. 修改POM (scala.version 要与Spark的编译Scala版本一致。不然又兼容性问题) 
4. JDK的版本也要和Server上的JDK版本一致。
5. mvn clean package 默认不会编译scala代码，可以去target/class目录下,需要添加maven-scala-plugin
6. mvn clean package ==> 打包
7. spark-submit --class spark.example.SparkDemo xxxxxx.jar

8. package spark.example
   
   import org.apache.spark.SparkConf
   import org.apache.spark.SparkContext
   
   object SparkDemo {
     def main(args: Array[String]): Unit = {
       if (args.length < 2) {
         System.err.println("Usage: SparkDemo <master> <input>")
         System.exit(1)
       }
   
       val Array(master, input) = args
       
       val sparkConf = new SparkConf().setMaster(master).setAppName("SparkDemo")
       
       val sparkContext = new SparkContext(sparkConf)
   
       val rdd = sparkContext.textFile(input)
   
       println(rdd.count())
   
       sparkContext.stop()
     }
   }

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.harry</groupId>
	<artifactId>hello-spark-scala</artifactId>
	<version>0.0.1-SNAPSHOT</version>

	<properties>
		<!-- <scala.version>2.10.5</scala.version> -->
		<scala.version>2.11.7</scala.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.scala-lang</groupId>
			<artifactId>scala-library</artifactId>
			<version>${scala.version}</version>
			<scope>compile</scope>
		</dependency>

		<dependency>
			<groupId>org.scala-lang</groupId>
			<artifactId>scala-compiler</artifactId>
			<version>${scala.version}</version>
			<scope>compile</scope>
		</dependency>

		<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11 -->
		<dependency>
			<groupId>org.apache.spark</groupId>
			<artifactId>spark-core_2.11</artifactId>
			<version>1.6.0</version>
		</dependency>
	</dependencies>
	<build>
		<!-- <resources> <resource> <directory>${basedir}/src/main/scala</directory> 
			</resource> <resource> <directory>${basedir}/src/main/java</directory> </resource> 
			</resources> -->
		<plugins>
			<plugin>
				<groupId>org.scala-tools</groupId>
				<artifactId>maven-scala-plugin</artifactId>
				<version>2.15.2</version>
				<executions>
					<execution>
						<goals>
							<goal>compile</goal>
							<goal>testCompile</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
			<plugin>
				<artifactId>maven-compiler-plugin</artifactId>
				<version>3.5.1</version>
				<configuration>
					<source>1.7</source>
					<target>1.7</target>
				</configuration>
			</plugin>
		</plugins>
	</build>
</project>

注意：

1. Exception in thread "main" Java.lang.NoSuchMethodError: scala.collection.immutable.HashSet$.empty()Lscala/collection/immutable/HashSet;
   • Scala的版本不兼容(code编译的版本与Spark编译的scala版本不兼容)
   • 执行spark-shell ==> 可以看到spark的scala版本
2. Unsupported major.minor version 52.0
   • JDK版本不兼容