Spark SQL operation Hive in IDEA

1. Add dependencies

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-hive_2.11</artifactId>
    <version>2.1.1</version>
</dependency>
<dependency>
    <groupId>org.apache.hive</groupId>
    <artifactId>hive-exec</artifactId>
    <version>1.2.1</version>
</dependency>

2. Copy the hive-site.xml file to the resources folder of the project

3. Code

• Note : builder() needs to add support for hive, enableHiveSupport()

    val conf: SparkConf = new SparkConf().setAppName(this.getClass.getSimpleName).setMaster("local[*]")
    val spark: SparkSession = SparkSession.builder()
                                          .enableHiveSupport()  //添加对hive的支持
                                          .config(conf).getOrCreate()
    //spark.sql("show tables").show()
    spark.sql("select * from dept").show()


    spark.stop()