20/09/27 18:32:57 WARN ObjectStore: Failed to get database spark, returning NoSuchObjectException
Exception in thread “main” org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database ‘spark’ not found;
Project scene:
Spark integrates Hive, and idea uses spark locally to read data in Hive
Problem Description:
Check that the database spark exists on the hive client, but connect to Hive with spark cannot find the database spark
exception is as follows
20/09/27 18:32:57 WARN ObjectStore: Failed to get database spark, returning NoSuchObjectException
Exception in thread “main” org .apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database'spark' not found;
hive client view database
hive> show databases;
OK
default
spark
Time taken: 0.11 seconds, Fetched: 3 row(s)
Cause Analysis:
Spark did not find Hive metadata
solution:
Plus metadata configuration, hive.metastore.uris
Integrate Hive
hdfs-site.xml in scala and add configuration
<property>
<name>hive.metastore.uris</name>
<value>thrift://node003:9083</value>
</property>
Spark integrates Hive spark in java to add configuration property config("hive.metastore.uris", "thrift://node003:9083")
SparkSession spark = SparkSession.builder().appName("随便写").config("hive.metastore.uris","thrift://node003:9083").enableHiveSupport().getOrCreate();