sparkSql连接hive,找不到Hive数据库或Hive表

20/09/27 18:32:57 WARN ObjectStore: Failed to get database spark, returning NoSuchObjectException
Exception in thread “main” org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database ‘spark’ not found;

项目场景:

spark整合Hive,idea本地用spark读取Hive中的数据

问题描述:

在hive客户端查看数据库spark是存在的但是用spark连接Hive查不到数据库spark
异常如下
20/09/27 18:32:57 WARN ObjectStore: Failed to get database spark, returning NoSuchObjectException
Exception in thread “main” org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database ‘spark’ not found;

hive客户端查看数据库

hive> show databases;
OK
default
spark
Time taken: 0.11 seconds, Fetched: 3 row(s)

原因分析:

spark没找到Hive的元数据


解决方案:

加上元数据配置,hive.metastore.uris

scala中整合Hive
hdfs-site.xml中加入配置

    <property>
        <name>hive.metastore.uris</name>
        <value>thrift://node003:9083</value>
    </property>

java中spark整合Hive
spark加入配置属性config(“hive.metastore.uris”,“thrift://node003:9083”)

SparkSession  spark = SparkSession.builder().appName("随便写").config("hive.metastore.uris","thrift://node003:9083").enableHiveSupport().getOrCreate();

猜你喜欢

转载自blog.csdn.net/weixin_43614067/article/details/108833075