SparkSQL hive using the read data in the table Spark On Hive, and the local client mode by tests, but in the yarn is thrown cluster model test following exception:
org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database ‘test’ not found;
Finally, find out why: when the spark job jar into yarn run above cluster node may not perform the job Hive node is located. Hive will not find the configuration file, a database error can not find the cause.
So use --files the hive-site.xml broadcast to all nodes
spark-submit --master yarn --deploy-mode cluster --files ... / hive-site.xml - ...
