Troubleshooting: spark access hive library, table error org.apache.spark.sql.AnalysisException: Table or view not found

table of Contents

1. Problem background

2. Problem description

Three, problem solving

1. Spark project + external configuration file [optimal: configuration changes, no need to recompile and package]

2. Spark project (hive-site.xml is stored in the resources directory) [Secondly: if configuration changes need to update the configuration file, recompile and package]

​3, spark project (the configuration options are hard-coded in the code) [worst: after the configuration modification is involved, the code needs to be modified, and then recompile and package]

---

1. Problem background

1. Complete the table ods_data operation through hive -f
2. Then operate the libraries and tables in hive through spark sql

---

2. Problem description

Error message: org.apache.spark.sql.AnalysisException: Table or view not found

---

Three, problem solving

1. Spark project + external configuration file [optimal: configuration changes, no need to recompile and package]

2. Spark project (hive-site.xml is stored in the resources directory) [Secondly: if configuration changes need to update the configuration file, recompile and package]

3. Spark project (the configuration options are hard-coded in the code) [worst case: after the configuration modification is involved, the code needs to be modified, and then recompile and package]

---

If none of the above three methods can be solved, you can try to check: hive-site.xml configuration file

---

If the problem is not solved by the above, you need to check whether the hive version is too low or too high.

 

If you still cannot solve your problem, please leave a message at the back of the blog~ I will help solve the problem!

 

---

At the end of the article, I recommend some popular technical blog links :

1. JAVA related deep technical blog link
2. Flink related technical blog links
3. Spark core technology link
4. Design Pattern-Deepin Technology Blog Link
5. Machine learning-deep technology blog link
6. Hadoop related technical blog links
7. Super dry goods--Flink mind map, it took about 3 weeks to compile and proofread
8. Deepen the core principles of JAVA JVM to solve various online faults [with case]
9. Please talk about your understanding of volatile? --Recently, a "hardcore contest" between Xiao Lizi and the interviewer
10. Talk about RPC communication, an interview question that is often asked. Source code + notes, package understanding
11. In-depth talk about Java garbage collection mechanism [with schematic diagram and tuning method]

---

Welcome to scan the QR code below or search the official account "Big Data Senior Architect", we will push more and timely information to you, welcome to communicate!