版权声明：转载请注明出处 https://blog.csdn.net/ZT7524/article/details/86611632
最近在学习spark，在Spark SQL这一块遇到如题所示的一个小错，在“Stack Overflow“上找到了类似的解决方法，写下了做个小记。
测试环境：Ubuntu 16.04；Spark2.4
错误1 描述
input = hiveCtx.jsonFile(inputFile)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-11-431897e98712> in <module>()
----> 1 input = hiveCtx.jsonFile(inputFile)
AttributeError: 'HiveContext' object has no attribute 'jsonFile'
错误分析及解决
在较新的Spark版本中如上所示的方法已经被read().json()代替，而一些书籍上还没更新过来。按照最近的使用方法即可解决。
In [14]: input = hiveCtx.read.json(inputFile)
2019-01-23 15:24:30 WARN Utils:66 - Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.debug.maxToStringFields' in SparkEnv.conf.
错误2 描述
In [22]: topTweetText = topTweets.map(lambda row : row.text)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-22-9a3f5d661d19> in <module>()
----> 1 topTweetText = topTweets.map(lambda row : row.text)
/home/spark/python/pyspark/sql/dataframe.pyc in __getattr__(self, name)
1298 if name not in self.columns:
1299 raise AttributeError(
-> 1300 "'%s' object has no attribute '%s'" % (self.__class__.__name__, name))
1301 jc = self._jdf.apply(name)
1302 return Column(jc)
AttributeError: 'DataFrame' object has no attribute 'map'
报错解决
In [23]: topTweetText = topTweets.rdd.map(lambda row : row.text)
In [24]: topTweetText.collect()
Out[24]: [u'Adventures With Coffee, Code, and Writing.']