Summary of error reporting problems in python3 operation hdfs library

Error message:

hdfs.util.HdfsError: Permission denied: user=dr.who, access=WRITE, inode="/user/hadoop/output":hadoop:supergroup:drwxr-xr-x

Solution:

Because of permission issues, it is necessary to modify the configuration file and grant permissions:
1. Grant permissions

hadoop fs -chmod 777 /user/hadoop

2. Modify the configuration file

hdfs-site.xml

<property>
<name>dfs.permissions.enabled</name>
<value>false</value>
</property>

Error message:

hdfs.util.HdfsError: Cannot create file/user/hadoop/output/result.txt. Name node is in safe mode.

Solution:

Because the Name node is in safe mode, you need to turn off safe mode

hadoop dfsadmin -safemode leave

Error message:

requests.exceptions.ConnectionError: HTTPConnectionPool(host='ubuntu', port=50075): Max retries exceeded with url: /webhdfs/v1/user/hadoop/input/file1.txt?op=OPEN&namenoderpcaddress=localhost:9000&length=20&offset=0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x11c44f670>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))

Solution:

问题原因 :出现host='16a187bf3b55', port=50075 表示客户端已经连接成功,但是无法解析域名

通过在python程序运行的机器上修改hosts文件

添加对应连接的ip 以及相关联的别名

如

119.10.1.15    16a187bf3b55  namenode