环境
windows 10
python 3.6
hive 3.1.0
hive部署在虚拟机上
Python依赖包
- thrift
- thriftpy
- thrift_sasl
- pure_sasl
- impyla
- bitarray
如果无法使用pip install安装上述包的话，可以在https://www.lfd.uci.edu/~gohlke/pythonlibs/网页下载相应的whl文件，使用python install XXX.whl命令进行安装
开启服务
- 开启hadoop集群 start-all.sh
- 开启hive server2并在后台运行
hive --service metastore &
hive --service hiveserver2 &
连接测试
from impala.dbapi import connect
conn = connect(host='192.168.33.101', port=10000, database='***', user='***', password='', auth_mechanism='NOSASL')
cur = conn.cursor()
auth_mechanism 需要与hive-site.xml文件中hive.server2.authentication字段value值相对应
报错
1
ThriftParserError: ThriftPy does not support generating module with path in protocol ‘d’
修改site-packages\thriftpy\parser\parser.py文件
修改前：
if url_scheme == '':
with open(path) as fh:
data = fh.read()
elif url_scheme in ('http', 'https'):
data = urlopen(path).read()
修改后：
if url_scheme == '':
with open(path) as fh:
data = fh.read()
elif url_scheme in ('c', 'd','e','f''):
with open(path) as fh:
data = fh.read()
elif url_scheme in ('http', 'https'):
data = urlopen(path).read()
2
Traceback (most recent call last):
File “”, line 1, in
File “D:\Coder\anaconda\lib\site-packages\impala\hiveserver2.py”, line 125, in cursor
session = self.service.open_session(user, configuration)
File “D:\Coder\anaconda\lib\site-packages\impala\hiveserver2.py”, line 995, in open_session
resp = self._rpc(‘OpenSession’, req)
File “D:\Coder\anaconda\lib\site-packages\impala\hiveserver2.py”, line 925, in _rpc
err_if_rpc_not_ok(response)
File “D:\Coder\anaconda\lib\site-packages\impala\hiveserver2.py”, line 704, in err_if_rpc_not_ok
raise HiveServer2Error(resp.status.errorMessage)
impala.error.HiveServer2Error: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate JoJo
参考stackoverflow，修改core-site.xml文件，将hive.server2.enable.doAs字段置为false问题解决