python operation hbase

1 python install thrift extension

(1) Installation in linux environment

After decompressing and installing, you can install it directly. Note that the installation package can be downloaded from https://pypi.python.org/pypi/thrift/0.11.0 , and downloaded from apache. This method is not applicable for installation

tar -zxvf thrift-0.11.0.tar.gz
cd thrift-0.11.0/
python3 setup.py install

(2) Installation in windows environment

The installation package is the same as under linux. Before installation, unzip the installation package and install it on the dos interface. Enter the unzipped directory in dos and execute the following command

python setup.py install

2 Install thrift

It can be installed in linux. Download the installation package on Apache http://thrift.apache.org/download

tar -zxvf thrift-0.11.0.tar.gz
cd thrift-0.11.0/
./configure
make
make install

• g++: command not found solution

yum -y install gcc+ gcc-c++

3 Compile hbase thrift

Download the source package corresponding to hbase and execute it in the path of hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift

thrift -gen py Hbase.thrift

After execution, there will be a gen-py folder, copy the hbase folder inside to the site-packages of python3, and the site-packages will be selected according to the personal installation situation

cp -R hbase /usr/python3/lib/python3.6/site-packages

After linux is executed, copy the hbase package to the python environment of windows

4 python query hbase demo

#!/usr/bin/python3
from hbase import Hbase
from thrift.protocol import TBinaryProtocol
from thrift.transport import TSocket, TTransport

#配置thriftServer主机名，端口
transport = TSocket.TSocket('A5-302-NF8460M3-157',9090)
#设置超时
transport.setTimeout(5000)
#设置传输方式（TFramedTransport或TBufferedTransport）
trans = TTransport.TBufferedTransport(transport)
#设置传输协议
protocol = TBinaryProtocol.TBinaryProtocol(trans)
#确定客户端
client = Hbase.Client(protocol)
#打开连接
transport.open()
#获取表名
client.getTableNames()

5 Use happyhbase to operate hbase

(1) Install happyhbase extension

Before using this extension, thrift needs to be installed

• Install under windows

pip install happybase

After the installation is complete, modify site-packages\thriftpy\parser\parser.py, about 488 lines

if url_scheme == '':

change into:

if url_scheme == '' or url_scheme == 'c':  

• Installation under linux

pip3 install happybase

(2) demo example

#!/usr/bin/python3
import happybase

# 创建hbase链接
# 自动链接方式
connection = happybase.Connection('192.168.59.129')
print(connection.tables())


# 非自动链接方式
connection2 = happybase.Connection('192.168.59.129', autoconnect=False)
connection2.open()
print(connection2.tables())