Introduction:
Python needs to connect to hbase through thrift connection. It seems that there is no thrift installed with hbase in the service installed by ambari. I saw that there is no thrift in the configuration name of hbase, but the cdh version has it, so I installed it myself thrift.
1. thrift installation:
1. Download thrift dependencies
yum install automake libtool flex bison pkgconfig gcc-c++ boost-devel libevent-devel zlib-devel python-devel ruby-devel openssl-devel
2. Install boost_1_53_0.tar.gz
[root@master ~]# wget http://sourceforge.net/projects/boost/files/boost/1.53.0/boost_1_53_0.tar.gz [root@master ~]# tar xvf boost_1_53_0.tar.gz [root@master ~]# cd boost_1_53_0 [root@master boost_1_53_0]# ./bootstrap.sh [root@master boost_1_53_0]# ./b2 install [root@master boost_1_53_0]# cp /usr/local/lib/libboost_unit_test_framework.a /usr/lib64/ [root@master boost_1_53_0]# make [root@master boost_1_53_0]# make install
3. Download the latest version of thrift from http://thrift.apache.org
4. Move to the default installation directory and unzip it
[root@master ~]# mv thrift-0.11.0.tar.gz /usr/local [root@master ~]# cd /usr/local [root@master local]# tar -zxvf thrift-0.11.0.tar.gz [root@master local]# mv thrift-0.11.0 thrift [root@master local]# cd thrift [root@master local]# ./configure --libdir=/usr/lib --without-java --without-python --without-c_glib [root@master local]# make [root@master local]# make install
2. Start thrift
1. Find the bin execution folder of hbase. The location of my execution is /usr/hdp/2.6.3.0-235/hbase/bin under the folder where ambari is installed by default
2. Start thrift, the default port is 9090
[root@master ~]# cd /usr/hdp/2.6.3.0-235/hbase/bin [root@master bin]# bin/hbase-daemon.sh start thrift
3. Use python to connect hbase
1. Install python dependencies
pip install thrift
pip install hbase-thrift
2. demo program
from thrift import Thrift from thrift.transport import TSocket from thrift.transport import TTransport from thrift.protocol import TBinaryProtocol from hbase import Hbase from hbase.ttypes import * transport = TSocket.TSocket('localhost', 9090) transport = TTransport.TBufferedTransport (transport) protocol = TBinaryProtocol.TBinaryProtocol(transport) client = Hbase.Client(protocol) transport.open() contents = ColumnDescriptor(name='cf:', maxVersions=1) # client.deleteTable('test') client.createTable('test', [contents]) print client.getTableNames() # insert data transport.open() row = 'row-key1' mutations = [Mutation(column="cf:a", value="1")] client.mutateRow('test', row, mutations) # get one row tableName = 'test' rowKey = 'row-key1' result = client.getRow(tableName, rowKey) print result for r in result: print 'the row is ', r.row print 'the values is ', r.columns.get('cf:a').value transport.close()