Please indicate the source www.cnblogs.com/hsxian !
Overview
This is a series of read hbase for .net core (c #) tutorial. I could not find c # native hbase client, repeatedly looking under a relatively compromise - by thrift cross-platform calls. This client efficiency compared to native java is still a certain gap, can meet their urgent needs. If an expert knows more efficient way, please inform hope.
This series is divided into three parts:
- habse relevant preparatory work
- thrift produced under c # client
- Using c # for hbase client
First, enter the first part: habse relevant preparatory work
Explanation
To make it easier to use personal hbase, reduce deployment pressure hbase, we have gone through in the course of running hbase docker cluster. But only as a personal test, specific business production environment to build your physical machine cluster hbase. Additional, also installed on the cluster scale, Hadoop, zookeep, spark and so on. For deletion, change dockerfile and its associated configuration file itself. In addition, this article assumes you already know docker, linux, hbase relevant knowledge.
centos-ssh base image ready
The base image based on centos, which installed the ssh service for free dense login preparation. For simplicity, in which passwords are all weak password, please pay attention to safety production work! ! ! dockerfile can be found on github, you can run directly mirrored building. From the readme find build information.
Big Data (already included hbase) Prepare the cluster mirror
The image based on the image step centos-ssh, please note that the name of the base image and the step of constructing the corresponding image name. The mirroring process is completed using a plurality of secondary scripts, we omitted the construction work most hbase. But inevitably still need some docker preparatory work, as long as the operation is familiar docker can understand the operation of their meaning. See specific readme
Start the cluster
From the above two steps, we've got a big data cluster. Node name, respectively master, slave01and slave02. Are logged into each node, performs su sparkswitching to spark user. After selecting the startup script to wait a few minutes waiting for the cluster starts. Under normal circumstances, the nodes masterexecute the command line jpsto see the process start situation. Gives the following process:
jps
2224 ThriftServer
1042 NodeManager
917 ResourceManager
2470 Jps
1463 Master
55 QuorumPeerMain
248 NameNode
377 DataNode
604 SecondaryNameNode
2078 HMasterWhich HMasteris the master process hbase, such as failure to start, try readme step 6.2 habse start command. Visit the website http: // localhost: 16010 / master -status view habse start situation. Everything can be normal in Region Servers seen from the node classification below slave01and slave02.
ThriftServerMediated process we call cross-platform. The default port is 9090.
to sum up
Although this article looks relatively brief. But most of the work lies in the readme file in two mirrored building. The test image for personal data for learning is large enough, such as the author of the image data by perspective view of the spark is useful. You can see the project StatisticalSample , if any help, remember to stars (⊙o⊙) Oh. Not long ago, .net camp also introduced a spark. Project .NET fo rApache® Spark ™ has just started, there can only be run locally, can not be submitted to the remote cluster execution. Well want it more and more.
Initial use, you may experience difficulties, please put on patience, Good luck.
After complete mirror the boot, you can browse the administration page of each module:
- YARN The default http port http: // localhost: 8088 / cluster
- Spark's default http port http: // localhost: 8900 /
- Hadoop's default http port http: // localhost: 50070 / dfshealth.html # tab-overview
- Hbase the default http port http: // localhost: 16010 / master -status
The following pages:
- fs.defaultFS default port http: // localhost: 9000 /
- http default port thrift of http: // localhost: 9090 /