hbase parameter configuration

Version: 0.94-cdh4.2.1

hbase-site.xml configuration

hbase.tmp.dir

  • The tmp directory of the local file system is generally set to the local mode, but it is better to set it, because many files will be set to the following by default.
  • Online configuration
    < property >
          < name >hbase.tmp.dir</ name >
          < value >/mnt/dfs/11/hbase/hbase-tmp</ value >
    </ property >
  • Defaults:
    ${java.io.tmpdir}/hbase-${user.name}
    Write to the system's /tmp directory

hbase.rootdir

  • All RegionServers in the HBase cluster share the directory, which is used to persist HBase data. Generally, the file directory of hdfs is set, such as hdfs://namenode. example.org:9000/hbase
  • Online configuration
    < property >
          < name >hbase.rootdir</ name >
          < value >hdfs://mycluster/hbase</ value >
    </ property >
  • Defaults:
    ${hbase.tmp.dir}/hbase

hbase.cluster.distributed

  • Cluster mode, distributed or stand-alone mode, if set to false, the HBase process and the Zookeeper process are in the same JVM process.
  • Online configuration is true
  • Default: false

hbase.zookeeper.quorum

  • The URL configuration of the zookeeper cluster, multiple hosts are separated by commas (,)
  • Online configuration
    < property >
        < name >hbase.zookeeper.quorum</ name >     < value >inspurXXX.xxx.xxx.org,inspurXXX.xxx.xxx.org,inspurXXX.xxx.xxx.org,inspurXXX.xxx.xxx.org,inspurXXX.xxx.xxx.org</ value >
    </ property >
  • Default: localhost

hbase.zookeeper.property.dataDir

  • Configuration in ZooKeeper's zoo.conf. where snapshots are stored
  • Online configuration: /home/hadoop/zookeeperData
  • Default: ${hbase.tmp.dir}/zookeeper

zookeeper.session.timeout

  • Client and zk connection timeout
  • Online configuration: 1200000 (20min)
  • Default: 180000 (3min)

hbase.zookeeper.property.tickTime

  • The time interval between the client and zk sending heartbeats
  • Online configuration: 6000 (6s)
  • Default: 6000

hbase.security.authentication

  • HBase cluster security authentication mechanism, the current version only supports kerberos security authentication.
  • Online configuration: kerberos
  • Default: empty

hbase.security.authorization

  • Whether HBase has enabled the security authorization mechanism
  • Online configuration: true
  • Default: false

hbase.regionserver.kerberos.principal

  • regionserver的kerberos认证的主体名称(由三部分组成:服务或用户名称、实例名称以及域名)
  • 线上配置:hbase/[email protected]xxx.xxx.COM
  • 默认:无

hbase.regionserver.keytab.file

  • regionserver keytab文件路径
  • 线上配置:/home/hadoop/etc/conf/hbase.keytab
  • 默认值:无

hbase.master.kerberos.principal

  • master的kerberos认证的主体名称(由三部分组成:服务或用户名称、实例名称以及域名)
  • 线上配置:hbase/[email protected]xxx.xxx.COM
  • 默认:无

hbase.master.keytab.file

  • master keytab文件路径
  • 线上配置:/home/hadoop/etc/conf/hbase.keytab
  • 默认值:无

hbase.regionserver.handler.count

  • regionserver处理IO请求的线程数
  • 线上配置:50
  • 默认配置:10

hbase.regionserver.global.memstore.upperLimit

  • RegionServer进程block进行flush触发条件:该节点上所有region的memstore之和达到upperLimit*heapsize
  • 线上配置:0.45
  • 默认配置:0.4

hbase.regionserver.global.memstore.lowerLimit

  • RegionServer进程触发flush的一个条件:该节点上所有region的memstore之和达到lowerLimit*heapsize
  • 线上配置:0.4
  • 默认配置:0.35

hbase.client.write.buffer

  • 客户端写buffer,设置autoFlush为false时,当客户端写满buffer才flush
  • 线上配置:8388608(8M)
  • 默认配置:2097152(2M)

hbase.hregion.max.filesize

  • 单个ColumnFamily的region大小,若按照ConstantSizeRegionSplitPolicy策略,超过设置的该值则自动split
  • 线上配置:107374182400(100G)
  • 默认配置:21474836480(20G)

hbase.hregion.memstore.block.multiplier

  • 超过memstore大小的倍数达到该值则block所有写入请求,自我保护
  • 线上配置:8(内存够大可以适当调大一些,出现这种情况需要客户端做调整)
  • 默认配置:2

hbase.hregion.memstore.flush.size

  • memstore大小,当达到该值则会flush到外存设备
  • 线上配置:104857600(100M)
  • 默认值: 134217728(128M)

hbase.hregion.memstore.mslab.enabled

  • 是否开启mslab方案,减少因内存碎片导致的Full GC,提高整体性能
  • 线上配置:true
  • 默认配置: true

hbase.regionserver.maxlogs

  • regionserver的hlog数量
  • 线上配置:128
  • 默认配置:32

hbase.regionserver.hlog.blocksize

  • hlog大小上限,达到该值则block,进行roll掉
  • 线上配置:536870912(512M)
  • 默认配置:hdfs配置的block大小

hbase.hstore.compaction.min

  • 进入minor compact队列的storefiles最小个数
  • 线上配置:10
  • 默认配置:3

hbase.hstore.compaction.max

  • 单次minor compact最多的文件个数
  • 线上配置:30
  • 默认配置:10

hbase.hstore.blockingStoreFiles

  • 当某一个region的storefile个数达到该值则block写入,等待compact
  • 线上配置:100(生产环境可以设置得很大)
  • 默认配置: 7

hbase.hstore.blockingWaitTime

  • block的等待时间
  • 线上配置:90000(90s)
  • 默认配置:90000(90s)

hbase.hregion.majorcompaction

  • 触发major compact的周期
  • 线上配置:0(关掉major compact)
  • 默认配置:86400000(1d)

hbase.regionserver.thread.compaction.large

  • large compact线程池的线程个数
  • 线上配置:5
  • 默认配置:1

hbase.regionserver.thread.compaction.small

  • small compact线程池的线程个数
  • 线上配置:5
  • 默认配置:1

hbase.regionserver.thread.compaction.throttle

  • compact(major和minor)请求进入large和small compact线程池的临界点
  • 线上配置:10737418240(10G)
  • 默认配置:2 * this.minFilesToCompact * this.region.memstoreFlushSize

hbase.hstore.compaction.max.size

  • minor compact队列中storefile文件最大size
  • 线上配置:21474836480(20G)
  • 默认配置:Long.MAX_VALUE

hbase.rpc.timeout

  • RPC请求timeout时间
  • 线上配置:300000(5min)
  • 默认配置:60000(10s)

hbase.regionserver.region.split.policy

  • split操作默认的策略
  • 线上配置: org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy(采取老的策略,自己控制split)
  • Default configuration: org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy (under the premise that the region does not reach maxFileSize, if the fileSize reaches regionCount * regionCount * flushSize, the split operation is performed)

hbase.regionserver.regionSplitLimit

  • Maximum number of regions on a single RegionServer
  • Online configuration: 150
  • Default configuration: 2147483647

Guess you like

Origin http://10.200.1.11:23101/article/api/json?id=327073435&siteId=291194637