HBase source code analysis -- create Table table

HBase code is based on 1.2.6

1. Client

The client code is as follows:

public class CreateTable {
	public static void main(String[] args) throws IOException, ServiceException {
		// new一个配置对象
		Configuration conf = HBaseConfiguration.create();
		Connection conn = ConnectionFactory.createConnection(conf);
		HBaseAdmin.checkHBaseAvailable(conf);
		Admin admin = conn.getAdmin();

		// 初始化表描述
		HTableDescriptor tableDescriptor = new HTableDescriptor(
				TableName.valueOf("student"));

		// 给表描述对象增加列族
		tableDescriptor.addFamily(new HColumnDescriptor("name"));
		tableDescriptor.addFamily(new HColumnDescriptor("age"));

		// 让admin根据tableDescriptor执行建表操作
		admin.createTable(tableDescriptor);
		System.out.println("hbase表创建成功! 表名为emp,列族有personal , professional");
	}
}

2. Server

After the client executes the code createTable(), the message will call the HMaster::createTable() function through RPC. Its overall process is as follows:

The client calls the createTable() function of HMaster through RPC. In this function,

 public long createTable() throws IOException {
    if (isStopped()) {
      throw new MasterNotRunningException();
    }

    String namespace = hTableDescriptor.getTableName().getNamespaceAsString();
    ensureNamespaceExists(namespace);

    final HRegionInfo[] newRegions =
        ModifyRegionUtils.createHRegionInfos(hTableDescriptor, splitKeys);
    checkInitialized();
    sanityCheckTableDescriptor(hTableDescriptor);

    return MasterProcedureUtil.submitProcedure(
      new MasterProcedureUtil.NonceProcedureRunnable(this, nonceGroup, nonce) {
      @Override
      protected void run() throws IOException {
        getMaster().getMasterCoprocessorHost().preCreateTable(hTableDescriptor, newRegions);

        LOG.info(getClientIdAuditPrefix() + " create " + hTableDescriptor);

        // TODO: We can handle/merge duplicate requests, and differentiate the case of
        //       TableExistsException by saying if the schema is the same or not.
        ProcedurePrepareLatch latch = ProcedurePrepareLatch.createLatch();
        submitProcedure(new CreateTableProcedure(
          procedureExecutor.getEnvironment(), hTableDescriptor, newRegions, latch));
        latch.await();

        getMaster().getMasterCoprocessorHost().postCreateTable(hTableDescriptor, newRegions);
      }

      @Override
      protected String getDescription() {
        return "CreateTableProcedure";
      }
    });
  }

Its overall flow chart is as follows:

1) The client's request will call the HMaster::CreateTable function, which will create a CreateTablePorcedure object (implementation of Procedure), and call submitPorcedure

2) In submitProcedure, this Procedure will be put into a Set set – runnables.

3) HMaster's ProcedureExecuteor will mobilize a thread, this thread executes the function execLoop(), this function will continuously scan the runnable object, if there is data, it will be taken out, and then call the function execProcedure() to process the Procedure

4) The execProcedure will eventually call the CreateTableProcedure::executeFromState() function to process.

5) When all processing is completed, execute exit. It shows that the table is created successfully.

2. The executeFromState process of CreateTableProcedure

As mentioned above, the actual action of the process of creating a table is done in executeFromState.
The flowchart of this function is as follows:

Some of the important steps are selected here for illustration.

2.1 The process of creating a Region

When creating a table, you need to create a Region first, and then deploy it to the corresponding RegionServer. These are two parts. The creation of Region is implemented in CREATE_TABLE_WRITE_FS_LAYOUT.
Its corresponding function is CreateTableProcedure::createFsLayout(), and its code is as follows:

  protected static List<HRegionInfo> createFsLayout(...) throws IOException {
    final MasterFileSystem mfs = env.getMasterServices().getMasterFileSystem();
    final Path tempdir = mfs.getTempDir();

    // 1. Create Table Descriptor
    // using a copy of descriptor, table will be created enabling first
    final Path tempTableDir = FSUtils.getTableDir(tempdir, hTableDescriptor.getTableName());
    new FSTableDescriptors(env.getMasterConfiguration()).createTableDescriptorForTableDirectory(
      tempTableDir, hTableDescriptor, false);

    // 2. Create Regions
    newRegions = hdfsRegionHandler.createHdfsRegions(env, tempdir,
      hTableDescriptor.getTableName(), newRegions);

    // 3. Move Table temp directory to the hbase root location 创建相应的HDFS目录
      ... ...
    return newRegions;
  }

As can be seen from the above comments, this process is divided into three parts: create the Descriptor of the Table, create the Region information, and finally create the corresponding HDFS directory

2.2 The process of deploying Region

After creating the Region, you need to deploy the Region on the actual RegionServer. it will call

protected static void assignRegions(final MasterProcedureEnv env,
      final TableName tableName, final List<HRegionInfo> regions)
      throws HBaseException, IOException {
    ProcedureSyncWait.waitRegionServers(env);
    
    final AssignmentManager assignmentManager = env.getMasterServices().getAssignmentManager();

    // Mark the table as Enabling 设置zookeeper中的状态信息
    assignmentManager.getTableStateManager().setTableState(tableName,
        ZooKeeperProtos.Table.State.ENABLING);

    // Trigger immediate assignment of the regions in round-robin fashion
    // 部署Region
    ModifyRegionUtils.assignRegions(assignmentManager, regions);

    // Enable table
    assignmentManager.getTableStateManager()
      .setTableState(tableName, ZooKeeperProtos.Table.State.ENABLED);
  }

Look at the implementation of assignRegions again.

The following is the call stack process in AssignRegions, where sendRegionOpen() is sent to the RegionServer for execution.

	org.apache.hadoop.hbase.master.ServerManager	ServerManager.java:sendRegionOpen 800
	org.apache.hadoop.hbase.master.AssignmentManager	AssignmentManager.java:assign 1739
	org.apache.hadoop.hbase.master.AssignmentManager	AssignmentManager.java:assign 2851
	org.apache.hadoop.hbase.master.AssignmentManager	AssignmentManager.java:assign 2830
	org.apache.hadoop.hbase.util.ModifyRegionUtils	ModifyRegionUtils.java:assignRegions 287
	org.apache.hadoop.hbase.master.procedure.CreateTableProcedure	CreateTableProcedure.java:assignRegions 452
	org.apache.hadoop.hbase.master.procedure.CreateTableProcedure	CreateTableProcedure.java:executeFromState 127
	org.apache.hadoop.hbase.master.procedure.CreateTableProcedure	CreateTableProcedure.java:executeFromState 58
	org.apache.hadoop.hbase.procedure2.StateMachineProcedure	StateMachineProcedure.java:execute 119
	org.apache.hadoop.hbase.procedure2.Procedure	Procedure.java:doExecute 498
	org.apache.hadoop.hbase.procedure2.ProcedureExecutor	ProcedureExecutor.java:execProcedure 1152
	org.apache.hadoop.hbase.procedure2.ProcedureExecutor	ProcedureExecutor.java:execLoop 947
	org.apache.hadoop.hbase.procedure2.ProcedureExecutor	ProcedureExecutor.java:execLoop 900
	org.apache.hadoop.hbase.procedure2.ProcedureExecutor	ProcedureExecutor.java:access$400 77
	org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2	ProcedureExecutor.java:run 497

Look at the process of assign in detail. There are two processes here that need attention: how the Master selects the RegionServer, and how the Region is deployed to the RegionServer.

2.2.1 Selection of RegionServer

Looking at the code of AssignmentManager::assign() you can see:

public void assign(List<HRegionInfo> regions)
      throws IOException, InterruptedException {
      ... 
  //首先找出现在属于online状态的regionserver，将这些regionserver放到list列表中
  List<ServerName> servers = serverManager.createDestinationServersList();
      ... 
  // Generate a round-robin bulk assignment plan
  //默认的情况下，是调用BaseLoadBalancer::roundRobinAssignment()函数
  Map<ServerName, List<HRegionInfo>> bulkPlan = balancer.roundRobinAssignment()(regions, servers);

  ... 
  processFavoredNodes(regions);
  assign(regions.size(), servers.size(), "round-robin=true", bulkPlan);
}

Further analysis function BaseLoadBalancer::roundRobinAssignment()

private void roundRobinAssignment(...) {

		int numServers = servers.size();
		int numRegions = regions.size();
		//设置regions数与numServers的比例，即一个Server上面有多少个regions
		int max = (int) Math.ceil((float) numRegions / numServers);
		int serverIdx = 0;

		for (int j = 0; j < numServers; j++) {
			//按server进行分配regions
			ServerName server = servers.get((j + serverIdx) % numServers);
			List<HRegionInfo> serverRegions = new ArrayList<HRegionInfo>(max);
			//需要注意的是这里的i的步进是server的个数。让region分散在各个server上面
			for (int i = regionIdx; i < numRegions; i += numServers) {
				//取出一个region
				HRegionInfo region = regions.get(i % numRegions);
					。。。 。。。 
					serverRegions.add(region);
					cluster.doAssignRegion(region, server);
				}
			}
			//将分配好的region与server的关系放到assignments的map中
			assignments.put(server, serverRegions);
			regionIdx++;
		}
	}

3. Summary

When HBase creates a table, it is divided into multiple steps, and the program is executed through the state quantities of different steps. All actions are also done in CreateTableProcedure::executeFromState().