Three ways to load HBase coprocessor

This article mainly lists three ways for HBase coprocessor loading: Shell loading (dynamic), API loading (dynamic), and configuration file loading (static). The static loading method needs to restart HBase.

We assume that we already have a ready-made coprocessor Jar package that needs to be loaded: HelloCoprocessor-0.0.1.jar .

Three ways of coprocessor loading

Shell loading

1. Upload HDFS

Upload the packaged HelloCoprocessor-0.0.1.jar to the server, and then put it on HDFS.

# 切换hadoop用户，创建演示目录
$ hdfs dfs -mkdir /usr/hbase/coprocessor
$ hdfs dfs -put HelloCoprocessor-0.0.1.jar /usr/hbase/coprocessor

2. Shell loading coprocessor

We assume that the package name of its coprocessor class is:
org.myname.hbase.Coprocessor.RegionObserverExample
Shell operation is as follows:

hbase> disable 'mytable' # 禁用表，可选
hbase> alter 'mytable', METHOD => 'table_att', 'coprocessor' => '/usr/hbase/coprocessor/HelloCoprocessor-0.0.1.jar'|org.myname.hbase.Coprocessor.RegionObserverExample|1001|arg1=1,arg2=2'
hbase> enable 'mytable' # 禁用表后启用表，可选

The structure is as follows:

hbase> alter 'mytable', METHOD => 'table_att', 'coprocessor' => ①|②|③|④'  

Explain the above parameters:
①: Coprocessor Jar package path, to ensure that all RegionServer can be read. It can also be a local path, but it is recommended to be placed on HDFS.
②: The complete class name of the coprocessor.
③: Coprocessor priority, integer representation. Can be empty.
④: The parameter passed to the coprocessor can be empty.

Note: There should be no spaces between the parameters.

3. Verify that the coprocessor is loaded

hbase> desc 'mytable'

Table mytable is ENABLE
mytable, {TABLE_ATTRIBUTES => {coprocessor$1 => '/usr/hbase/coprocessor/HelloCoprocessor-0.0.1.jar'|org.myname.hbase.Coprocessor.RegionObserverExample|1001|arg1=1,arg2=2'}}

This verification does not guarantee that the coprocessor can work properly: the shell command neither guarantees that a jar file exists in a specific location, nor verifies whether a given class is actually included in the jar file.

4. Uninstall the coprocessor

hbase> disable 'mytable' # 禁用表，可选
hbase> alter 'mytable', METHOD => 'table_att_unset', NAME => 'coprocessor$1'
hbase> enable 'mytable' # 禁用表后启用表，可选

Pit: Without modifying HBase, if you modify the coprocessor code and reload the coprocessor, rename the coprocessor Jar package, otherwise it will not take effect. (Because the current JVM references the existing coprocessor, you must restart the RegionServer to restart the JVM in order to replace it.)

API loading

The specific method is to call the addCoprocessor method of HtableDescriptor. This method has two overloaded methods:

• addCoprocessor(String className)
• The
  second overload method of addCoprocessor (className, jarPath, priority, kvs) provides the parameters required in the above shell ①②③④, method one requires the user to manually distribute the jar package to the lib directory of each RegionServer.

The sample code is as follows:

TableName tableName = TableName.valueOf("mytable");
Path path = new Path("hdfs://<namenode>:<port>/usr/hbase/coprocessor/HelloCoprocessor-0.0.1.jar");
Configuration conf = HBaseConfiguration.create();
Connection connection = ConnectionFactory.createConnection(conf);
Admin admin = connection.getAdmin();
admin.disableTable(tableName);
HTableDescriptor hTableDescriptor = new HTableDescriptor(tableName);
HColumnDescriptor columnFamily1 = new HColumnDescriptor("f1");
columnFamily1.setMaxVersions(1);
hTableDescriptor.addFamily(columnFamily1);
HColumnDescriptor columnFamily2 = new HColumnDescriptor("f2");
columnFamily2.setMaxVersions(3);
hTableDescriptor.addFamily(columnFamily2);
hTableDescriptor.addCoprocessor('org.myname.hbase.Coprocessor.RegionObserverExample', path,
Coprocessor.PRIORITY_USER, null);
admin.modifyTable(tableName, hTableDescriptor);
admin.enableTable(tableName);

Reload the table definition without setting the coprocessor value using the addCoprocessor () method. This will delete any coprocessors attached to the table.

Configuration file loading

1. Modify the configuration file: hbase-site.xml , the configuration items are as follows:

1.1 RegionObservers/Endpoints

<property>
    <name>hbase.coprocessor.region.classes</name>
    <value>org.myname.hbase.Coprocessor.RegionObserverExample</value> 
</property> 

1.2 WALObservers

<property>
    <name>hbase.coprocessor.wal.classes</name>
    <value>org.myname.hbase.Coprocessor.RegionObserverExample</value> 
</property> 

1.3 MasterObservers

<property>
    <name>hbase.coprocessor.master.classes</name>
    <value>org.myname.hbase.Coprocessor.RegionObserverExample</value> 
</property> 

If you want to configure multiple coprocessors at the same time, you can separate the class names of multiple coprocessors with commas.

2. Add Jar package

Put your code on HBase's classpath. A simple method is to put the jar (including the code and all dependencies) into the installation directory lib / of HBase.

3. Restart HBase to take effect.

4. Static uninstall

• Remove the coprocessor from hbase-site.xml Elements, including child elements.
• Restart HBase.
• Delete the coprocessor JAR file from the classpath or HBase's lib / directory. (Optional)

Please indicate the source! Welcome to pay attention to my WeChat public account [HBase working notes]