Cloud Computing in Big Data Introductory Experiment 3 - Familiar with Commonly Used HBase Operations

Enterprise 2023-05-04 20:25:10 views: null

Cloud Computing in Big Data Introductory Experiment 3 - Familiar with Commonly Used HBase Operations

Purpose

Understand the role of HBase in the Hadoop architecture

Proficiency in using shell commands commonly used in HBase operations

Familiar with Java APIs commonly used in HBase operations

Experimental requirements

  1. Save the program, and archive itself

  2. The final program must be tested and verified to be correct

  3. According to the format of the experiment report, carefully record the experiment process and results, and answer the questions in the experiment report. The template of the experiment report can be downloaded in the information of Xuetong. The experimental report submitted by the student needs to be converted into a PDF file for submission

image

Experimental procedure

Hbase common commands

  1. Create a table in Hbase
create 'student','Sname','Ssex','Sage','Sdept','course'

image

  1. View table structure
describe 'student'

image

  1. adding data
put 'student', '95001','Sname','LiYing'
put 'student','95001','course:math','80'

image

  1. view data
get 'student','95001'

image

  1. delete data
delete 'student','95001','Sname:firstName'
deleteall 'student','95001'

image

  1. delete table
disable 'student'      #让表不可用
drop 'student'         #删除表

image

  1. Query historical data
create 'teacher',{NAME=>'username',VERSIONS=>5}
put 'teacher','91001','username','Mary'
put 'teacher','91001','username','Mary1'
put 'teacher','91001','username','Mary2'
put 'teacher','91001','username','Mary3'
put 'teacher','91001','username','Mary4'
put 'teacher','91001','username','Mary5'
get 'teacher','91001',{COLUMN=>'username',VERSIONS=>3}

image

  1. quit HBase
exit

image

programming practice

Example: create a table, insert data, view data in the table

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.util.Bytes;
 
import java.io.IOException;
public class ExampleForHBase {
    public static Configuration configuration;
    public static Connection connection;
    public static Admin admin;
    public static void main(String[] args)throws IOException{
        init();　　　　//主要操作就是为了连接到数据库hbase
        createTable("student",new String[]{"score"});　　　　//创建表，shell命令：create '表名'，'列族名1'，'列族名2'，'列族名3' ...
        insertData("student","zhangsan","score","English","69"); //shell命令： put 'student'，'张三'，'score：English'，'69'
        insertData("student","zhangsan","score","Math","86");
        insertData("student","zhangsan","score","Computer","77");
        getData("student", "zhangsan", "score","English");
        close();
    }
 
    public static void init(){
        configuration  = HBaseConfiguration.create();
        configuration.set("hbase.rootdir","hdfs://localhost:9000/hbase");
        try{
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        }catch (IOException e){
            e.printStackTrace();
        }
    }
 
    public static void close(){
        try{
            if(admin != null){
                admin.close();
            }
            if(null != connection){
                connection.close();
            }
        }catch (IOException e){
            e.printStackTrace();
        }
    }
 
    public static void createTable(String myTableName,String[] colFamily) throws IOException {
        TableName tableName = TableName.valueOf(myTableName);
        if(admin.tableExists(tableName)){
            System.out.println("talbe is exists!");
        }else {
            TableDescriptorBuilder tableDescriptor = TableDescriptorBuilder.newBuilder(tableName);
            for(String str:colFamily){
                ColumnFamilyDescriptor family = 
ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes(str)).build();
                tableDescriptor.setColumnFamily(family);
            }
            admin.createTable(tableDescriptor.build());
        } 
    }
 
    public static void insertData(String tableName,String rowKey,String colFamily,String col,String val) throws IOException { 
        Table table = connection.getTable(TableName.valueOf(tableName));
        Put put = new Put(rowKey.getBytes());
        put.addColumn(colFamily.getBytes(),col.getBytes(), val.getBytes());
        table.put(put);
        table.close(); 
    }
 
    public static void getData(String tableName,String rowKey,String colFamily, String col)throws  IOException{ 
        Table table = connection.getTable(TableName.valueOf(tableName));
        Get get = new Get(rowKey.getBytes());
        get.addColumn(colFamily.getBytes(),col.getBytes());
        Result result = table.get(get);
        System.out.println(new String(result.getValue(colFamily.getBytes(),col==null?null:col.getBytes())));
        table.close(); 
    }
}

image

Experiment 3 Familiar with common HBase operations

Program to implement the following specified functions, and use the HBase Shell commands provided by Hadoop to complete the same tasks:

  • List information about all tables in HBase, such as table names;

  • Print out all record data of the specified table on the terminal;

  • Add and delete specified column families or columns to already created tables;

  • Clear all record data of the specified table;

  • The number of rows in the statistics table.

List information about all tables in HBase, such as table names:

/**
* 同样是正常的建立 数据库连接，执行操作，然后最后关闭连接
*  重点是：HTableDescriptor hTableDescriptors[] = admin.listTables(); 获取到 表格列表，然后遍历
*/
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import java.io.IOException;
public class Test_1 {
    public static Configuration configuration;
    public static Connection connection;
    public static Admin admin;
    /**
     * 建立连接
     */
    public static void init() {
        configuration = HBaseConfiguration.create();
        configuration.set("hbase.rootdir", "hdfs://localhost:9000/hbase");
        try {
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    /**
     * 关闭连接
     */
    public static void close() {
        try {
            if (admin != null) {
                admin.close();
            }
            if (null != connection) {
                connection.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
 
    }
    /**
     * 
     * 查看已有表,通过方法listTables()
     * 
     * @throws IOException
     * 
     */
    public static void listTables() throws IOException {
        init();
        HTableDescriptor hTableDescriptors[] = admin.listTables();
        for (HTableDescriptor hTableDescriptor : hTableDescriptors) {
            System.out.println(hTableDescriptor.getNameAsString());
        }
        close();
    }
    
    public static void main(String[] args) {
        Test_1 t = new Test_1();
        try {
            System.out.println("以下为Hbase 数据库中所存的表信息");
            t.listTables();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

image

Print out all record data of the specified table on the terminal;

/**
* 同样是正常的建立 数据库连接，执行操作，然后最后关闭连接
* 重点是： 
* Table table = connection.getTable(TableName.valueOf(tableName));获取到表格对象
* Scan scan = new Scan(); ResultScanner scanner = table.getScanner(scan); 然后通过Scanner对象，获取到ResultScanner扫描结果对象，遍历输出
*/
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
import org.apache.hadoop.hbase.client.*;
import java.io.IOException;
import java.util.Scanner;
public class Test_2 {
    public static Configuration configuration;
    public static Connection connection;
    public static Admin admin;
    // 建立连接
    public static void init() {
        configuration = HBaseConfiguration.create();
        configuration.set("hbase.rootdir", "hdfs://localhost:9000/hbase");
        try {
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    // 关闭连接
    public static void close() {
        try {
            if (admin != null) {
                admin.close();
            }
            if (null != connection) {
                connection.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
 
        }
    }
    /**
     * 
     * 根据表名查找表信息
     * 
     */
    public static void getData(String tableName) throws IOException {
        init();
        Table table = connection.getTable(TableName.valueOf(tableName));
        Scan scan = new Scan();
        ResultScanner scanner = table.getScanner(scan);
        for (Result result : scanner)
        {
            showCell((result));
        }
        close();
    }
 
    /**
     * 
     * 格式化输出
     * 
     * @param result
     * 
     */
    public static void showCell(Result result) {
        Cell[] cells = result.rawCells();
        for (Cell cell : cells) {
            System.out.println("RowName(行键):" + new String(CellUtil.cloneRow(cell)) + " ");
            System.out.println("Timetamp(时间戳):" + cell.getTimestamp() + " ");
            System.out.println("column Family(列簇):" + new String(CellUtil.cloneFamily(cell)) + " ");
            System.out.println("column Name(列名):" + new String(CellUtil.cloneQualifier(cell)) + " ");
            System.out.println("value:(值)" + new String(CellUtil.cloneValue(cell)) + " ");
            System.out.println();
        }
    }
    public static void main(String[] args) throws IOException {
// TODO Auto-generated method stub
        Test_2 t = new Test_2();
        System.out.println("请输入要查看的表名");
        Scanner scan = new Scanner(System.in);
        String tableName = scan.nextLine();
        System.out.println("信息如下：");
        t.getData(tableName);
    }
}

image

Add and remove specified column families or columns to created tables:

put 'student','95003','Sname','wangjinxuan' 　　　　(添加列)
 
put 'student','95003','Sname:nickName','wang'　　　　(添加列族)
 
put 'student','95003','Sname:firstName','jinxuan'    (添加列族)
 
put的反向操作的delete：
 
delete 'student' ,’95003’,’Sname’
 
delete 'student' ,’95003’,’Sname:nickName’
 
deleteall 'student' ,’95003’   　(删除整个行记录)

image

image

image

image

/**
 * hbase只关注rowkey，column Family(列族)，并没有说在创建表的时候指定cq(列限定修饰符)有多少，这也是hbase列式存储的特点，
 *     所以在hbase API中是没有提供delete 一个列下的所有数据的
 * 
 *     同样是正常的建立 数据库连接，执行操作，然后最后关闭连接
 * 1，Table table = connection.getTable(TableName.valueOf(tableName)); 先获取到表
 * 2，插入：(① 创建Put对象，② 然后通过方法 addColumn将列、列限定符、值 放到put对象，③ 最后将put对象put到表格)
 *     Put put = new Put(rowKey.getBytes());
 *  put.addColumn(colFamily.getBytes(), col.getBytes(), val.getBytes());
 *  table.put(put);        
 * 3，删除：
 * Table table = connection.getTable(TableName.valueOf(tableName)); 同样首先获取到表
 * Delete delete = new Delete(rowKey.getBytes());    //通过传入行键，new一个删除对象    
 * //删除对象添加要被删除的列或列族                        
 * ① 删除指定列族的所有数据(此情况是列族下无列限定符时的情况)：delete.addFamily(colFamily.getBytes());        
 * ② 删除指定列的数据(此列主要说的是列限定修饰符)：delete.addColumn(colFamily.getBytes(), col.getBytes());
 * table.delete(delete); //最后就是表格delete掉 delete对象
 */
import java.io.IOException;
import java.util.Scanner;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.CellUtil;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.Delete;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.Table;
 
public class Test_3 {
    public static Configuration configuration;
    public static Connection connection;
    public static Admin admin;
 
    // 建立连接
    public static void init() {
        configuration = HBaseConfiguration.create();
        configuration.set("hbase.rootdir", "hdfs://localhost:9000/hbase");
        try {
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
 
    // 关闭连接
    public static void close() {
        try {
            if (admin != null) {
                admin.close();
            }
            if (null != connection) {
                connection.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
 
    /**
     * 向某一行的某一列插入数据
     * 
     * @param tableName 表名
     * @param rowKey    行键
     * @param colFamily 列族名
     * @param col       列名(如果其列族下没有子列，此参数可为空)
     * @param val       值
     * @throws IOException
     */
    public static void insertRow(String tableName, String rowKey, String colFamily, String col, String val)throws IOException {
        init();
        Table table = connection.getTable(TableName.valueOf(tableName));
        Put put = new Put(rowKey.getBytes());
        put.addColumn(colFamily.getBytes(), col.getBytes(), val.getBytes());
        table.put(put);
        table.close();
        close();
    }
 
    /**
     * 根据表名查找表信息
     */
    public static void getData(String tableName) throws IOException {
        init();
        Table table = connection.getTable(TableName.valueOf(tableName));
        Scan scan = new Scan();
        ResultScanner scanner = table.getScanner(scan);
        for (Result result : scanner) {
            showCell((result));
        }
        close();
    }
    /**
     * 
      * 格式化输出
     * 
     * @param result
     * 
     */
    public static void showCell(Result result) {
        Cell[] cells = result.rawCells();
        for (Cell cell : cells) {
            System.out.println("RowName(行键):" + new String(CellUtil.cloneRow(cell)) + " ");
            System.out.println("Timetamp(时间戳):" + cell.getTimestamp() + " ");
            System.out.println("column Family(列簇):" + new String(CellUtil.cloneFamily(cell)) + " ");
            System.out.println("column Name(列名):" + new String(CellUtil.cloneQualifier(cell)) + " ");
            System.out.println("value:(值)" + new String(CellUtil.cloneValue(cell)) + " ");
            System.out.println();
        }
    }
    /**
     * 
     * 删除数据
     * 
     * @param tableName 表名
     * 
     * @param rowKey    行键
     * 
     * @param colFamily 列族名
     * 
     * @param col       列名
     * 
     * @throws IOException
     * 
     */
    public static void deleteRow(String tableName, String rowKey, String colFamily, String col) throws IOException {
        init();
        Table table = connection.getTable(TableName.valueOf(tableName));
        Delete delete = new Delete(rowKey.getBytes());
        if(col == null) {
            //删除指定列族的所有数据(此情况是列族下无列限定符时的情况)
            delete.addFamily(colFamily.getBytes());
            table.delete(delete);
            table.close();
        }else {
            //删除指定列的数据(此列主要说的是列限定修饰符)
            delete.addColumn(colFamily.getBytes(), col.getBytes());
            table.delete(delete);
            table.close();
        }
        close();
    }
 
    public static void main(String[] args) {
        Test_3 t = new Test_3();
        boolean flag = true;
        while (flag){
            System.out.println("------------向已经创建好的表中添加和删除指定的列簇或列--------------------");
            System.out.println("              请输入您要进行的操作   1- 添加          2-删除                       ");
            Scanner scan = new Scanner(System.in);
            String choose1 = scan.nextLine();
            switch (choose1) {
            case "1":
                try {
                    //put 'student','95003','Sname','wangjinxuan' 　　　　(添加列)
                    //put 'student','95003','Sname:nickName','wang'　　　　(添加列族)
                    //put 'student','95003','Sname:firstName','jinxuan'  (添加列族)
//                    t.insertRow(tableName, rowKey, colFamily, col, val);
                    t.insertRow("student", "95003", "Sname",null, "wangjingxuan");
                    t.insertRow("student", "95003", "Sname", "nickName", "wang");
                    t.insertRow("student", "95003", "Sname", "firstName", "jingxuan");
                    System.out.println("插入成功：");
                    t.getData(tableName);
                } catch (IOException e) {
                    e.getMessage();
                }
                break;
            case "2":
                try {
                    System.out.println("----------------------删除前，表的原本信息如下---------------------");
                    t.getData(tableName);
                    //delete 'student' ,’95003’,’Sname’
                    //delete 'student' ,’95003’,’Sname:nickName’
//                    t.deleteRow(tableName, rowKey, colFamily, col);    
                    t.deleteRow("student", "95003", "Sname", "firstName");    
                    System.out.println("-----------------------删除成功-----------------------------\n");
                    System.out.println("---------------------删除后，表的信息如下---------------------");
                    t.getData(tableName);
                } catch (IOException e) {
                    e.getMessage();
                }
                break;
            }
            System.out.println(" 你要继续操作吗？ 是-true 否-false ");
            flag = scan.nextBoolean();
        }
        System.out.println("   程序已退出！    ");
    }
}

image

Clear all record data of the specified table:

import java.io.IOException;
import java.util.Scanner;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.CellUtil;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.Table;
import org.apache.hadoop.hbase.util.Bytes;
public class Test_4 {
    public static Configuration configuration;
    public static Connection connection;
    public static Admin admin;
    // 建立连接
    public static void init() {
        configuration = HBaseConfiguration.create();
        configuration.set("hbase.rootdir", "hdfs://localhost:9000/hbase");
        try {
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    // 关闭连接
    public static void close() {
        try {
            if (admin != null) {
                admin.close();
            }
            if (null != connection) {
                connection.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    /**
     * 
     * 清空制定的表的所有记录数据
     * 
     * @param args
     * 
     * @throws IOException
     * 
     */
    public static void clearRows(String tableName) throws IOException {
        init();
        HBaseAdmin admin1 = new HBaseAdmin(configuration);
        // 读取了之前表的表名 列簇等信息，然后再进行删除操作。
        HTableDescriptor tDescriptor = admin1.getTableDescriptor(Bytes.toBytes(tableName));
        // 总思想是先将原表结构保留下来，然后进行删除，再重新依据保存的信息重新创建表。
        TableName tablename = TableName.valueOf(tableName);
        // 删除表
        admin.disableTable(tablename);
        admin.deleteTable(tablename);
        // 重新建表
        admin.createTable(tDescriptor);
        close();
    }
    /**
     * 
     * 根据表名查找表信息
     * 
     */
    public static void getData(String tableName) throws IOException {
        init();
        Table table = connection.getTable(TableName.valueOf(tableName));
        Scan scan = new Scan();
        ResultScanner scanner = table.getScanner(scan);
        for (Result result : scanner){
            showCell((result));
        }
        close();
    }
 
    /**
     * 
     * 格式化输出
     * 
     * @param result
     * 
     */
    public static void showCell(Result result) {
        Cell[] cells = result.rawCells();
        for (Cell cell : cells) {
            System.out.println("RowName(行键):" + new String(CellUtil.cloneRow(cell)) + " ");
            System.out.println("Timetamp(时间戳):" + cell.getTimestamp() + " ");
            System.out.println("column Family(列簇):" + new String(CellUtil.cloneFamily(cell)) + " ");
            System.out.println("column Name(列名):" + new String(CellUtil.cloneQualifier(cell)) + " ");
            System.out.println("value:(值)" + new String(CellUtil.cloneValue(cell)) + " ");
            System.out.println();
        }
    }
 
    public static void main(String[] args) {
        Test_4 test_4 = new Test_4();
        Scanner scan = new Scanner(System.in);
        System.out.println("请输入要清空的表名");
        String tableName = scan.nextLine();
        try {
            System.out.println("表原来的信息：");
            test_4.getData(tableName);
            test_4.clearRows(tableName);
            System.out.println("表已清空：");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

image

Number of rows in the statistics table:

import java.io.IOException;
import java.util.Scanner;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.Table;
public class Test_5 {
    public static Configuration configuration;
    public static Connection connection;
    public static Admin admin;
    //建立连接
    public static void init() {
        configuration = HBaseConfiguration.create();
        configuration.set("hbase.rootdir", "hdfs://localhost:9000/hbase");
        try {
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
    // 关闭连接
    public static void close() {
        try {
            if (admin != null) {
                admin.close();
            }
            if (null != connection) {
                connection.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
 
    public static void countRows(String tableName) throws IOException{
        init();
        Table table = connection.getTable(TableName.valueOf(tableName));
        Scan scan = new Scan();
        ResultScanner scanner = table.getScanner(scan);
        int num = 0;
        for (Result result = scanner.next(); result != null; result = scanner.next()){
            num++;
        }
        System.out.println("行数：" + num);
        scanner.close();
        close();
    }
    public static void main(String[] args) throws IOException {
        Test_5 test_5 = new Test_5();
        Scanner scan = new Scanner(System.in);
        System.out.println("请输入要统计行数的表名");
        String tableName = scan.nextLine();
        test_5.countRows(tableName);
    }
}

image

Guess you like

Origin blog.csdn.net/m0_59161987/article/details/130356152
Recommended
Ranking
error: (-215:Assertion failed) !_img.empty() in function ‘cv::imwrite‘
Database migration between Navicat servers
Minimum number of rotation of the array: Array
balenaEtcher for mac (make a boot disk software) v1.5.67
Custom processing serialization and deserialization in jackson
Mu-en-mask system development software
Mastering Regular Expressions
Find mileage Java--
Web pages can not directly concern the public micro-channel number how to do? A key to arouse public concern number of micro-channel solutions
[CodeForces - 739B] Alyona and a tree Tree + [difference] + bipartite
Daily
More
2024-05-12(28)
2024-05-11(32)
2024-05-10(34)
2024-05-09(32)
2024-05-08(18)
2024-05-07(34)
2024-05-06(6)
2024-05-05(0)
2024-05-04(18)
2024-05-03(8)

Code World

© 2018 codetd.com