Create a table directly with Phoenix (recommended, build a table directly through Phoenix, you cannot use bulkload, and subsequent backup and restoration are inconvenient)
CREATE TABLE NFT:T_COLLECTION_TEST (
a_key VARCHAR PRIMARY KEY, a_col VARCHAR) SALT_BUCKETS = 20;
SALT_BUCKETS pre-partition field, in order to solve the hotspot region problem; the range is 1-256.
Tables that use pre-partitioning cannot use hbase api to insert data, but use Phoenix API. SALT_BUCKETS can mainly improve concurrent read and write performance.
The Hbase method (not recommended) is suitable for writing and modifying through the Hbase API, but this method can be used for stock data.
| first step: Hbase table creation create 'table name', {NAME=>'column family', VERSION=>maximum number of versions (generally set to 1), [MIN_VERSIONS=>minimum number of versions], [TTL=Set version survival time in seconds] ,BLOOMFILTER='ROW'or'ROWCOL'} hbase(main):>create 't1',{NAME =>'f1', COMPRESSION => 'SNAPPY', VERSIONS => 1, BLOOMFILTER => 'ROW'},{SPLITS => ['00e1fb4fb0674d24e5a6579f8fe32060','1d148c39d210c886dc02e8468f89a539']} //It also supports compression formats such as lzo and snappy. splits are split points for pre-partitioning. The number of partitions is set according to the data size (row/column). The collection of split points needs to be considered as rowkey data Step 2: Phoenix view, which can only be queried but not modified CREATE view "NFT:T_COLLECTION_TEST"(
CREATE table "NFT:T_COLLECTION_TEST"( |
Tips View partitions: scan 'hbase:meta',{FILTER=>"PrefixFilter('table_name')"}
Index: (It is recommended to build an index before writing data)
The more difficult areas are local indexes and global indexes:
Simple definition: Local indexes are preferred for those with small amount of data and frequent updates. Global indexes are preferred for those with a large amount of data and low update frequency;
The difference and comparison between global index and local index To put it
bluntly: the global index is a table, which is suitable for the scenario of heavy reading and light writing, and a separate table will be written. The
local index is a column family, which is suitable for the scenario of heavy writing and light reading.
1. Index data
The global index stores the index data in a table alone, which ensures the security of the original data. The intrusive
local index writes the data into the original data, which is highly intrusive. The data volume of the original table = original data + index data, so that Raw data is bigger
2. Performance
The global index needs to write an extra piece of data, and the writing pressure is a little higher, but the reading speed is very fast, and the index also needs to be partitioned. The table mapped with HBAE needs to manually specify the number of partitions.
Local index only needs to write one index data, which saves a lot of space, but there is an extra step to find data through rowkey, the writing speed is very fast, and the reading speed is not as fast as directly fetching its own column family data.
| Create an index online: CREATED INDEX IDX_SLUG_RELATION_DISTINCT ON NFT.T_COLLECTION_SLUG_RELATION_INFO("info"."CHAIN_ID","info"."COLLECTION_ADDRESS","info"."SLUG","info"."EXTERNAL_SLUG") INCLUDE("info"."CHAIN_ID","info"."COLLECTION_ADDRESS"); -- 可选 Specify the number of partitions for the index Create indexes offline and asynchronously: 1) create index IDX_T_COLLECTION_SLUG_RELATION_INFO_TEST on NFT.T_COLLECTION_SLUG_RELATION_INFO_TEST(col1, col2) async |
The index field requires the query field of the where condition to be written in on, and the query result field can be written in INCLUDE.
View index status
| SELECT |
遇到失效的索引直接重建
ALTER INDEX IF EXISTS "IDX_SLUG_RELATION_DISTINCT" on "NFT.T_COLLECTION_SLUG_RELATION_INFO" REBUILD;
序列：
| CREATE SEQUENCE use_users_sequence CACHE 1000; --创建序列 SELECT sequence_schema, sequence_name, start_with, increment_by, cache_size FROM SYSTEM."SEQUENCE"; --查询所有序列 select next value for use_users_sequence; -- 下一个值 select current value for use_users_sequence; -- 当前序列 DROP SEQUENCE use_users_sequence; -- 删除序列 UPSERT VALUES INTO use_users(autoid, col1, col2) VALUES( NEXT VALUE FOR use_users_sequence, '11', '22'); -- 使用序列插入 |
快照：
|
hbase shell>disable 'NFT:T_COLLECTION_ITEM_INFO' enable 恢复表 2.制作快照 3.克隆快照为新的名字 4.删除快照 5.删除原来表 hbase shell> drop 'NFT:T_COLLECTION_ITEM_INFO' |
Tips：
-- 统计数据条数
hbase org.apache.hadoop.hbase.mapreduce.RowCounter 'NFT:T_COLLECTION_ITEM_INFO'
--SELECT COUNT (*) FROM NFT.T_COLLECTION_ITEM_INFO