Today there are friends add friends and I discuss some of the issues, I think these issues ll be very valuable; so wanted to set up a public Q & A column in this number, to facilitate technology exchange and sharing it as column name: "Letters." In case of my limited ability to solve difficult problems, which will be forwarded to stick my hand bigwigs resources lap seek help, along with the questioner micro-channel two-dimensional code. Also welcome to actively explore solutions - in the message area
Letter: Huang Wei *
Small ape Question
If I use HBase written Spark processing files, incremental issued by date every day, if want to keep only the last 90 days of file data in HBase, there is no good way to do it? TTL will disable the operating table, the back-end query will throw an error. In addition to TTL, there are other solution?
Small ape analysis
The main crux of the problem lies in: the beginning of the construction of the table, there is no time to set the TTL column family, I thought I could set up the table after the TTL attribute data to ensure the timeliness of the data but do not want to disable the table. How to do it?
Small ape Answers
Here, a small ape gives two solutions:
Option One:
In fact, slightly higher HBase version, set the table TTL property has it online, you do not need to disable the table. If you are unsure, you can build a table online test TTL set about to try. If it does, then you can select during off-peak by timeliness HBase Shell manually modify the column family, once and for all.
hbase(main):030:0> create 'test','f1'
0 row(s) in 1.2990 seconds
=> Hbase::Table - test
hbase(main):031:0> desc 'test'
Table test is ENABLED
test
COLUMN FAMILIES DESCRIPTION
{NAME => 'f1', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS
=> '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}
1 row(s) in 0.0480 seconds
hbase(main):032:0> alter 'test',{NAME => 'f1',TTL => '86400'}
Updating all regions with the new schema...
1/1 regions updated.
Done.
0 row(s) in 1.9870 seconds
Unit: seconds (s)
Option II:
If a program does not work, by way of the API may try, real-time data is provided when inserting a single data TTL. But historical data will manually deleted.
Put put = new Put(Bytes.toBytes("row1"));
put.setTTL(86400L);
Unit: seconds (s)
Knowledge supplement
If you want to reset the TTL is 'FOREVER' how to do?
A: HBase TTL maximum value is the maximum INT 2147483647, simply set the TTL integer can be.
hbase(main):033:0> alter 'test',{NAME => 'f1',TTL => '2147483647'}
Updating all regions with the new schema...
1/1 regions updated.
Done.
0 row(s) in 1.9170 seconds
hbase(main):034:0> desc 'test'
Table test is ENABLED
test
COLUMN FAMILIES DESCRIPTION
{NAME => 'f1', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS
=> '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}
1 row(s) in 0.0140 seconds
If you have a better answer, leave a message to welcome the comments section oh ~
Please indicate the source! I welcome the focus on micro-channel public number HBase work [notes]