Mukesh Kumar, Machine Learning Professional and Big Data Expert
关注
Mukesh Kumar, Machine Learning Professional and Big Data Expert
Proven experience in multiple domains and BigData, a right fit for any data science project.
Apache HBase 1.3.0 was released mid-January 2017 and ships with support for date-based tiered compaction and improvements in multiple areas, like write-ahead log (WAL), and a new RPC scheduler, among others. The release includes almost 1,700 resolved issues in total.
Below are some bold points on enhancement made in HBase 1.3.0:-
- The "date-based tiered compaction" support shipped in HBase 1.3.0 is beneficial for where data is infrequently deleted or updated and recent data is scanned more often than an older one.
- Records time-to-live (TTL) can be easily enforced with this new compaction strategy.
- Improved multiple WAL support in Apache HBase 1.3.0. Performance tests report 20% improvements in average latency when running on pure SATA disks, and 40% when WAL files are written to SATA-SSD disks.
- The new RPC scheduler is based on CoDel algorithm and is used to prevent long standing call queues caused by discrepancy of request rate and available throughput, bounded by IO.
- A throughput controller for disk flushers to avoid huge IO spikes.
- A better performance of Apache Phoenix, OpenTSBD, and other software projects that rely on HBase engine for data persistence and fast lookup.
- Add normalization support to hbase shell 'tools' command group for region normalizer.
- Add configuration to skip validating HFile format when bulk loading:- A new config, hbase.loadincremental.validate.hfile , is introduced - default to true When set to false, checking hfile format is skipped during bulkloading.
- Bulk Loaded HFile Replication enhances the HBase replication to support replication of bulk loaded data.
- Introduces collection and reporting of various client-perceived metrics. Metrics are exposed via JMX.
- Added 2 new pages to the region server Web UI to ease debugging and provide greater insight into the physical data layout.
- The performance of region splitting has been improved by using a thread pool to split the store files concurrently. Prior to this change, the store files were always split sequentially in a single thread, so a region with multiple store files ended up taking several seconds.
- Web UI Framable Page adds protection from clickjacking using X-Frame-Options header. This will prevent use of HBase UI in frames.
- Now flush all system tables in every 5 minute period using Memstore periodic flusher.