HBase X-Pack is based on the deep expansion of Apache HBase, Phoenix, and Spark, and integrates Solr retrieval technologies to support one-stop storage, retrieval and analysis of massive data. The integration of cloud kafka + cloud HBase X-Pack can build an integrated data processing platform that supports scenarios such as risk control, recommendation, retrieval, portrait, social networking, Internet of Things, time and space, form query, offline data warehouse, etc., helping enterprise data intelligence.
The following is a big data middle-office architecture widely used in the industry, among which HBase&Spark chooses cloud HBase X-Pack product related user documents:
https://help.aliyun.com/document_detail/93899.html
Purchase page:
https://hbase.console.aliyun.com/hbase/cn-shenzhen/clusters
Message inflow: Flume, Logstash, or binlog of online library flows into the message middleware Kafka
Real-time computing: Real-time consumption of Kafka messages through X-Pack Spark Streaming, and write them to the cloud HBase to provide online queries to the outside world
Real-time storage and retrieval: Cloud HBase fusion solr and phoenix sql layer can provide massive real-time storage, as well as online query and retrieval
Batch processing, data warehouse and algorithm: Online storage of HBase data can be automatically archived to X-Pack Spark data warehouse. The full amount of data is deposited in the Spark data warehouse (HiveMeta), and complex calculations such as batch processing and algorithm analysis are performed, and the results are returned to the online library for external query.
The practical operation of this set of programs can refer to the article:
https://help.aliyun.com/document_detail/114567.html
At the same time, there are sample code libraries of cloud HBase&Spark for reference:
https://github.com/aliyun/aliyun-apsaradb-hbase-demo/tree/master/spark
