Data warehouse layered architecture

Storing data layer is generally as follows:
Data Source
Data source libraries and business in general is buried point, of course, there will be a variety of data sources ways to purchase third-party data. Library storage business is generally Mysql and PostgreSql.
ODS layer
ODS is generally the amount of data is very large, so most companies will choose to exist on HDFS, Hive that is or Hbase, Hive majority.
ODS can be made into a wide table, in conjunction with the fact table and the dimension table DW or more.
DW layer
General and storage of ODS consistent, but in order to meet more demand, there will be deposited in the case of PG and ES.
DW has conducted general dimensions, the fact table design; design dimension tables and fact table based on different themes.
APP layer
Application layer data, generally require a relatively fast response speed, and therefore is generally placed Mysql, PG, Redis in.
DH had general, data marts, for different users, data isolation. Storing the aggregated data.

Guess you like

Origin www.cnblogs.com/xibuhaohao/p/11718889.html