Data warehouse modeling
When it comes to data warehouse modeling, we have to mention two sets of classic theories:
-
Paradigm modeling
Inmon's hub-top-down (EDW-DM) data warehouse architecture. -
Dimensional modeling
Kimball proposed a bus-based bottom-up (DM-DW) data warehouse architecture.
The modeling or layering of data warehouses is actually to better organize, manage, and maintain data. During actual development, the two methods will be integrated and used. Of course, there are others, such as the Data Vault model and the Anchor model. I haven't applied it yet, so I won't talk about it.
For dimensional modeling, star model and snowflake model are generally mentioned. Star model is very convenient for OLAP analysis.
Data warehouse stratification
To be simpler, just use ODS+DM, synchronize all the data, and then directly develop some application layer reports. This is the simplest; when there is more content in the DM layer and you want to reuse it, you will Split a common layer out and turn it into a 3-tier architecture. I recently read Alibaba’s book, "The Road to Big Data", which contains a lot of content related to data warehouses. It is very good. After reference, the currently used layering model is as follows. :
According to this layering method, our development focus is on the dwd layer, which is the detailed data layer. Here are mainly some wide tables, which store detailed data; when it comes to the dws layer, we will aggregate the data for different dimensions. Well, logically speaking, the dws layer is considered the market layer, which is generally divided according to themes and belongs to the category of dimensional modeling; ads is the application layer, which is the output of various reports.
Based on this layered approach, it feels very convenient to use. Well, let’s go here first.
Author: Does orange cat eat to gain weight?
Link: https://www.jianshu.com/p/2b0509851df1
Source: Jianshu
Copyright belongs to the author. For commercial reprinting, please contact the author for authorization. For non-commercial reprinting, please indicate the source.