Ali onedate's layered thought

For the stratification of data warehouse , The name of each enterprise is different , But their thoughts are similar , All through space for time , Can precipitate the public model , Avoid repeated data processing , Data inconsistency caused by multiple references . Based on Ali OneData There are three layers of methodological data , namely ： Data import layer （ODS,Operational Data Store）、 Data common layer （CDM,Common Dimenions Model） And data application layer （ADS,Application Data Store）.

First , We should clarify the original intention of layering ：：

• Lack of reusable data model

• The result set is cleaned based on the original data 、 Process and calculate indicators

• The development of a lot of repetitive code consumes resources

The root of the problem is that the data model cannot be reused , And data development are chimney . So to solve this problem , We need to figure out how to design a robust data model , You can learn about the function of each layer and the data stored .

Data import layer ： It is to store the raw data without processing in the data warehouse system , The structure is basically consistent with the source system , It's the data preparation area of the data warehouse . The main responsibility of this layer is to synchronize the basic data 、 Storage .

Data common layer ： Storage of detailed fact data 、 Dimension table data and public index summary data . among , Detailed facts 、 Dimension table data are generally based on ODS Layer data processing generation . The summary data of public indicators is generally generated from dimension table data and detailed fact data .CDM Layers are subdivided into dimension layers （DIM）、 Detail data layer （DWD） And aggregate data layer （DWS）, The dimension model method is used as the theoretical basis ,  Take more broad table means to build public index data layer , Improve the reusability of common indicators , Reduce repetitive processing .

Data application layer ： Storage of data products personalized statistical indicators data , according to CDM Layer and ODS Layer processing generates .