Powering Industrial Internet of Things, industrial big data, industrial big data, and gas station dimension design [14]

01: Gas station dimension design

• Objective : Master the requirements and design of gas station dimensions

• path

  • step1: Requirements
  • step2: design

• implement

  • Requirements : Construct a gas station dimension table to get the gas station id, gas station name, geographical area to which the gas station belongs, the company it belongs to, the status of the gas station, etc.

    

  • design

    • Data Sources

      • ciss_base_oilstation: oil station information table

        select
            id, name, code,
            customer_id, customer_name,
            province, city, region, township,
            status, customer_classify, dt
        from one_make_dwd.ciss_base_oilstation
        where id != '' and name is not null and name != 'null' and customer_id is not null;
        

      • eos_dict_type: Dictionary status category table, records all tables that need to be marked with a dictionary

        select * from eos_dict_type where dicttypename = '油站状态';
        

      • eos_dict_entry: Dictionary status list, recording all specific status or category information

        select * from eos_dict_entry where dicttypeid = 'BUSS_OILSTATION_STATUS';
        

      • ciss_base_baseinfo: customer company information table [company ID, company name]

        select ygcode, companyname from one_make_dwd.ciss_base_baseinfo group by ygcode, companyname;
        

        • The data is duplicated, do a de-duplication

      • ciss_base_customer: customer information table [customer id, customer province name, company ID]

        select code, province, company from one_make_dwd.ciss_base_customer;
        

      • ciss_base_areas: administrative area information table

        • Associate all region information with a specific id

    • Realize the design

      • All tables are associated according to the corresponding fields, and the corresponding attribute fields are obtained

• summary

  • Master the requirements and design of gas station dimensions

02: Dimensional construction of gas stations

• Goal : Realize the construction of gas station dimensions

• implement

  • Create dimension table

    -- 创建油站维度表
    create external table if not exists one_make_dws.dim_oilstation(
        id string comment '油站ID'
        , name string comment '油站名称'
        , code string comment '油站编码'
        , customer_id string comment '客户ID'
        , customer_name string comment '客户名称'
        , province_id int comment '省份id'
        , province_name string comment '省份名称'
        , city_id int comment '城市id'
        , city_name string comment '城市名称'
        , county_id int comment '县城ID'
        , county_name string comment '县城名称'
        , area_id int comment '区域id'
        , area_name string comment '区域名称'
        , customer_classify_id string comment '客户分类ID'
        , customer_classify_name string comment '客户分类名称'
        , status int comment '油站状态（1、2）'
        , status_name string comment '油站状态名（正常、停用）'
        , company_id int comment '所属公司ID'
        , company_name string comment '所属公司名称'
        , customer_province_id int comment '客户所属省份ID'
        , customer_province_name string comment '客户所属省份'
    ) COMMENT '油站维度表'
    PARTITIONED BY (dt STRING)
    STORED AS TEXTFILE
    LOCATION '/data/dw/dws/one_make/dim_oilstation';
    

  • extract data

    insert overwrite table one_make_dws.dim_oilstation partition (dt ='20210101')
    select oil.id, oil.name, oil.code, customer_id, customer_name
           , oil.province province_id, p.areaname province_name
           , oil.city city_id, c.areaname city_name
           , oil.region county_id, county.areaname county_name
           , oil.township area_id, a.areaname area_name
           , oil.customer_classify customer_classify_id, ede.dictname customer_classify_name
           , oil.status status, eosde.dictname status_name
           , cbc.company company_id, binfo.companyname company_name
           , proname.id customer_province_id, proname.areaname customer_province_name
    from (
         select id, name, code, customer_id, customer_name, province, city, region, township, status, customer_classify, dt
         from one_make_dwd.ciss_base_oilstation where id != '' and name is not null and name != 'null' and customer_id is not null
    	 ) oil
         left join (select id, areaname, parentid from one_make_dwd.ciss_base_areas where rank = 1) p on oil.province = p.id
         left join (select id, areaname, parentid from one_make_dwd.ciss_base_areas where rank = 2) c on oil.city = c.id
         left join (select id, areaname, parentid from one_make_dwd.ciss_base_areas where rank = 3) county on oil.region = county.id
         left join (select id, areaname, parentid from one_make_dwd.ciss_base_areas where rank = 4) a on oil.township = a.id
         left join (select dictid, dictname  from one_make_dwd.eos_dict_entry) ede on oil.customer_classify = ede.dictid
         left join (select dictid, dictname from one_make_dwd.eos_dict_entry t1  left join one_make_dwd.eos_dict_type t2 on t1.dicttypeid = t2.dicttypeid where t2.dicttypename = '油站状态') eosde on oil.status = eosde.dictid
         -- 客户所属公司id，所属公司名称，所属省份id，所属省份名称
         left join (select code, province, company from one_make_dwd.ciss_base_customer) cbc on oil.customer_id = cbc.code
         left join (select id, areaname from one_make_dwd.ciss_base_areas where rank = 1 and id != 83) proname on cbc.province = proname.areaname
         left join (select ygcode, companyname from one_make_dwd.ciss_base_baseinfo group by ygcode, companyname) binfo on cbc.company = binfo.ygcode where dt = '20210101';
    

  • View Results

    

• summary

  • Realize the construction of gas station dimensions