Recently, I am studying related courses in the geek learning data center. Now I sort out some study notes and share with you.
table of Contents
Construction stage, which challenges facing the war ?
Can Zhongtai solve the above problems?
How does the data center solve these problems?
Which companies are suitable for building data center?
Challenges before the construction of the central stage
The emergence of a large number of data products, while continuously improving the efficiency of business operations, also exposed many acute problems:
- Inconsistent indicators
- Data repeated construction, long demand response time
- Low access efficiency
- Poor data quality
- Linear growth in data costs
Can Zhongtai solve the above problems?
Question 1: The indicators are inconsistent. There may be three reasons for inconsistencies: business standards, calculation logic, and data sources.
Response: For the same indicator, there is only one business caliber, and the data source must be the same for one processing.
Problem 2: Data duplication construction, long demand response time, repeated development for reasons, and a lot of duplication of logic codes.
The way to deal with it: to achieve data reuse, ensure that the same data is processed only once, and realize data sharing.
Problem 3: The fetching efficiency is low, because the data cannot be found or the data cannot be fetched.
The way to deal with it: If you can't find data, you need to build a global enterprise data asset catalog to realize the data map function. If data is not available, it is necessary to provide a visual query platform for non-technical personnel to facilitate use.
Problem 4: Data quality is poor. In fact, data problems are difficult to find. Data link processing is generally long. When problems occur, the complexity of troubleshooting is increased and failures cannot be restored in time.
The way to deal with it: Find out in time and recover data quickly.
Question 5: The linear growth of data costs has led to an increase in the cost of big data, which is related to the duplication of data behind the slow demand response.
Response: Eliminate redundant data and reduce redundant construction.
Data center: The enterprise builds a standard, secure, unified, and shared data organization, and supports front-end data applications through data service. Zhongtai aims to achieve high-efficiency, high-quality, and low-cost support business.
How does the data center solve these problems?
Index caliber issue: Indexes are the result of data processing, and unified management and control of index calibers are carried out to improve index management efficiency. Clarify the business caliber, data source, and calculation logic of each indicator, and manage it in a manner similar to the data warehouse subject domain. Sort out the global indicators, eliminate the ambiguity of the indicator interface in the product, and provide an indicator management system that is convenient for analysts and operational queries.
Repetitive construction problem: All data are processed only once, and metrics or indicators of the same granularity are processed only once, to construct a globally consistent public dimension table. Two tool products are required:
- Data Warehouse Design Center, in the model design stage, force models with the same aggregation granularity, and the measurement cannot be repeated. Realize sinking and reuse of common computing logic.
- Data map is convenient for data development to quickly understand the precise meaning of a table.
Efficiency issues: Data warehouse data is provided to data applications through API interfaces (the difference in access methods of different query engines is shielded from the application side) to improve access and management efficiency. At the same time, it provides a visual access platform + enterprise data map . Complete non-technical personnel self-service access.
Quality: full link quality monitoring data, for each table outputs an indicator upstream link involved are real now consistency, integrity, accuracy and timeliness of monitoring to ensure that the first time found Recovery, notification of data problems.
Cost issues: From the application dimension, table dimension, task dimension, file dimension to comprehensively manage. Take low-value reports offline in time to reduce data governance costs.
Which companies are suitable for building data center?
Enterprises consider important factors when selecting data centers:
- Whether there are a large number of data application scenarios, the middle station itself cannot directly generate business value. Its essence is to support the rapid incubation of data applications.
- There are many isolated islands of business data. For example, in the early stage of e-commerce, warehousing, supply chain, and market operations are all independent data warehouses. When doing data analysis, it spans multiple data systems.
- When the team is at a loss for efficiency, quality, and cost issues, the leader also requires control of data costs.
- Companies are facing operational difficulties and want to improve operational efficiency through lean operations.
- The size of the enterprise must be matched, with large investment in China and Taiwan, and long-term income, which is more suitable for large companies with relatively stable business.
Data center construction requires a lot of investment and cannot do without system support. Whether these systems can match the middle station construction requires continuous iterative polishing. At the same time, facing a large amount of data requirements, additional manpower is required to reconstruct the data model.
Reminder: You can't blindly follow the trend, it's best for you.