01 Data Center Definition
The data center is a system and mechanism that turns the sleeping data of an enterprise into a data asset, continuously uses the data, generates intelligence, and serves the business, thereby realizing the realization of the value of the data. Through the methods and operating mechanisms provided by the data center, it forms aggregation and integration, purification processing, modeling processing, algorithm learning, and provides data for business use in the form of shared services, thereby linking with the business.
Furthermore, combined with the data production capacity of the business center, a closed loop of data production-consumption-regeneration is finally constructed. In order to better understand the data center, we compare it with related concepts such as data warehouse, data lake, BI, and big data.
1. Comparison with data warehouse
The data warehouse is a subject-oriented, integrated, relatively stable data collection that reflects historical changes and is used to support management decision-making. Therefore, the focus is on the collection of data. The data warehouse can use the dimensional modeling methodology to abstract common dimensions and metrics from the business process, form a data model, and provide general data analysis capabilities for decision analysis.
Compared with data warehouse , data center has at least four advantages .
First , the data center emphasizes data business, so that data can be used to meet the needs of enterprise data analysis and application.
Second , the process of data centralization is more complicated and comprehensive than the construction of a data warehouse. The data center has added a link to sort out the data domain from the overall perspective of the enterprise, which is a very important part of the construction of the data center. The sorting out of the data field just reflects the ability of middle-Taiwanization.
For example, in the new retail scenario, companies have many transaction scenarios, including self-built mall channels, third-party e-commerce channels, takeaway order channels, offline store channels, etc. When building a data center, a transaction domain needs to be planned. This transaction domain must abstract the business processes of various channels and cover the dimensions and metrics that online and offline operations departments need to evaluate during operations.
Therefore, the process of building a data center should focus more on the overall situation of the enterprise, opening up data from multiple dimensions of people, goods, and markets , so that consumers can gain insight into the trajectory of contact with the enterprise no matter which channel they come in.
The construction of the data warehouse is relatively simple, focusing on how to design the dimensional model, how to disassemble the indicators and dimensions, but seldom pay attention to the entity connection based on the subjects of people, goods, and fields, and then make the overall profile data for the front-end business. transfer.
Third , the scope of data center construction is far greater than the construction of data warehouses. In addition to completing the modeling of the data warehouse, it is also necessary to formulate a complete data governance plan, and even in the process of construction, a special data governance committee needs to be established to facilitate complexity. Data governance work.
The important point is that in the planning stage of the data center, it is necessary to actively cater to the business, and it is necessary to comprehensively sort out which business scenarios need to use the empowerment of data to form a business closed loop. Therefore, while building the data center, you must focus on the business. Empowerment of the scene.
Fourth , for enterprises, building a data center is not just a capability platform. As we mentioned in the book "China-Taiwan Strategy", the construction of China-Taiwan requires China-Taiwan culture and a matching China-Taiwan organization.
Therefore, from a macro point of view, the data center assumes the function of rebuilding the data organization of the enterprise, forcing the enterprise to build a set of data center organization that can match it in order to operate the data center. The data warehouse focuses purely on system solutions and does not involve organizational forms.
Therefore, in simple terms, the data warehouse focuses on building data, while the data center puts the establishment, governance, management, and service to the same height. The data warehouse is only a subset of the data center.
Then why did we develop from a data warehouse to a data center? Because traditional data warehouses can no longer fully meet the needs of enterprise data analysis. Enterprises have changed from the original statistical analysis to predictive analysis and provide algorithms such as tags and recommendations, from passive analysis to active analysis, from non-real-time analysis to real-time analysis, and from structured data to structured, semi-structured and Unstructured diversified data.
2. Comparison with data lake
The concept related to the data center is also the data lake (Data Lake). A data lake is a data storage concept. As a centralized repository, it can store data of any scale in a natural format, including structured data from rows and columns of relational databases, semi-structured data such as XML, JSON, and logs. Unstructured data such as e-mails and documents, as well as binary data such as images, audio and video, etc., so as to achieve centralized management of data.
Currently Hadoop is the most common technology to implement the data lake concept. For example, HBase allows the data lake to store massive amounts of data, Spark allows the data lake to analyze data in batches, and Flink allows the data lake to access and process IoT data in real time.
3. Comparison with BI
BI (Business Intelligence) is a series of methods, technologies, and software that analyze data and obtain insights to help companies make decisions. Compared with data warehouses, BI also includes tools such as data mining and data visualization, and can support users to arbitrarily combine dimensions and indicators within a certain range, thus rising to the level of supporting decision-making, rather than just as a data warehouse.
4. Comparison with big data
Data center is not equal to big data. The data center is a platform for data acquisition, storage, communication, management, and use based on big data, artificial intelligence and other technologies.
The data center needs the support of big data processing technologies represented by Hadoop, Spark, etc., but the data center and big data must not be equated. The data center is not only big data processing technology, but also includes intelligent algorithms, features that are linked to business, data assets, data tools, etc.
5. Summary
It can be said that the data center is a master of the above concepts and technologies.
-
First of all, the rich data computing and storage technology of big data provides powerful data processing capabilities for the data center.
-
Secondly, the data center is the gathering place for enterprise data, and the bottom layer of course also carries the functions of the data lake.
-
Thirdly, the domain-based modeling of data by the data warehouse is an important part of the data center, and it carries the function of managing enterprise data in an orderly manner.
-
Finally, based on powerful data capabilities, combined with business scenarios to provide real-time and intelligent services and applications are the core values of the data center.
02 The value of Taiwan in the data
The data center is not equal to the big data platform, and the core work of the data center is not enough to collect all the data of the enterprise and summarize it. The mission of the data center is to use big data technology to manage the company's data assets through overall planning, so that data users can obtain reliable data anytime and anywhere.
Therefore, once the data center is built and continues to operate, its value will increase exponentially over time. The value of Taiwan in the data is numerous, the three values of which are detailed below, as shown in Figure 4-1.
▲Figure 4-1 The three major values of Taiwan in the data
1. Help companies establish data standards
Before there is a data center, companies basically will not have a global data standard. Even if there is a related data standard, because there is no physical form of the data center, the data standard cannot be implemented. The construction of the data center will naturally help companies build data standards, including data construction specifications and data consumption specifications .
Data construction specifications include data access specifications, data modeling specifications, data storage specifications, and data security specifications. Data consumption specifications include data authority specifications, data calling specifications, and data destruction specifications. These standards must be established when building a data center and rely on the data center to implement and land.
2. Promote the formation of Chinese and Taiwanese organizations
No matter how grand corporate strategic planning is, it is inseparable from a scientific and reasonable organization to implement it. The construction of the data center will be an important part of the company’s macro-strategic planning, so in the process of practicing the data center construction, the first problem for the enterprise is how to build a stable escort for the construction and operation of the data center The Taiwanese team in the data.
This kind of systematic project of data center will connect the relevant parties of enterprise data horizontally, including the center of Taiwan construction team, the center of Taiwan operation and maintenance team, the data product manager team, the data asset management team, the data operation team, etc., forming a standard enterprise data committee , Thus forming a true middle-level organization of the enterprise.
It should be noted that the middle office organization can be a weak matrix organization that spans various business departments, or it can be a complete entity organization. This needs to be adapted to local conditions and varies from company to company.
3. Fully empower business to promote cost reduction and efficiency increase
The ultimate value of data center is to reduce costs and increase efficiency. Whether it is building data standards or forming a center organization, its core goal is to help companies achieve strategic planning.
Through the data center, the team can be deployed more reasonably; the entire time period from data processing to use will be greatly shortened; the power of the center will be used to integrate corporate marketing, transaction, service, inventory, logistics and other data, and combine the two Three-party and three-party data, from a global perspective, form powerful data assets to nourish various business segments.
At the same time, it purposefully designed data applications for enabling scenarios to help them shorten the product development cycle from research, production, sales and other aspects, produce products that will sell well in the future, and accurately find groups who are willing to buy the company's products. , So as to enhance the user's friendly experience of the company's products and services, increase the user's loyalty to the company's brand, reduce the loss in the operation of the company, and reduce the cycle of the supply chain.
These values are the goals that the company has been pursuing diligently.