Recently, I have focused on the field of data and artificial intelligence, from data warehouse, business intelligence, master data management to the construction of big data platform. After the precipitation and summary of many projects, my team and I finally summarized the system of lean data innovation. Has been fighting in the front line of enterprise informatization.
Why do enterprises need to build a data center, and what is the value of a data center to an enterprise. From the perspective of concept and framework, it will provide you with a more comprehensive perspective. The prerequisite for doing anything well is to figure out why.
1 The demands of the data center platform
Baidu Search Index
The Baidu search index of the data center finally surpassed digital transformation and data warehouse. Many companies are already on the ground, and companies that have not yet started are also under consideration. Obviously, the data center can meet some demands of enterprises.
The word cloud map obtained from the Taiwan industry research in the data center.
What everyone expects the most from Data Center:
- closer to business
- Provide direct business value to the enterprise
- Provide data services, not just reports
1.1 The data systems of traditional enterprises are far away from users and businesses
- The data system is only technical support and cannot directly generate business value. Building a business system, such as an e-commerce platform, can directly bring in income. But traditional data applications, such as data warehouses, cannot bring direct value
- When the business personnel want to modify some content in the report, the response is slow, because the business personnel cannot directly use the data to generate insights, and need to go to the data team
- In the past, a large amount of investment in data was spent on data collection, processing and modeling, but not many were actually used in the business field
From another perspective, the business development team has great demand for data utilization, which is reflected in the hope that the data center can solve the efficiency problems, collaboration problems and capacity problems of enterprise data development.
The data middle platform gives application developers such hope: they don't need to pay attention to the specific data retrieval logic, but only need to pay attention to customer needs, and combine the data services of various data middle platforms to their own applications like building Lego, and the data is accurate and consistent.
Therefore, the value of the data center is to change the original form of data utilization by enterprises.
In the past, the main form of data utilization was BI. To put it bluntly, it was to make reports. To make reports was to let managers know what was happening in the current business, why these things happened, and what might happen next. All these were provided to Our managers went to see it and helped managers make a business decision.
With the increase in business complexity, there are many factors behind a decision, and a phenomenon requires multi-dimensional interpretation to reflect the overall picture of the business. As a result, managers need more and more reports. Many companies have multiple data warehouses with different business lines. Each data warehouse has more than a thousand reports, and finally falls into a maze of reports.
Looking back at this process, we found that the report is not what we need, and the data itself is not what we need. What we need is a business decision, a business behavior. For example, when a user opens an e-commerce product catalog, the product he is most likely to purchase is displayed on the first page, but the original OLTP and OLAP separation data processing process cannot do it, and in the process of business transactions, it is also impossible to obtain information from historical data and global data. Guidance for action is obtained from the analysis results.
The market's expectation for the data center is to provide data services that directly drive business processes, not just data visualization reports that need to be transformed and interpreted by humans. The original business intelligence era is over, and the market and users look forward to the data intelligence era.
2 The fundamental purpose of building a data center
The construction of a data center seems to be able to provide a solution to this demand. However, it is not easy to build a middle platform. It needs to make corresponding adjustments in technology and organizational structure, and the implementation process also faces various challenges. Why do companies still mobilize people to land in the data center? The vision of Data Center is to build a data-driven intelligent enterprise.
Enterprises build data centers and become data-driven intelligent enterprises
3 Benefits to the business
- optimize existing business
- Realize the transformation of new business
3.1 Optimize existing business
Optimize the original business process through the application of data analysis and artificial intelligence technology.
1 Increase revenue from existing business
For example, by analyzing product prices, sales volume, and user data to optimize product pricing, optimize product mix, and conduct precise marketing, it can promote product sales and increase revenue from existing products.
the case
A data center for an energy company. After completion, it can be modeled based on data such as historical sales, market share, and market capacity, thereby helping the sales department of the company to optimize the allocation of sales tasks and increase sales. What value can this small project bring to the enterprise?
In the past, at the beginning of each year, the company determined the performance of dozens of sales across the country and continued to track it. It was very painful, and the goals were not anchored. Track sales performance.
In the past, all of this relied on the experience of the sales director to make numbers, and more on negotiation power, with high uncertainty. With the data center, the industry data, market competition data, sales data of previous years, and dealer data are all pulled together to see the whole picture of the sales director at once, and it can also be simulated, making the work of sales performance allocation a predictable task. Quantified, predictable deterministic work.
2 Boost production efficiency
Through the construction of data center, it can promote the improvement of production efficiency.
For example, a large telecommunications service provider, through the modeling of survey, planning, and design work, realizes automatic data processing, reduces manual intervention and the occurrence of problems, greatly improves the efficiency and accuracy of engineer design, and shortens the engineering design cycle by half.
Analyze the reasons. The bidding of telecom service providers is a very complicated job, from the customer's request to survey, plan, implement the design, and then transform the implementation design into material design, engineering design, financial design, and finally form the bidding plan , this process used to take at least a month, requiring the collaboration of many different business departments and professional skills, and most of the work was spent on merging, pulling, aligning and mapping different data.
After the enterprise builds a data center, all data can be processed automatically. Everyone can obtain, modify, and process the same set of data in the same data service, and each time the process of making a plan is precipitated into a new service, and subsequent projects can be reused, greatly To shorten the construction period, some highly standardized project types can be shortened from the original one month to three days.
Reduce operating costs and increase operating profits
At present, the most used scenarios are mainly to optimize business processes and shorten operation cycles through data analysis, thereby improving operating profits.
For example, formula planning optimization for a large iron and steel plant, through the comprehensive analysis and modeling of formula data, market price, and sales data, the production combination with the optimal cost and the highest output value is given, reducing operating costs and increasing profits.
In the iron and steel industry, the decision-making of ore blending is very complicated but very important. Different ore blending schemes have different costs and processes, which have a great impact on profits. How to choose the optimal formulation according to many technical and commercial factors? In the past, the maintenance and calculation of ore blending rules were based on experience, which was low in efficiency and long in cycle.
With a data center, the data of technical factors such as raw material performance, chemical process, and product quality and commercial factors such as price, composition, operating costs, and sales revenue are uniformly modeled, and a comprehensive plan is finally made after unified calculation, which greatly improves profits and reduce operating costs.
As shown in the figure below, this is a typical table of steel powder formula and manufacturing cost. Changes in each item will bring about cost changes. In addition to manufacturing costs, sales prices and operating costs affect profits. etc. In this way, how to design the optimal ore blending decision is a very important factor. Through data modeling, ML's intelligent mineral distribution model can plan the optimal plan in an all-round way to achieve specific business goals, such as shortening the production cycle or increasing profits, and optimizing inventory.
Improve user experience
The core of improving user experience is that companies need to understand their users, know their perceptions of their products and services, and then optimize their products and services accordingly. This requires the construction of a user data platform, a unified user view, and a user portrait.
Here I will give an example of Wells Fargo Bank. During data transformation, they used the data center to analyze user behavior data to reconstruct the online banking website and improve user experience. Wells Fargo faced great performance challenges in 2016. In order to better understand users, they established an enterprise-level data middle platform, which opened up the user information of the entire bank, made user portraits, and put various labels on them. And based on these user portraits and tags, the e-banking website was redesigned to make the service and style of the website user-centered.
After several years of data transformation journey, Wells Fargo has also become the industry's "retail king" because of this project. For more details, "Wells Fargo's Data Transformation Journey" .
Improve asset utilization
Analyze and optimize high-value assets to improve asset utilization.
Path optimization in logistics field
Logistics companies do route optimization projects to improve the utilization rate of personnel and vehicles. In the past, each district had an experienced employee every morning to plan the delivery and pickup orders received the day before, and distribute these orders to the corresponding teams.
The purpose of this process is to maximize the use of the two core assets of vehicles and couriers. But this planning is very complicated, because not only the cost must be considered, but also the backlog time of each piece is different, the degree of urgency is different, and the requirements for vehicles in different locations and road conditions are different. Modeling is a very important basic work and all of this depends on the connection with data.
After building a data center and connecting the data, the path of dispatching and receiving orders is more optimized, better allocated to couriers, and the vehicle utilization rate is increased by 20%.
This scenario is very typical, reflecting the value of the intelligent planning business supported by the data center.
3.2 Business Innovation and Transformation
The second benefit of building a data center is to realize business innovation and transformation. Four main values.
① Digital product innovation
A customer of an overseas real estate transaction website who has cooperated with them for more than ten years, and regularly conducts hackathons with them.
Once, a team in our hackathon found a small pattern through data analysis. A group of users visited the website frequently for a period of time, but did not have any behavior of viewing or selling houses. Finally, the data analysis found that such users have common characteristics. Most of the women have the longest basic access links, and many of them are pictures, and they are indoor pictures.
It was speculated that this group of people came to see the decoration, so the team incubated a new product, specializing in providing decoration services. This product finally succeeded and became a new business line of the company besides real estate intermediary services.
This is also a typical scenario in which new business value is discovered through data insights to realize digital product innovation.
② Digital asset sales
Combine, package, analyze, and desensitize the accumulated data to form data assets that are valuable to some users, such as industry reports or high-quality content, and sell them directly to generate income.
A typical scenario is a search engine. After the search engine performs statistical analysis and desensitization processing on user information, it turns it into a series of knowledge and analysis reports, and then provides them to users in need as members.
In the Baidu Index, users can define and purchase keywords they are interested in at 198 yuan a year, and then Baidu will count all the records that have been searched for this keyword and turn it into a search index for this keyword. For example, the keyword of Taiwan in the data is what I purchased last year, and I can track the popularity of this keyword in the Chinese market in real time. This is the value model of digital asset sales.
③ Benefits from business platformization
There is a saying that was very popular last year, "Enterprises in the future must either build their own platforms or be platformized by others." The platform economy has become a hot concept in the digital field in recent years. In general, platformization means that you build a platform, let the demand side and the supply side come up for transactions, and finally you charge the service fee.
How to build a platform? Pull together a field of data to form a digital platform, and then operate a specific business and customer group through the platform, so as to generate revenue through the platform.
Typical scenario - transaction matching platform, such as Bitcoin trading platform. What does this have to do with data center?
This process is actually the construction process of the platform in the field of data, because the main business of the platform is the business of data, connecting information and connecting both parties of the transaction. The data center is a platform for data collection, processing, and transactions within the enterprise. The business party may be a consumer or a producer of data services, and the final product is data services.
④ Digital ecological business
From a higher perspective, on the basis of platformization, by breaking through the industrial supply chain, it helps enterprises build their own digital ecology, thereby generating new business value and income in the ecology. Such as Google App Store.
When enough partners conduct transactions on this platform, it will be able to discover many patterns in these massive transaction and behavioral data, and then generate more product innovations, using data to guide the development of this ecology in the direction of its own design.
In this ecology, there are many roles involved, such as developers, freelance developers, advertisers, app buyers, etc., and Google has all the data of all parties, user browsing, downloading, payment, transaction, all data can be analyzed and used to help Google The Play operator discovers new business value and generates revenue.
4 Summary
What is the use of building a data center? A data center revenue framework includes two dimensions and nine subdivisions. The core thing is to find the goal for us to build a data center. We can use these 9 items as a guide, first clarify the value and direction, and then find the application scenario, and use this as a traction to build our own data center.
Most enterprises are going through a transformation, evolving towards digital intelligence. In the transformation of enterprises, from the earliest informatization to digitalization, the next goal is digital intelligence.
Informatization solves the internal management problems of the enterprise, enabling the enterprise to operate efficiently in an organized and process-based manner.
Digitalization solves the problem of connecting enterprises with the outside world, allowing enterprises to directly contact customers and establish online businesses.
Digital intelligence solves the problem of making enterprises become intelligent enterprises and business more intelligently. The core production factor of this process is data.
Digital and intelligent transformation can bring disruptive changes to enterprises, but how to discover the value of data, build the ability of data intelligence, and empower business at scale? Enterprises need a starting point to use it to align business and technology and keep moving forward. The data middle platform is like this starting point. Whoever can build the data middle platform around the value of the above two aspects of business optimization and transformation will gain a leading edge in the digital and intelligent transformation.
The wave of data center platform provides us with opportunities, but this opportunity also puts forward many and high capability requirements.
FAQ
What capabilities are needed to become a digital intelligence enterprise? I have done many researches on the transformation of leading companies at home and abroad, and found some regular and interesting things.
When the concept of data center platform came out, there were many theories. If you analyze it from different levels, you will find that the root of all problems comes from the word "positioning".
- For programmers, the data center may be a good way to solve the problems of data specification and data drilling
- For product managers, the data center may be the key to solving the fusion of business and data
- For middle-level managers, the data center is an important basis for opening up data sharing between departments
- For entrepreneurs, keeping up with industry trends and following them is an essential part of telling a good story
But there is one general trend, informatization -> digitization -> intelligence, that is, digital intelligence.
Just like the advent of 5G is bound to drive some new unicorns. The unpredictable style of play on the Internet is more like ancient military warfare. Whoever can be keenly aware of the enemy's situation will be able to make the fastest response quickly. It is also like the one described in the dark forest of "Three-Body Problem". The hunter in the dark forest, whoever spots the prey first, shoots. It brings about a series of knock-on effects.
Our company is engaged in data acquisition (extraction, conversion, and sharing of data from various storage media such as relational databases and big data components of various information systems) services; it is a functional product, and if we don’t understand the business, then we can participate in the part of the work in the data center ? Because when the company is discussing, more views are mentioned, and even the definition of the data model is not clear, how to intervene.
The data integration work that your company mainly does is located in the entire data R&D link, the first link in the R&D phase.
The division of the entire data R&D link:
After understanding the position of your work in the data center data R&D link, let's see how you participate in the data center stage.
Data integration products, participating in the link of the data center:
- When importing data, establish a data link relationship from the data source to the table in the data center, so that the blood relationship of the table in the data center can be extended to the data source of the business system. When the data source changes, we can get it immediately to change information
- Data transmission should be connected with the metadata center to obtain various data source information from the metadata center
- For data integration, it is necessary to support both batch data integration and real-time data integration
- Data transmission must be able to establish task dependencies with data development tasks, and subsequent data cleaning tasks depend on data integration tasks
As for the unclear definition of the data model, data transmission must be connected to the metadata center. In the metadata center, there is a data dictionary definition for each table, and data transmission can be self-adapted based on the format of the field.