Data is more important than ever, yet most organizations still struggle with some common problems: They focus more on data infrastructure than data products; data is often created with department-specific needs in mind, but rarely Less consideration for end use; they lack a common "data language" and each department codes and categorizes it with its own system; they increasingly focus on external data but have little quality control system. These issues can be addressed by focusing on "data supply chain" management. Similar to physical supply chains, companies should think systematically, focusing on the end product, defining standards and measurements, introducing quality controls, and continuously improving their methods at all stages of data collection and analysis.
Data management has plagued large corporations for decades. Almost all companies spend a lot of money on this and find the results unsatisfactory. While the problem doesn't appear to be getting worse, addressing it is becoming more urgent as managers and companies strive to become more data-driven, leverage advanced analytics and artificial intelligence, and compete with data. In this article, we will explore an effective data management approach through the perspective of "data products" and "data supply chain".
Most companies struggle with some common but important data management issues.
First, enterprises focus on the technical capabilities of data management, which are controlled by the IT function and need to acquire, store and move data. This is no easy feat - building a technology "pipeline" is challenging work. But in doing so, they focus more on the infrastructure than the output: the data products used to make decisions, differentiate products and services, and satisfy customer needs.
Second, data is created in different parts of the organization to meet the needs of various departments, not for later use by others in data products, business decisions, or processes. Contrast this with a physical product such as a car, where components such as the chassis and starter are designed with the final product in mind.
Third, most organizations lack a common data language. Data is subtle and nuanced, and means different things to different people in different contexts. To add insult to injury, some departments have ownership of "their data" and may not be willing to share it. Or while willing to share, they don't take the time to explain the nuances so others can use it effectively. This led to other departments setting up their own "near-redundant" databases, adding to the overall confusion.
Finally, companies are increasingly interested in what's happening externally, using external data to answer a variety of questions. But external data is largely unmanaged, with little vendor qualification or data quality assessment.
Data supply chain management, with data products as the end result of the process, can help address these issues. It pays equal attention to all phases of data management - from collection to organization to consumption of data products. It's a way to balance the benefits of generic data with unique and custom data in the product, and it applies equally to internal and external data. Relatively few companies have adopted DSM, but those that do tend to report better results.
Process and supplier management for data products
Companies have been producing data products in the form of financial statements, reports to regulators, etc. Nonetheless, the scope and importance of such products continues to grow. For many, the goal is to embed analytics and AI-derived models into products that serve internal and external customers. Morgan Stanley's Next Best Action, LinkedIn's People You May Know, Google's many search products, and MasterCard's SpendingPulse and Business Locator are good examples. As the issues mentioned above are on full display, "arguing" the data takes much longer than building the model, and still doesn't fix everything.
After research, there is a better way to get high quality data. It builds on the processes and supplier management techniques used by manufacturers of physical products. In particular, manufacturers dig deep into their supply chains to clarify their requirements, qualify suppliers, insist on suppliers measuring quality, and make necessary improvements at the root of the problem. This allows them to assemble components into finished products with a minimum of "physical product wrangling", improving quality and reducing costs.
One organization that has adopted supplier quality management in its data supply chain is Altria, a US-based supplier of tobacco and smoke-free products. Altria relies on point-of-sale data from more than 100,000 convenience stores every day for its market reporting and analysis. A team reporting to Kirby Forlin, Vice President of Advanced Analytics manages this area. The data requirements are clearly specified in the contract, and the team aims to help stores meet those requirements. First, Altria focused on its most basic requirements. The quality is poor, with only 58% of daily submissions meeting their requirements. But the Altria team worked patiently to improve the quality to 98% within three years. As the base quality improved, the Altria team added its more advanced requirements to the mix. As Forlin puts it, "This is a work in progress. The evidence that we can increasingly trust the data saves us a lot of work in our analytical practice and builds trust in our work."
Steps to build a data supply chain
A data supply chain can be established within a company using some of the same steps used in the process and quality management of a physical supply chain:
1. Establish management responsibilities.
Step 1a, the chief data officer or product manager should appoint a "data supply chain manager" from among their staff to coordinate the work, and recruit "responsible parties" from each department in the entire supply chain (including external data sources).
Step 1b is about bringing issues related to data sharing and ownership front and center. We find that most problems go away because few managers are willing to take a strong stand against data sharing in front of their colleagues.
Identify and document the data required to create and maintain data products and the associated cost, time, and quality requirements.
2. Describe the supply chain. Develop a flowchart describing the point of data creation/source of raw data and the steps taken to move, enrich and analyze the data for use in a data product.
3. Define and establish measurements. Usually, the idea is to implement a measure of whether the indicator satisfies the requirements. Start with data accuracy and the time it takes from data creation to incorporation into data products. The supply chain for each data product will have different measures.
4. Establish process controls and assess compliance with requirements. Use the measurements from Step 4 to control the process and determine to what extent the requirements of Step 2 are being met and to identify gaps.
5. Survey the supply chain to identify needed improvements —both overall and specific data products. Identify the gaps found in step five from the flowchart in step three.
6. Make improvements and monitor continuously. Identify and eliminate the root cause of the gaps identified in step six, returning to previous steps if necessary. Continuously monitor input data and data products, seeking to improve products as well as new and better sources of data as needed.
7. Ensure "qualified" data sources. Companies will continue to recruit more and more external data vendors, which helps identify those that consistently deliver high-quality data. An audit of its data quality program provides a way to "identify" those who do and identify weaknesses in those who don't.
Key Bank, a top 20 US bank by assets, used broad data supply chain concepts to structure its data management program. It decomposes its process into three areas of "capture/organization/consumption" and tries to improve the efficiency and effectiveness of each area. Key Bank recently moved most of its data storage and analysis to the cloud and saw major improvements in flexibility and speed across its supply chain. Its consumption activities have historically focused on classic business intelligence capabilities, but now it also has strong data science capabilities.
This requires changes to the data supply chain to enable greater data virtualization and the ability to build views of data that span disparate data sets and include external data. The bank has been able to leverage its data supply chain to rapidly develop new banking products that are highly data-dependent. For example, Key Bank, one of the largest lenders for Paycheck Protection Program loans in the U.S., also recently launched a national digital bank for physicians. The bank's chief data officer, Mike Onders, is effectively a data supply chain manager. He and his staff assessed the ability of the bank's data supply chain to deliver the various data products needed.
All companies are advised to actively manage their most important data supply chains. Data is as important to businesses as any other type of asset, and data products are increasingly as important as physical products. Adopting the same philosophies and approaches as for physical supply chains has proven to be equally valuable for digital supply chain management.