01. Definition of research scope
Using a variety of data intelligence technologies to achieve data-driven analysis and decision-making has become one of the most important goals of the current digital transformation of enterprises. With the increasing abundance of data sources and the rapid growth of data volume, enterprises rely more and more on data mining, which not only brings complex and diverse data application scenarios and data user roles, but also makes enterprises agile and real-time for data applications. Sexual demands are getting higher and higher.
Traditional data development and management, data calculation and analysis are facing huge challenges. In order to meet the ever-increasing demand for data applications, iAnalytics observed that enterprises are shifting to focus on business in terms of data capacity building logic, and the market is also being driven by the realization of specific business value in terms of supply of technology and solutions. More differentiated and focused.
The core logic of data capacity building turns to realizing business value. In the past, when an enterprise built data capabilities, the technical department or data department usually carried out unified planning, and carried out data development and management, while the business department passively used the enterprise's data capabilities. However, as business departments pay more attention to the value of data and the demand for data applications increases rapidly, the enterprise technology department or data department can no longer meet the data usage needs of business departments. In order to enable business departments to use data better, the construction of enterprise data intelligence infrastructure is gradually changing to focus on business departments, such as trying to achieve efficient collaboration between data and business departments through concepts such as DataOps and index middle platform.
Facing the value realization of business scenarios, technical solutions are more subdivided and more focused. The practice of the past few years has shown that a large and comprehensive data platform is not suitable for all enterprises. According to the differences in data sources and data uses within the enterprise, a variety of data platform solutions for specific scenarios have been differentiated in the market, such as for Real-time data platform for risk control and marketing scenarios, edge-cloud collaborative data platform for industrial and Internet of Things scenarios, data federation analysis platform for accelerating multi-data source joint analysis, heterogeneous data real-time analysis platform for accelerating heterogeneous data analysis, etc. .
Based on the above background, Aianalysis divides the data intelligence market into data infrastructure and application solutions. The data infrastructure covers multiple technology stacks in the data life cycle, and the application solutions cover multiple vertical industries and general intelligent solutions. The specific market division is shown in the figure below.
This evaluation report focuses on the data development and management platform market, and Aianalysis selects manufacturers with mature solutions and implementation capabilities to provide reference for enterprises when selecting data development and management platform vendors. At the same time, in this market, Aianalysis focused on selecting data development and management platform manufacturer Yanhuang Data for capability assessment.
02. Data development and management platform market analysis
Market definition: Data development and management platform refers to the establishment of a complete data processing link covering data integration, development, storage, computing, service, task scheduling, etc. in response to the needs of business users, and data that provides global data asset management capabilities platform.
Party A's end users: heads of big data departments and IT departments in industries such as finance, manufacturing, automobiles, consumer goods retail, and energy
Party A’s core needs:
As the business department’s demand for data analysis becomes more and more extensive, Party A’s enterprise needs to build a set of data development and management processes and mechanisms oriented to the business department’s data usage needs, and improve the corresponding data development and management capabilities . In the past, Party A mostly completed data integration and management as a phased goal and project of the enterprise, and did not pay enough attention to how to apply data and how to exert value in business scenarios. In practice, after investing a lot of resources and manpower to complete the data integration, problems such as "difficult to obtain data", "difficult to use data" and low data quality still exist, and Party A still cannot give full play to the value of data. Therefore, what Party A really needs to have is a complete set of business-oriented data development and management capabilities, and its core requirements include:
-
Build an end-to-end data development and management platform with complete functions. The platform needs to provide complete functions around the needs of the full link of data development and management, and have automated development capabilities. Party A needs to be able to complete the development and management of various structured, unstructured and semi-structured data on this platform, covering the integration, development, storage, computing, service, task scheduling and other requirements of the whole link of data processing, Capable of global data management. At the same time, in order to cope with more and more time-sensitive development tasks, it is also necessary to use automation tools to improve efficiency.
-
The platform needs to be compatible with existing data infrastructure and support secondary development. After years of informatization and digitalization construction, the vast majority of Party A already have a certain data foundation, the technical architecture represented by MPP and Hadoop, and the data development and management tools with the big data platform as the core. Therefore, the data development and management platform needs Compatible with existing data infrastructure. At the same time, with the development of business, there will be more and more innovative business scenarios in the future. The platform needs to be fully scalable, and can be re-developed to access external tools to meet diverse needs, so as to support various types of business scenarios.
-
Establish a unified data development and management process and mechanism. In the existing process of Party A, application development and data development are often carried out separately, but considering the trend that more and more digital applications are driven by data, enterprises need to consider integrating the two. Although the data middle platform built in the past can support reports, self-service analysis and other applications to a certain extent, it still fails to support the entire data development and management system in essence, and cannot meet the needs of more and more data-driven applications. Exploratory applications represented by ad hoc query and machine learning. Therefore, Party A needs to integrate application development and data development, and establish a unified process and mechanism.
Manufacturer capability requirements:
-
The data development and management platform products have complete functions. It can cover the whole process of data development and management, including data integration, development, storage, calculation, service, etc. It can provide multi-person collaborative project space management, and has the ability of continuous integration and release.
-
The product architecture needs to have strong scalability. It needs to have the ability of decoupling, build in a modular way, and be able to separate functional modules separately and provide them on demand. In terms of scalability, it needs to be able to adapt to other ecosystems in the enterprise, support multiple interface protocols, and have been packaged and tested and connected to various software or hardware interface calls to quickly meet the innovative applications of the enterprise in the future.
-
Establish a unified data development and management process according to the needs of business scenarios, provide consulting services or internalize the process into product standards. Manufacturers need to have an in-depth understanding of data application scenarios, as well as rich experience in customer service, build a development and management process that meets the data application needs of Party A’s business department, realize efficient data processing, and provide customers with corresponding consulting suggestions. In response to the common needs of some industries, manufacturers need to combine their product and technical capabilities, integrate development and management circulation into data platform products, and provide industry best practices.
Inclusion Criteria Description:
- Comply with the capability requirements of all manufacturers of the data development and management platform;
- In 2022, the number of paying customers in this market is ≥ 5;
- In 2022, the contract revenue of this market is ≥ 10 million yuan.
Manufacturer panoramic map:
03. Vendor evaluation
Manufacturer introduction: Yanhuang Data is a company dedicated to building a big data processing platform with independent intellectual property rights. Its core product, Yanhuang Data Platform, focuses on providing real-time analysis capabilities for heterogeneous and multi-source big data. The company's core team comes from the former Splunk China R&D Center, with profound experience in big data analysis, architecture design and system development.
Product service introduction: Yanhuang data platform is a new generation of real-time analysis platform for heterogeneous big data. Combined with its one-stop data development and management capabilities, as well as unique time-reading modeling, search engine and other technologies, users can analyze heterogeneous original data from various machines, Internet of Things devices, mobile terminals, business systems, and databases. Instant analysis.
The service scope of Yanhuang data platform covers pan-finance, Internet, new energy, industrial manufacturing and other industries. Typical application scenarios include: data security, AIOps, process mining, spatio-temporal data analysis, industrial Internet of Things, etc. Has served benchmark clients such as Zhongan Insurance, Shanghai Electric Power, and Knowledge Planet.
The core technologies such as big data search engine and time-reading modeling of Yanhuang data platform are all self-developed, so that the source code is controllable and have reached the international advanced level. It is a reliable choice for domestic alternatives.
Vendor evaluation:
Yanhuang data platform is specially designed for modern enterprises with a wide range of data sources, diverse and variable data formats, and data analysis requirements with certain time series characteristics. By providing a one-stop data development and management It brings flexible, instant, easy-to-use, and fast-deployment experience in the query and analysis of structural data, as follows:
-
Based on the self-developed time-reading modeling storage and computing engine, the Yanhuang data platform can flexibly support the storage, query and analysis of heterogeneous and multi-source big data in various scenarios. In terms of data storage, the platform does not need to define the data structure in advance, but can store unstructured, semi-structured and structured data from various production management systems of the enterprise in the platform in a unified manner according to the original format of the data, breaking the data silos and at the same time The integrity of the data is guaranteed; in terms of data query and analysis, the platform's read-time modeling technology allows users to customize rules when reading data, and automatically extracts the fields required for analysis from the original data according to the algorithm, and supports users according to business needs Dynamically adjust data query rules to avoid heavy traditional ETL work and improve the flexibility of heterogeneous data processing. At the same time, Yanhuang Data's self-developed search engine provides simple keyword queries similar to Google, as well as high-level queries such as interactive queries and event context searches to meet users' data query needs in different scenarios. In addition, the platform also supports write-time modeling of structured data to improve the ability to analyze structured data.
-
The Yanhuang data platform has carried out multiple optimizations in terms of data processing and platform architecture, which can ensure that users can obtain analysis results in real time. Aiming at the problem that the inherent computational overhead of the read-time modeling is high and affects the query speed, Yanhuang Data has done a lot of exquisite work in vectorized computing, real-time compilation, data compression based on columnar storage, and concurrent task scheduling. Designed and optimized to speed up computing speed and increase data throughput, so that relatively instant analysis results can be achieved in most scenarios; in terms of platform architecture, Yanhuang Data Platform adopts cloud-native architecture, and all services can be quickly deployed to In addition to various cloud environments, storage and computing resources can be independently and elastically expanded according to demand to meet the requirements of large-scale data processing.
-
Yanhuang data platform provides standard SQL query, dashboard and other practical functions, which makes the platform more easy to use. Different from similar big data platforms in the industry that usually use customized search languages, such as SPL, Elastic query DSL, etc., Yanhuang data platform supports users to use standard SQL language for data query, in addition to supporting filtering, mapping, deduplication, aggregation, sorting, and association In addition to the basic SQL query capabilities, it also provides a large number of scalar functions and table function extensions, and also supports user-defined functions, which greatly reduces the learning threshold for users; Yanhuang Data Platform provides dashboard functions, based on platform integration The visualization library echarts, users can use various common visualization solutions, and store data analysis methods and analysis results in it, speeding up the transfer of data value within the enterprise.
-
Yanhuang data platform provides users with standardized products, which is convenient for users to quickly build data platforms on demand and realize data value. For enterprise users with relatively weak data infrastructure, Yanhuang Data provides a one-stop data platform solution, with end-to-end solutions from data import, data integration, data modeling, data storage, data analysis, data service, data visualization, etc. The capabilities of the terminal enable users to use it out of the box; for enterprise users who have strong data development capabilities and already have certain mature data development and management tools inside, Yanhuang Data understands the capabilities of each layer of the platform and provides API interfaces , allowing users to access external systems or tools based on the core data storage and search functions of the platform to achieve secondary development of the platform.
Typical customers: Zhongan Insurance, Shanghai Electric Power, Knowledge Planet, etc.
04. Selected certificate
