On August 26 , the China Software Innovation and Intelligent Operation and Maintenance Summit and Cloud Intelligence AIOps Community Conference , organized by the School of Software of Tsinghua University, the National Engineering Laboratory of Big Data System Software, and Cloud Wisdom, was held online.
At the meeting, Cloud Wisdom released the Moore platform, Hours service and GAIA data set, hoping to build an AIOps community with more social forces through the power of open source .
Cloudwise AIOps Open Source Community: http://www.cloudwise.ai
Conventional open source software releases often only release source code and usage methods. However, in this release of the AIOps community, not only the source code, but also " data x algorithm x computing power " is included , which realizes immediate enjoyment and value flow, and quickly releases the potential of researchers, developers and users. .
At present, cloud wisdom has open source projects including: Data-Processer (convenient, configurable test data generator) and CW-Zabbix (a Zabbix derivative version with 150 times better performance than Zabbix).
The open source software this time includes: Moore (a set of cloud service management platform, which provides developers with resource allocation - environment management - coding, deployment, and supports seamless access to open source services, components, and algorithms) and GAIA ( a cloud service management platform for developers). Analyze a full dataset of O&M issues such as detection, log analysis, and fault location).
At the same time, Cloud Wisdom also announced that it will open source the FlyFish project (a data visualization efficiency tool that separates the development side and the rendering side).
Intelligent operation and maintenance is the inevitable choice of the times
With the deepening of the digitalization process of enterprises, intelligent operation and maintenance ( AIOps ) ushered in continuous rapid growth.
Haibi Research and China Software Network have surveyed more than 2,000 companies. According to their report, as of 2020 , the number of APPs that Chinese companies need to use and maintain accounts for as high as 69% as of 2020 . Whether it is an enterprise or a government agency, the number of application systems that it needs to conduct business will continue to grow.
Behind such rapid development, there are two phenomena:
On the one hand, from the perspective of users, the service experience is becoming more and more convenient. Whether it is a large enterprise such as government affairs service, financial service, operator, energy and power, the information systems of almost all walks of life are gradually moving from internal support to direct-to-user service, providing "directly accessible services" through WeChat, mini programs, APP and other channels. "service experience.
On the other hand, from the perspective of system operation and maintenance, the topology, IT architecture, and process dependencies of the application system are becoming more and more complex, and the channel from the front-end business to the bottom layer of IT is becoming more and more complicated.
When an abnormality occurs in the front-end business, the troubleshooting process for problem location will be time-consuming and labor-intensive. If a systematic approach is not used for information system management, from the perspective of managers, information systems and service capabilities are often farther and farther apart.
Ning Gu, Executive Deputy Secretary-General of China Open Source Software Promotion Alliance, said that the IT structure of enterprises is becoming more and more complex, the volume of software is becoming larger and larger, and IT operation and maintenance is gradually evolving from traditional operation and maintenance process management to platform-based and intelligent evolution. The demands of IT practitioners for IT tools, platform capabilities, solutions, AI scenarios and available data sets are bursting in various industries, and the innovation of domestic software is imminent.
A survey conducted by Haibi Research and China Software Network on Chinese medium-sized enterprises shows that in 2020 , 54.1% of enterprises choose to purchase artificial intelligence applications, and 41.9% of enterprises choose to purchase big data and BI applications. This will continue to increase system complexity.
It is an inevitable choice of the times to move towards intelligent operation and maintenance.
Different from artificial intelligence, big data, blockchain and other technical systems, intelligent operation and maintenance is not a "new" technology, but an application and integration of intelligent technology based on intelligent operation and maintenance scenarios.
It is not practical to talk about "intelligent operation and maintenance" separately from the scenario. The core of intelligent operation and maintenance is to explore how intelligent technology can transform, serve, and adapt to the development of the operation and maintenance industry, and how to bring new ideas to solve problems for the operation and maintenance industry. As Cloud Wisdom has always advocated, "data-based, scenario-oriented, and algorithm-supported" is the way to win in the operation and maintenance industry.
Open Source Trends for Digital Innovation
In his speech at this conference, Ning Gu said that three years ago, Mark Anderson, a famous investor in Silicon Valley, said : " Software will eat the world . "
However, software won't necessarily eat the world, but open source will certainly create the future.
Taking the performance of open source in the new crown epidemic as an example, more than 6,000 contributors have participated in and supported more than 3,000 free and open source projects in response to COVID-19 on the open source collaboration platform GitHub , and these projects have been adopted by more than 140,000 people . Unique users viewed more than 2 million times.
Developers, experts and scholars from China, France, Germany, India, Italy, Japan, the United States, South Korea, the United Kingdom and other countries and regions have actively participated in the development of new coronavirus pneumonia epidemic maps, datasets, APIs , analysis, and application development on GitHub and other open source projects. It can be said that when countries are still in their own state of physical isolation, mankind has been united in the open source world.
Geng Hang, head of Mulan open source community operations and deputy secretary-general of China Open Source Cloud Alliance, said that in the future, open source will be more closely related to our enterprises and our lives.
At the same time, open source was mentioned for the first time in the " 14th Five-Year Plan " , and open source has become a major trend in the development of software engineering in China.
The intelligent operation and maintenance industry, which is still in the exploratory period, wants to stand out in the vertical track, and practice its own business model and future path in the development. Combining with " open source " may be a feasible way.
Therefore, Cloud Wisdom announced the official establishment of the AIOps community at the meeting, aiming to work with customers, users, researchers and developers in various industries to contribute and solve technical problems in the intelligent operation and maintenance industry, and build a healthy and win-win AIOps developer ecosystem.
Cloud smart AIOps community emerges as the times require
In the context of the rapid development of the AIOps field, the urgent needs of IT tools, platform capabilities, solutions, AI scenarios and available data sets are bursting in various industries.
In the realization of smart operation and maintenance, the data of all parties is extremely complex, and it requires the joint efforts of many enterprises, organizations and individuals to collect the data completely. Only on the premise of sufficient data can standardization and products be built to achieve the established value goals.
The establishment of the AIOps open source community can gather many outstanding developers to promote the sustainable development of AIOps and effectively solve the problems encountered in the innovation and development of operation and maintenance.
Gao Chitao, general manager of the R&D Department of Cloud Intelligence, mentioned at the meeting that an open source project requires a more comprehensive perspective, not only open source code and development methods.
When " data + algorithm + computing power " becomes an open source software with open governance and open services, it will be able to provide a strong growth impetus for various scenarios where the AIOps system and platform tools are applied to effectively solve the operation and maintenance needs.
The composition of the AIOPs community includes not only the AIOps user community, a user group composed of internal users, cooperative users and external users, but also internal and external communities.
Among them, the internal source community is composed of AIOps system designers, developers and some senior users. Technical products and core technologies will be open sourced as much as possible, and contributions and feedback will be accepted. The external community connects the external open source community, absorbs contributions through events, tasks, strategies, etc., and builds a brand in the AIOps field.
Specifically, the three key projects released by Cloud Wisdom are:
1. Hours open source service , an algorithm platform that integrates a variety of algorithms and basic capabilities
Hours is a set of algorithm technology stacks based on the core capabilities of cloud intelligence AIOps . It abstracts and unifies algorithms, and forms concepts such as processing units, algorithms, applications, data, data sets, interfaces, and tools.
It selects six types of AIOps commonly used algorithms for detailed examples: anomaly detection algorithm based on automatic threshold, anomaly detection algorithm based on the same chain ratio, fixed threshold anomaly detection algorithm, standard deviation-based ordinary detection algorithm, frequency domain analysis-based anomaly detection algorithm, linear regression prediction algorithm.
Hours is also a development framework. Algorithms are developed on the basis of Hours . They only need to follow the design logic of Hours . Algorithm researchers do not need to consider how to read data, how to preprocess data, how to encapsulate website API interface, how to visualize, etc. Details, just focus on researching good algorithms.
In this release, Hours ' core APIs are open to registered community members.
2. Moore cloud service management platform, greatly improving the work efficiency of developers
Using the Hour service through the Moore platform, you can quickly implement the service experience with just a few lines of code.
As the most important part of the cloud smart AIOps community, the Moore platform can not only provide developers with services such as AI data annotation services, AI training services, MySQL basic services, neo4j basic services, Hours SDK services, visual data platforms, etc. Developers provide resource allocation, environment management, coding, deployment, and support seamless access to open source services, components, and algorithms.
Personalized service requirements are connected in a process-based manner, allowing developers to focus on customization of business requirements and greatly improving the work efficiency of developers.
Wang Hongjiang, chief architect of the Cloud Intelligence Research Institute, said that under the 5A+4S theoretical system, the Moore platform is an open platform, and anyone can provide code and enjoy services for the platform;
The Moore platform is also a collaborative platform, and all participants can join the platform's R&D system to stimulate the vitality of the platform;
The Moore platform is also a platform with strong technological innovation capabilities. It should have the most advanced technical indicators and technological innovation capabilities in the world, and bring the world's advanced technical algorithms and advanced concepts to the Moore platform.
Wang Hongjiang also provided Moore platform cases including development environment CloudIDE , CV computer vision, NLU and smart campus solutions at the press conference.
3. The full data set "GAIA" is the hard work in the field of intelligent operation and maintenance
Li Feifei, a famous scholar, once said: " A major change that ImageNet has brought about in the field of AI is that people suddenly realize that the hard work of building data sets is the core of AI research. People really understand that data sets, like algorithms, are as important to research. vital."
Based on years of industry practice of cloud intelligence and the research results of the AIOps team, the full data set "GAIA" ( GAIA-Generic AIOps Atlas ) is used to analyze operation and maintenance problems such as anomaly detection, log analysis, and fault location, including 6,500 real-time indicator data, 7,000,000 Line log information and corresponding call chain data. At the same time, six typical scenarios and tasks are selected from the intelligent operation and maintenance algorithm system to describe and define.
From the perspective of the AIOps field, open source will promote the entire industry to become bigger and stronger; from the perspective of the entire software industry, the traditional pattern can no longer meet the diverse needs of industry users, and open source in the AIOps field will be conducive to the innovation and development of the digital economy.
Although GAIA has spent a lot of effort in the Cloud Wisdom Intelligence Research Institute, Cloud Wisdom only defines a small part of the intelligent operation and maintenance algorithm system. At this point, Cloud Wisdom hopes to use this to attract practitioners, researchers, and workers in the industry to contribute to the real maturity of intelligent operation and maintenance.
Currently, GAIA has been open sourced on GitHub :
https://github.com/CloudWise-OpenSource/GAIA-DataSet