On December 13, Jia Yangqing, head of Alibaba Cloud Computing Platform, pointed out at the Flink Forward Asia 2020 summit that embracing "integration" is the new direction of big data evolution. The integration of big data and artificial intelligence is an inevitable trend, while artificial intelligence is big data. The ultimate outlet for business development. At the same time, he discussed Flink's open source technology from different perspectives, as well as the development and future thinking of open source technology in the cloud era.
Flink open source community "getting better"
From the perspective of an ordinary open source technology enthusiast, Jia Yangqing defines the development of Flink in 2020 as "getting better", which is mainly reflected in three aspects:
Technology. This year, Flink has made two new breakthroughs in technology: first, Flink’s streaming-batch integration technology has been successfully applied on a large scale in Alibaba’s Double 11 core data business scenarios; second, Flink’s real-time computing peak exceeded 4 billion data per second for the first time. Compared with last year, the computing performance has been doubled without increasing the cluster size.
application. Flink is no longer a high-level technology, but has become the de facto standard for open source streaming, flying into the "home of ordinary people". For example, the China Academy of Information and Communications Technology uses Flink as a technical blueprint to design streaming standards, and cloud computing companies, including Alibaba Cloud, use Flink as the underlying engine for streaming.
community. The Flink open source community has grown rapidly and has reached the top of Apache's most active mailing list; the Flink project is one of the top Apache projects with the most submissions on Github. Last year, the number of participants in Flink Forward Asia reached 2,000, and the Flink Geek Challenge attracted 4,000+ developers to participate; this year, the conference was switched to online and offline at the same time, and the awards of the Flink Geek Challenge let everyone see more More and more developers are enthusiastic about open source technology and have the courage to challenge.
Open source makes the cloud more standardized
Jia Yangqing believes that open source makes the cloud more standardized. He mentioned that cloud computing is becoming more and more standardized, which is very beneficial to its own development. A large number of development technology systems on the cloud are becoming more and more consistent, which means that after a large number of cloud computing developers are familiar with this system, it is easier to turn their applications into part of the cloud.
Open source helps Alibaba Cloud break its own technology system and form a standard, open, and public technology system on the basis of fully respecting users' original usage habits. It is very difficult for a single company to set standards.
What does Alibaba Cloud mean for open source? Jia Yangqing mentioned the following three points:
From an organizational point of view, last year, Alibaba Group established an open source technology committee to sort out and promote the implementation of Alibaba's open source technology strategy at the group level.
From the perspective of participation , Alibaba Group has widely used open source technology to build its own technical system from the very beginning. Take the big data products of Alibaba Cloud Computing Platform Division as an example. For example, Flink for real-time computing uses a large number of Hadoop/K8S ecological technologies; machine learning PAI also uses open source deep learning frameworks such as TensorFlow.
From the perspective of contribution , Alibaba Group continues to contribute to the open source community. Take the Flink technical community as an example. Last year alone, Alibaba Cloud contributed millions of lines of code to the Apache Flink community. At present, the Group has more than 1,000 open source projects, covering big data, AI, cloud native, database, middleware, hardware and other fields. There are more than 700,000 developers around the world who light up GitHub Star for Alibaba, thousands of Thousands of enthusiasts participated in the project contribution.
Embracing "fusion" is the new direction of big data evolution
Jia Yangqing said that the three major development trends of cloud are the cloudification of IT infrastructure, the interconnection of core technologies, and the dataization and intelligence of applications, which are also the core values of cloud.
After enterprises go to the cloud for IT and Internet business, their applications must change from process-driven to data intelligence-driven, realizing dataization and intelligence. Cloud computing and underlying IaaS technologies compete for the maturity, stability, and price of infrastructure, while the upper-layer PaaS technologies require data capabilities and intelligence capabilities.
At present, pan-big data including databases and IOT is still the most basic way of using big data, and more big data and AI capabilities need to be integrated.
Therefore, embracing "integration" is the new direction of big data evolution, the integration of big data + artificial intelligence is the inevitable direction of big data development, and artificial intelligence is the ultimate outlet for big data business development.
Only with AI can big data play its real business value. The reason why Alibaba Cloud big data has core competitiveness is because big data is big data combined with AI capabilities and intelligently upgraded big data. For example, Flink+Alink, Flink can not only provide the integrated processing capability of stream and batch, but also realize the integration of big data and artificial intelligence of Flink with the help of the open source Alink machine learning algorithm library.
In the future, the social economy will be more integrated with the Internet, and the business of enterprises and institutions will also be fully transformed into Internet business. It is not enough for an enterprise to have IT infrastructure alone. Enterprise applications will also be transformed from process-driven to data intelligence-driven. The Alibaba cloud computing platform will output full-stack big data technology capabilities and provide the world's top AI technology to achieve this together. Data and intelligence.
This article is original content of Alibaba Cloud and may not be reproduced without permission.