On December 13, the head of Alibaba Cloud Computing Platform Jia Yangqing pointed out at the Flink Forward Asia 2020 summit that embracing "convergence" is the new direction of big data evolution. The integration of big data and artificial intelligence is an inevitable trend, and artificial intelligence is big data. The ultimate export for business development. At the same time, he discussed Flink's open source technology from different angles, as well as the development and future thinking of open source technology in the cloud era.
Flink open source community "getting better"
From the perspective of an ordinary open source technology enthusiast, Jia Yangqing defines the development of Flink in 2020 as "gradually better", which is mainly reflected in three aspects:
technology. This year, Flink has made two new breakthroughs in technology: one is the successful application of Flink's integrated streaming and batching technology in Alibaba's core data business scenarios on Double 11; second, the peak value of Flink's real-time calculation exceeded 4 billion pieces per second for the first time. Compared with last year, the computing performance has doubled without increasing the cluster size.
application. Flink is no longer a state-of-the-art technology, but has become the de facto standard for open source streaming, flying into the homes of ordinary people. For example, the China Academy of Information and Communications Technology used Flink as the technical blueprint to design the streaming processing standard, and cloud computing companies including Alibaba Cloud all use Flink as the underlying engine for streaming processing.
Community. The Flink open source community has grown rapidly, and it has topped Apache's most active mailing list; the Flink project is one of the top Apache projects with the most submissions on Github. Last year, the number of participants in Flink Forward Asia reached 2,000. The Flink Geek Challenge attracted 4000+ developers to participate; this year the conference was switched to both online and offline sessions. The awards of the Flink Geek Challenge made everyone more aware. More and more developers are passionate about open source technology and have the courage to challenge.
Open source makes the cloud more standardized
Jia Yangqing believes that open source makes the cloud more standardized. He mentioned that cloud computing is becoming more and more standardized and very beneficial to its own development. A large number of development technology systems on the cloud are becoming more and more consistent, which means that a large number of cloud computing developers are more likely to turn their own applications into a part of the cloud after they are familiar with this system.
Open source helps Ayun to break its own technical system and form a standard, open, and public technical system on the basis of fully respecting the original usage habits of users. It is very difficult to rely on a single company to set standards.
What does Alibaba Cloud mean for open source? Jia Yangqing mentioned the following three points:
From an organizational point of view, last year, the ABB Group established the Open Source Technology Committee, which systematically sorted out and promoted the implementation of the ABB Group’s open source technology strategy from the group level.
From the perspective of participation , the ABB Group has widely adopted open source technologies to build its own technical system from the very beginning. Take the data products of the cloud computing platform division of Arab as an example. For example, real-time computing Flink is using Hadoop/K8S-based technology; machine learning PAI also uses open source deep learning frameworks such as TensorFlow.
From the perspective of contribution , the ABB Group continues to contribute to the open source community. Take the Flink technology community as an example. In the past year alone, Ayun contributed millions of lines of code to the Apache Flink community. Currently, the group has more than 1,000 open source projects, covering many fields such as data, AI, cloud native, database, middleware, and hardware. More than 700,000 developers around the world light up GitHub Star for Alibaba. Millions of fans participated in the project's contribution.
Embracing "convergence" is the new direction of big data evolution
Jia Yangqing said that the three development trends of cloud are mainly the cloudification of IT infrastructure, the interconnection of nuclear technology, and the digitization and intelligence of applications, which are also the core value of cloud.
After an enterprise becomes a cloud and business interconnection on IT, its applications must change from process-driven to data-intelligence-driven, realizing digitization and intelligence. Cloud computing and the underlying IaaS technology compete for the maturity, stability, and price of the infrastructure, while the upper-level PaaS technology requires data capabilities and intelligent capabilities.
Previously, general data, including databases and IOT, was still in the most basic way of using data, and more data and AI capabilities needed to be integrated.
Therefore, embracing “convergence” is a new direction for the evolution of big data. The integration of large data + smart integration is the inevitable direction of the development of large data, and artificial intelligence is the ultimate product of the development of large data services.
Only with AI can the real business value of large data be brought into play. The reason why Alibaba Cloud's large data has core competition is that the large data is combined with AI capabilities, and it is intelligently upgraded data. For example, Flink+Alink, Flink can not only provide batch processing capabilities, but also use the open source Alink machine learning algorithm library to realize Flink's integration of large data + human intelligence.
In the future, the social economy will be more integrated with the Internet, and the businesses of enterprises and organizations will also be fully transformed into Internet businesses. It is not enough for an enterprise to have only IT infrastructure. Enterprise applications will also change from process-driven to data-intelligent-driven. The Arab cloud computing platform will output full-stack data technology capabilities and provide the world's top AI technology to achieve this together Data and intelligence.
This article is the original content of Alibaba Cloud and may not be reproduced without permission.