From standard to open source, Alibaba's "innovation responsibility" of Taobao technology

Recently, China's network communication standards and open source have made important progress in many aspects. Alibaba (China) Co., Ltd. and the domestic academic community submitted the "Interactive Low-Latency Internet Multimedia Transmission System Requirements" (GRTN) international standard project proposal was officially approved, and Alibaba's Taobao technical team led the self-developed IETF QUIC standardization protocol The library XQUIC was also officially open sourced on January 7 this year. These two achievements are important advances in the ICT field, and they are of great value to user experience such as low-latency interaction of live broadcasts and stable video viewing.

Technology media commentator Huang Haifeng interviewed Wu Pengqiang and Liu Yanmei, two technical experts of Alibaba's Taobao technology, to discuss the value and advantages of GRTN and XQUIC, and to analyze in-depth the Taobao technology's focus on end-user experience and the determination to actively practice technological innovation. .

The following is the original interview

In recent years, China's scientific and technological strength has achieved more and more achievements on the international standard stage. In addition to becoming the main force of innovation in mobile communication fields such as 5G and 6G, the author has recently paid attention to the important progress of China's network communication standards and open source.

First, on January 17-28 this year, at the ITU-related meeting of the International Telecommunication Union, Alibaba (China) Co., Ltd. and the domestic academic community submitted the "Interactive Low-Latency Internet Multimedia Transmission System Requirements" (GRTN) international standard. The project proposal was officially approved.

Second, on January 7 this year, the IETF QUIC standardized protocol library XQUIC, which was led by the Alibaba Taobao technical team, was officially open sourced. Different from other current open source QUIC protocol stacks, XQUIC provides multi-path QUIC access capability, and the multi-path QUIC technology is jointly developed by Taobao Technology and the XG Lab of Dharma Academy.

In November last year, the multi-path QUIC draft was approved by 46 votes (48 votes in total) in the relevant meeting of the international standards organization IETF, and was officially incorporated into the QUIC Working Group draft in February this year, opening the RFC (Request For Comments) way.

The approval of the GRTN proposal and the official open source of XQUIC are all important developments in the ICT field. Friends outside the circle may not understand, what are these two standard technologies and why are they important? What does it mean again?

In fact, they are not only of great value to Alibaba's various businesses, but also of great value to user experience such as low-latency interaction of live broadcasts and stable video viewing. Moreover, we can feel from these two major technologies that Alibaba pays attention to the end-user experience, actively implements technological innovation, and is willing to contribute to the development of the industry!

GRTN: Breaking through the technical bottleneck and competing for the right to speak internationally

The value and advantages of GRTN can be analyzed from the following aspects.

First, meet the needs of interactive live broadcasting.

With the popularization of 5G, live shopping, online teaching, high-definition live broadcasting, entertainment interaction, and online meetings have become mainstream living and working methods. In the past two years, the epidemic has accelerated this trend again. More people choose to study, work, play and consume online, and have frequent two-way interactions in live broadcasts.

But many people are troubled by the delay and lag in live broadcast. Different from the one-way streaming media transmission of traditional live streaming applications, interactive live streaming applications, such as Taobao Live, require an ultra-low-latency two-way streaming media transmission mechanism to meet the needs of interactivity.

图片

What should be done about this? Internet video live broadcast applications, usually using a content distribution network (CDN) for video distribution. However, due to the traditional CDN architecture and protocol limitations, the end-to-end streaming media transmission delay is usually between 5-10 seconds, which cannot meet the requirements of Internet interactive live broadcast applications.

In this regard, Alibaba Da Taobao Technology and other teams and academia jointly researched key problems, built a bandwidth discrimination estimation model for network state cognition, and proposed a reinforcement learning-driven interactive video flow control method, which realized the high dynamic and strong time-varying network. Smooth video transmission.

Second, the application advantages are obvious.

Compared with the international mainstream method, the freeze rate of live video using the new technology is reduced by 60.65%, and the video throughput is increased by 16.52%. The research results have been published in ACM MobiCom, the top international conference in the field of mobile computing for three consecutive years.

In order to effectively reduce the end-to-end delay of live broadcast, Alibaba created a global realtime transport network GRTN (Global Realtime Transport Network) based on a distributed cloud platform, and launched GRTN for the first time during Double Eleven in 2020.

It is understood that GRTN adopts a hybrid network architecture combining hierarchical network and peer-to-peer network, the media transmission link is shorter, the whole link adopts UDP-based RTC protocol with lower delay, and uses deeply customized dynamic routing and transmission control. , edge computing, two-way real-time signaling network and other mechanisms, achieve ultra-low latency streaming media transmission capabilities within 1 second, and can ensure the service quality of large-scale Internet interactive video live broadcasts.

从5-10秒的互动直播延迟降低到1秒以内,GRTN可谓是极为出色的技术突破。业界虽然在推出类似形态的产品,但相比GRTN,其系统架构和媒体系统还没有完全的融合,只是多个系统和服务的叠加,并没有完成新系统的改造升级。

再次,技术生态发展很快。

经过阿里多个团队多年的协作研发和打磨,GRTN已经应用到淘宝、钉钉、优酷、Lazada等多个平台的低延时直播业务中,并帮助众多国内外用户提升互联网直播体验。

目前,GRTN所服务的域名数已达到数千个,基于GRTN构建的直播应用包括电商直播、视频会议、在线教育、视频通话、云游戏等全场景互联网直播业务。

面向行业和未来,大淘宝技术团队的技术专家吴鹏强告诉笔者,为了将GRTN的设计理念和新一代CDN系统架构方案,普惠给视频领域公司和从业者,大淘宝技术因此推动将GRTN系统的架构和实现做成标准,在给业界提供解决方案的同时,也希望视频行业一起参与丰富和完善整个标准的内容,加速整个视频行业在低延时传输方面的软硬件的技术和体验的升级。

图片

大淘宝技术团队技术专家吴鹏强

最后,提升我国标准话语权。

从生态角度看,GRTN是联合国内学界进行的关键技术创新,致力于实现在流媒体技术领域对欧美国家的弯道超车。

以GRTN技术为基础,阿里巴巴(中国)有限公司和学界在2022年1月的ITU-T SG16全体会议上成功立项了《可交互低延时互联网多媒体传输系统需求》国际标准,获得了中国电信、中国联通、中兴通讯等国内传统CDN技术标准主导单位的支持,并在会上与英国、日本、加拿大、巴西等国代表进行了深入的交流和激烈的讨论,最终赢得了相关国际同行的认可和支持。

该国际标准项目的成功立项,标志着我国在流媒体技术领域迈出了重要的一步,为争夺新一代多媒体传输系统国际标准制定的话语权打下了重要基础。

XQUIC:具备四大优势,填补国内开源空白

与GRTN类似,我们也可以从需求、创新、部署、生态等角度获得XQUIC(阿里自研的IETF QUIC标准化传输协议库)的价值。

首先,短视频等业务渴求稳定网络。

你是否曾经经历过看视频刷剧刷的正嗨,突然发现视频变得很卡,怎么重连也没有用?当你打着语音电话从商场走向停车场,电话一下子就断了,必须要拨号重连?当你想要争分夺秒地在高速上办公,但是发现邮件怎么也发不出去?

上述问题就是因为这些场景的网络信号太弱。由于无线网络天生的频谱限制,无线信号的覆盖不足,多用户间的相互竞争资源,高移动场景下频繁的基站切换等等,都可能导致“弱网”的频发。克服弱网对于用户的体验至关重要。

为此,大淘宝技术团队与达摩院XG实验室,共同研发了MPQUIC多路传输技术(基于XQUIC协议库),希望从根本上解决了由单路径弱网带来的用户体验问题,也解决5G中前期网络覆盖不足挑战。

据悉,MPQUIC技术基于阿里巴巴提出的IETF多路径QUIC草案框架实现(目前已经作为工作组草案进入IETF QUIC WG)。MPQUIC技术主要通过在网络传输层实现多径传输,本质是通过多条网络链路(5G/4G、WiFi)提升传输吞吐量和鲁棒性(网络的稳定和健壮程度)。

其次,XQUIC具备四大技术创新优势。

要想了解XQUIC优势,我们可以追溯到QUIC(基于UDP的可靠传输协议)。谷歌2013年提出了QUIC。2016年,QUIC在标准组织IETF开始标准化。目前QUIC已成为移动互联网中最核心和关键的传输技术,超过50%的Chrome浏览器流量和75%的Meta流量都在使用QUIC进行传输。

图片

QUIC有多家企业进行开源实现,阿里巴巴从2014年开始关注QUIC,开始迈向标准协议+自研实现的道路,从QUIC技术的追随者,快速成长为QUIC技术的创新者,并在多路径QUIC技术实现上取得了突破。

多条路径同时传输数据并不容易。业界此次前有基于传统内核态多路径解决方案MPTCP等多路径实现,如Apple的Siri、Apple Music等场景,还有荣耀的Link Turbo。但MPTCP等迟迟未能被业界大规模采用,除了内核态部署带来的复杂度之外,多路传输的核心问题MP-HoL一直没有得到非常好的解决。

基于传统MPTCP的方案,在路径切换、选择和聚合时存在明显的性能瓶颈,不能完全解决如下问题:由于是内核实现而无法为应用场景提供定制优化;异构网络带来带宽不可预测;带宽利用效率低、流量成本高。

相较于传统的MPTCP,而阿里主导设计和实现的MPQUIC,基于quic-v1进行的最小扩展,在兼顾多路径传输效率的同时最小化实现成本,具有用户态部署、高性能、低成本以及轻量化等四大优势。

MPQUIC除了有易于部署迭代等优势,同时作为用户态协议栈,也更容易结合应用层需求进行调度算法和Reinjection策略的优化。

据悉,在淘宝移动端应用测试结果表明,在弱网下,MPQUIC技术可以实现短视频分片平均下载耗时减少15.03%,视频分片下载弱网耗时降低25.28%。此外,在旅途中,使用MPQUIC的用户可以同时利用WiFi热点与手机LTE,在高移动性场景下仍然保持流畅的视频观看体验。

再次,多处部署,久经考验。

阿里巴巴大淘宝技术团队在2018年底开始研发XQUIC实现,2021年8月正式对外发布。如今,XQUIC针对IETF QUIC协议的实现,已相对成熟,在淘宝体系里也做了大规模应用来验证其稳定性,在今年1月正式开源。

据悉,目前淘宝里面的导购场景RPC请求、短视频下载与文件上传等核心场景中应用了XQUIC实现网络加速。淘宝短视频和逛逛等应用在今年逐步使用XQUIC。此外,阿里内部其他APP,如闲鱼、AliExpress等,也开始尝试使用XQUIC进行网络体验优化。

阿里巴巴大淘宝技术团队也在应用实践中,不断做新功能迭代/AB实验和Congestion Control算法调优等工作。

最后,正式开源,繁荣生态。

在将IETF QUIC在移动端上进行较多应用后,阿里巴巴正在弥补当前国内缺少相对成熟的开源实现这一空缺。现在较成熟的开源实现大部分还是以 Google、Meta等国外企业为主。阿里希望通过开源等方式,也可以为行业尽一份微薄之力。

阿里XQUIC技术面向的业务域,包括交易导购、视频传输和虚拟现实等。谈及未来,大淘宝技术团队的技术专家刘彦梅告诉笔者,希望更多互联网大厂参与到QUIC技术建设中,来提供更多应用场景对协议栈能力设计的需求输入。

图片 大淘宝技术团队技术专家刘彦梅

对于XQUIC开源社区,后续也计划把非可靠传输/半可靠传输场景纳入研发Milestone,将结合社区开发者诉求,逐步纳入更多传输需求场景。

笔者说

大淘宝技术成国内技术创新重要力量

业界有句话,一流企业做标准,二流企业做品牌,三流企业做产品。可见技术标准和技术实现的重要性。在互联网领域,网友说阿里巴巴大淘宝各项业务是商业创新,对行业创新贡献小。

事实真是如此吗?从这GRTN以及XQUIC两个自研创新可以看出,阿里巴巴并非如业界所说的那样。

比如作为阿里巴巴新零售技术的王牌军,大淘宝技术支撑淘宝、天猫核心电商业务,服务9亿用户与各行业1000万商家,保障了13次双十一购物狂欢节的成功。

大淘宝技术成为内部业务的定海神针,也在将技术能力,赋能给更多行业企业,提升用户体验。通过调研和交流,我们得到如下结论。

其一,阿里大淘宝技术牵头推动的GRTN以及XQUIC,相比传统技术具备较多优势,在直播交互等场景表现出色,在标准层面有新进展,将提升更多用户体验,加速行业升级。

其二,我们在撰文过程中,不只是感受到GRTN以及XQUIC的优势,更是看到大淘宝技术在网络技术方面的巨大投入,为行业标准形成贡献力量,并在主动将先进技术开放,赋能给更多企业。 

其三,能力越大,责任越大。我们期望行业领头者探索科技研究的无人区。大淘宝技术的创新投入巨大,研究内容既有广度,也有深度,触达网络底层。这个深度以前很少有互联网企业愿意投入。

其四,这些网络底层投入和工作,不仅对阿里多个淘宝直播等业务带来了技术支持和商业价值,更重要是在外部获得标准组织肯定,为更多互联网、视频、终端等生态内企业带去价值,推动直播、短视频等行业发展。

其五,从大淘宝技术,我们看到阿里巴巴的技术创新,更加注重用户真实体验,是从用户需求层发力(而非网络内核层面),进行软件算法、调度算法优化,发挥自身to C优势。

其六,GRTN以及XQUIC在阿里的业务中应用多,在其他厂商还比较少,还走在标准化、规模应用的道路上,要经历一个循序渐进的过程,未来几年在更大范围使用还有一些挑战。

基于此,我们呼吁更多视频、直播等软硬件领域的企业和从业者,可以更多关注GRTN以及XQUIC的发展,为新技术的规模落地、为新体验普惠大众而建言献策。

Guess you like

Origin juejin.im/post/7085248561641685006