streaming protocol
Metaverse business scenarios put forward higher requirements for real-time and interactive streaming media transmission, which requires real-time interactive support on the basis of traditional RTMP, SRT, HLS, etc. Real-time interaction refers to communication and collaboration under remote conditions, which can be accessed anytime, anywhere, and real-time transmission of multi-dimensional information that integrates virtual and real, and an immersive interactive experience. As the next-generation Internet infrastructure, real-time interaction has achieved an important transformation from "online" to "presence", and will promote the upgrading and transformation of the Internet to the direction of the Metaverse with "presence" as the main feature. Several current mainstream technology directions as follows.
MPEG-DASH is an HTTP-based dynamic adaptive streaming technology launched by MPEG in 2012. It does not limit the encoding format and content, and can adaptively realize flexible switching between different bit rates according to the current bandwidth capacity and network performance, providing users with a low-jamming experience while ensuring the quality of the playback content. Currently, the MPEG-DASH protocol has become the main transmission protocol for panoramic video.
WebRTC is a real-time communication technology. It was open sourced by Google in the early days to realize web-based real-time communication capabilities. It was adopted as an official standard by the World Wide Web Consortium (W3C) and the Internet Engineering Task Force (IETF) in 2021. WebRTC can achieve ultra-low-latency, low-jamming real-time communication effects, but it lacks media transmission and interaction support for virtual reality content-oriented metaverse new media types. WebRTC-NV (Next Version) is the next generation of WebRTC, a standard after the current WebRTC1.0, and aims to support new use cases that are impossible or difficult to implement with the current WebRTC API, such as VR. Mainly from the four aspects of channel scalability, module maturity and perfection, acquisition scalability, and independent standards.
QUIC is a UDP-based low-latency general-purpose transmission protocol launched by Google. It optimizes the UDP protocol in terms of reliable transmission, security mechanism, and delay. Through encryption, flow control, congestion control and other technologies, it realizes More flexible, safer, and low-latency transmission. Currently, multiple browsers already support QUIC, such as Google Chrome, Microsoft Edge, Firefox, etc. At the same time, the protocol has been widely used in business scenarios such as mobile live broadcast, short video, and high-speed image file download.
Architecture diagram of computing power network supporting Metaverse:
To sum up, in order to meet the needs of the immersive experience of the metaverse in the future, the low-latency and efficient transmission of 3D visual media information is an urgent problem to be solved. Therefore, how to optimize the transmission protocol based on the characteristics of 3D visual information to achieve low-latency transmission will be the direction of further development of the transmission protocol.
media transfer
The Metaverse scene needs to support multiple types of video and audio data transmission, and has high requirements for real-time performance and interactivity.
3GPP SA4 is conducting standard research projects such as 5G_RTP, iRTCW, and FS_eiRTCW, which will aim at real-time transmission of immersive media and related metadata for immersive real-time services (such as XR services), as well as immersive real-time communication.
At the same time, MPEG has developed or is developing transmission standards for immersive media such as panoramic video, multi-view video, and point cloud data that support metaverse scenes, and uses the extended DASH/MMT protocol to transmit MPEG immersive media package files.
The IETF and W3C organizations adopted WebRTC as an official standard in 2021, and are currently working on the next-generation WebRTC standard.
The WebRTC working group is developing specifications such as media capture and media streaming and screen capture, while reviewing technical proposals to support new use cases for WebRTC; exploring the impact of edge computing on the web platform and related use cases and requirements, integrating network quality monitoring and predict. In order to support media transmission in different scenarios in the metaverse, standard research will be conducted on the function extension and optimized use of potential media transmission protocols (such as RTP protocol, WebRTC protocol), and the functional components of transmission, as well as potential, Emerging immersive media data formats provide flexible transmission/access mechanisms (such as space-based media access, perspective-based media transmission) for standard research to improve transmission efficiency, reduce terminal overhead, increase immersive experience, and meet different business scenarios.