Introduction: The IT technology competition begins!
Author | Song Hui
Listing | CSDN (ID: CSDNnews)
Currently, the 2022 Qatar World Cup is in full swing and is in an even more intense and critical semi-final stage. As the world's top football event, the World Cup has ignited the enthusiasm of football enthusiasts and fans around the world. At the same time, behind the game, how to build a powerful technical service and support system to ensure high concurrency and high quality during live broadcast of the game. High-definition, high-smooth global synchronization and distribution of a large amount of real-time interactive content is definitely a real challenge for technology service providers.
Fortunately, Douyin Group has become the rights-holding broadcaster of the 2022 World Cup and the live broadcast partner of China Central Radio and Television. It also owns the live broadcast + short video copyright. It is an important force in content services for this World Cup’s exciting events and has won a large number of With the attention and use of fans and audiences, the live broadcast room is as popular as the Spring Festival Gala live broadcast and the live video broadcast of the hot e-commerce season. According to CSDN tracking, behind the Douyin World Cup service system is stable, high-definition live broadcast technology support provided by Volcano Engine, a cloud service platform owned by ByteDance, as well as various novel online interactive products that are close to fans and audiences. Trendy new way to play. In addition, Volcano Engine Video Cloud also provides CDN services for Douyin and CCTV's live event broadcasts. These are "insider" technologies that are worthy of the attention and understanding of developers in addition to hot events.
Behind the ultra-high-definition World Cup live broadcast comes from the self-developed technical support of Volcano Engine
Audio and video technology has been developed for many years, and the most intuitive feeling for the audience to enjoy a live sports event is the visual experience of the picture. The most highlighted experience upgrade in the technical service of Douyin World Cup live broadcast is that the Volcano Engine Multimedia Laboratory uses a number of core video technologies and algorithms to achieve ultra-high definition, providing fans and audiences with a high-quality viewing experience. .
Specifically, the video encoding algorithm for large-scale sports events such as football must not only ensure the clarity and smoothness of live content in scenes of high-speed movement and complex textures , and ensure the user's viewing experience, but also take into account constraints such as bit rate and delay. Sensitive indicators at the network transmission level . The BVC encoder developed by Volcano Engine takes on the important task of encoding Douyin World Cup live video. It is deeply optimized for sports events and HDR scene videos. It not only obtains relatively better picture quality and higher quality at an average bit rate lower than the industry. Rich details, and significantly ahead of the industry level in terms of encoding latency and other aspects.
In addition to the core video encoding task, the Volcano Engine has designed an adaptive ToneMapping algorithm for the HDR (high dynamic range image) content of the World Cup . Nowadays, mainstream large-scale events such as the World Cup have adopted HDR shooting methods. HDR film sources have a wider color gamut and a larger dynamic range. However, many terminal display devices do not fully support HDR signal playback. In the past, traditional TonaMapping algorithms such as Reinhard, Filmic or Uncharted 2 used a fixed curve method to convert HDR video into SDR video. The resulting conversion pattern was fixed and could not be adapted to large-scale sports. The ever-changing scene of the competition. Because the dynamic range span of large-scale competitions is large and the brightness of the venue's lights/grass/players is significantly different, the player information that the audience is interested in is actually concentrated in the dark area, resulting in the SDR signal processed by traditional ToneMapping being too dark. In the World Cup live video, the Volcano Engine uses the content-adaptive ToneMapping algorithm to dynamically map the brightness information of video frames to achieve better conversion effects.
Left: hable algorithm, right: content-adaptive ToneMapping to optimize the brightness of various images at the World Cup match
In addition to the HDR content of the live broadcast signal, the Volcano Engine uses a color enhancement algorithm to perform corresponding equalization processing on the video to optimize the subjective effect by analyzing video brightness/color/contrast and other information for camera images with only SDR signals. .
Color enhancement before and after contrast, you can see the French team players and the background auditorium color contrast enhancement, highlighting the players
Color enhancement before and after comparison 2, the main color of the Canadian auditorium is more vivid after optimization
In addition, in the Douyin World Cup live broadcast, the Volcano Engine also used video optimization technologies such as adaptive sharpening, spatiotemporal domain noise reduction, super-resolution and other image quality enhancement technologies. After optimization, the JND subjective evaluation result was 1.64 (the JND score interval is - 3-3, if it is greater than 1, it is significantly positive). From the perspective of objective evaluation, it can be seen that the optimization effect is significant.
Comparison before and after image quality enhancement and optimization, JND subjective evaluation shows that the effect is significant
Watch football in the cloud while chatting, audio and video technology inspires more interactive gameplay in the World Cup
In recent years, real-time audio and video RTC technology has been applied in various industry fields and scenarios. The reason behind it is to allow ordinary users to achieve more interactive audio and video experiences. This year's World Cup in Qatar has far exceeded the amount of online interaction among viewers in previous important events. In addition to conventional interaction in the form of pictures and texts, friends can also "watch the game online in the cloud" from different places at the same time, providing an exciting and interactive sense of participation. It’s full while also adding another layer of sports fun.
For example, Douyin has launched a chat-while-watching method in the live broadcast of the 2022 World Cup. Each viewer can create their own chat channel while watching the game, invite friends to watch the game online together, and express their opinions while watching and chatting according to the battle situation. , is a further interest interaction and social interaction. However, in order to obtain a good user experience in this complex audio scene with multiple sound sources, more hard-core audio and video technologies must be used to support it, such as echo cancellation, adaptive volume equalization, intelligent audio dodging, etc.
For example, users on the Douyin platform watch football passionately with their friends and cheer loudly, usually using audio playback. At this time, the microphone will not only collect the user's voice, but also the sound of the field and commentary during the live broadcast, generating an echo. Echo cancellation is an important audio optimization technology in RTC scenarios. Volcano Engine RTC uses audio hosting and adopts self-developed software intelligent 3A . On the basis of traditional algorithms, it introduces an echo suppression algorithm based on deep learning to effectively eliminate Echo in dual-talk scenarios, while avoiding problems such as vocal stuttering and poor sound quality caused by excessive echo cancellation, can ensure the best sound quality performance of live events and enhance the communication experience.
Another highlight of experience optimization is intelligent audio ducking. In the past live broadcasts of events, the audience only listened to the commentary and live audio in one direction, but in the scene of watching and chatting at the World Cup, friends chatted about the game together, especially sharing the cheers with friends at the wonderful goal scoring moment. The live broadcast of the game and the commentary sound have become a kind of "sound interference", so in "Watching and Chatting", balancing the volume of the live broadcast sound and the user's voice in the chat room has become a key point to improve the user experience. Volcano Engine RTC adopts an adaptive volume equalization strategy , which can automatically adjust the volume ratio of human voices based on the live broadcast volume, so that users can speak clearly. At the same time, in order to better solve the problem of user voices being obscured by live broadcast sounds, Volcano Engine RTC provides an intelligent audio avoidance function , which detects accurate human voices through AI voice. When friends talk and discuss, the live broadcast sound on the user side will automatically reduce. When everyone concentrates on watching the ball without talking, the live broadcast sound will return to normal volume, achieving a very clear and natural audio experience.
Of course, in addition to these meticulous audio and video optimization technologies to improve user experience, the World Cup live broadcast, as a super copyrighted event, currently has a peak viewing rate of more than 160 million people on Douyin. As the game enters the semi-finals, the moments of popular events Concurrency pressure and chat user data will also continue to trend higher. The smooth operation of live event services requires the backend to meet ultra-high concurrency and stable performance and operation and maintenance guarantees. Through the SFU+MCU integration solution , the Volcano Engine RTC team can, on the one hand, reduce the number of concurrent video streams in the entire link of the RTC system and expand the concurrent capacity of the RTC system. On the other hand, it ensures that users can communicate smoothly on the microphone at any time, while effectively reducing the number of viewers. Competing users’ device performance consumption pressure.
The quadrennial World Cup is a major event in the football and sports world. While the world's top teams compete, it is also an IT technology competition and a real-time audience experience competition. In the 2022 Qatar World Cup live broadcast, we saw that solid and excellent technology and exploration of innovative and novel product applications allowed Douyin and Huoshan Engine Video Cloud to win real good experience and reputation from fans and viewers, which made us truly appreciate To the meaning of using technology to change the world and using technology to pursue a better life. Audio and video technology is still developing forward, and innovative gameplay and popularity will continue to play significant value in more scenarios. Developers are welcome to pay more attention, and CSDN will continue to report and introduce it.