//
From H.265 to AV1 to various self-developed codec standards, codecs have always been the "popular fried chicken" in the audio and video industry. This special session is even more about the Eight Immortals crossing the sea, each showing their magical powers. On July 29th, LiveVideoStackCon2023 Shanghai Station will hold a special session on AI and video codecs, which will gather the most popular codec topics and share them with you.
AI and Video Codec
From following to leading, China's ultra-high-definition video codec technology has gone through 20 years. From the beginning of formulating different video codec standards, such as H.264/265, AV1, VVC, to AI-based video coding technology, in the seemingly simple process of greatly compressing, encoding, transmitting, and decoding video data to restore clarity In the process, every small improvement is a big progress. Faced with increasingly complex algorithms, the demand for computing power of new video standards is increasing exponentially, and the dependence on hardware such as GPUs and ASICs is also increasing day by day.
TOPIC1 "Overview of AVS Perceived Lossless Compression Standard"
Yang Haitao Shanghai HiSilicon Technical expert in video field
Perceptual Lossless Compression PLC (Perceptual Lossless Compression) standard is the first attempt of the AVS working group in the field of visually lossless quality-level video image compression. "Lightweight image compression standard. In May 2023, the standard technical plan will be finalized and the FCD 1.0 text will be released. This speech will introduce PLC standard technology and application.
TOPIC2 "System on Chip (SoC) for Low Complexity Enhanced Video Coding (LCEVC)"
Rick Clucas V-Nova Senior Vice President of Innovation and Technology
Low Complexity Enhanced Video Coding (LCEVC) is a video coding format newly defined by the MPEG organization. V-Nova has implemented several innovative LCEVC hard decoding, by using the hardware modules and processing resources provided by the terminal platform to achieve safe and effective driver layer decoding, this solution can help terminal equipment manufacturers implement LCEVC on existing equipment Decoding, in order to deploy this new MPEG standard that can significantly reduce transmission costs for streaming media service operators.
This article will introduce the outline of LCEVC technology and two existing SoC-based LCEVC hardware decoding schemes implemented by V-Nova.
TOPIC3 "High Capacity Streaming Media Accelerator Card Solution Supporting AI Video Processing"
Xie Min AMD AECG System Solution Architect
It mainly introduces the basic architecture and functions of AMD Alveo™ MA35D video accelerator card, and the application of MA35D transcoding card in video solutions in various fields.
TOPIC4 "Deep Neural Network Compression Technology and Application"
Hu Haoji Zhejiang University Associate Professor
Deep neural networks often result in massive computational and storage resource consumption, hindering their deployment on mobile and embedded devices. Therefore, reducing the computing and storage resource consumption of deep neural networks has become one of the important issues in the application of deep learning.
In this presentation, we will first review the classic work in the field of deep neural network compression and acceleration, and then introduce the research work in this field in our laboratory, including: (1) pruning of convolutional neural networks; (2) compression Neural networks for specific tasks, such as face recognition, style transfer, and super-resolution networks; (3) Compression of Transformer networks; (4) Collaborative knowledge distillation of Transformer and CNN. In the era of large models, where are the opportunities and challenges in the field of deep neural network compression and acceleration? We will also conduct a heuristic discussion on the above questions.
TOPIC5 "Full link intelligent coding based on CPU server"
Xie Yi Intel Senior Software Architect
With the rapid development of live broadcast and short video services, the demand for high quality and personalization is increasing. In recent years, AI technology has been widely used in video pre-processing, and it tends to replace traditional numerical methods. Because AI video pre-processing consumes a lot of computing power, it has surpassed the traditional codec part, and is becoming a research hotspot in the industry. In order to solve the bottleneck of AI pre-processing computing power, an independent GPU cluster is often used as a separate module for AI reasoning. And latency poses a big challenge.
The fourth-generation Intel® Xeon® Scalable processor has a built-in Advanced Matrix Extensions (AMX), and its BF16/INT8 computing power has reached the capability of over 100 TOPS of a single CPU, which makes the entire pre-processing + encoding and decoding process in the It can be implemented on the CPU, which not only reduces the cost, but also reduces the operation and maintenance cost. In addition, Intel also provides a rich tool chain for performance optimization, making video codec optimization more intuitive and easier.
TOPIC6 "Prospects and Optimization of AV1 in RTE"
Wei Dai Soundnet Head of Video Codec
With the continuous development of RTC, high-definition or even ultra-high-definition video has gradually become a rigid demand in real-time interaction. The VP8, VP9 and H.264 initially supported by RTC lack the ability to support this type of video. In order to improve the subjective experience of high-definition and ultra-high-definition, RTC began to support two higher-generation coding standards, AV1 and H.265. attention of many developers.
This sharing will first introduce the characteristics of AV1 and its development history in RTC, and further combine the difficulties and pain points in the implementation of AV1 in the communication process, and analyze the advantages and future of AV1 in the field of RTC.
TOPIC7 "Optimization Strategy of Hardware Video Encoder for Internet Video"
Fan Yibo Fudan University Doctoral supervisor
Traditional hardware video encoders are mainly used in terminals, such as security IPC, mobile phones, cameras and other equipment. Hardware encoders in these fields pay more attention to PPA (Power Performance Area) optimization, and put the compression rate optimization in a secondary position. Therefore, these Hardware encoders are difficult to apply directly to Internet video. Internet video puts more emphasis on compression rate optimization, and requires extreme compression rate to save bandwidth. Usually, software encoding schemes will achieve better results. With the increase of Internet video's demand for resolution, delay, and computing intensity, traditional software encoding schemes are becoming more and more difficult to meet the demand, and the PPA advantages of hardware encoders are gradually improving. Hardware encoder-VPU chip research and development. This sharing is mainly divided into three parts: 1) Encoder technical characteristics for Internet video; 2) Hardware encoder architecture optimization strategy; 3) XK265 VPU beta release (based on U250 FPGA).
TOPIC8 "Tencent self-developed VVC codec Tencent266"
Tang Minhao Tencent Multimedia Lab Expert Researcher
VVC is the latest generation of video codec standard, and it is also the video codec standard with the strongest compression capability at present. With the great investment of major manufacturers, the VVC standard has gradually entered the stage of implementation.
This sharing will be divided into three parts. The first part introduces the characteristics of the VVC standard and some work of Tencent in the VVC standard; the second part introduces Tencent's self-developed Tencent266 decoder; the third part introduces Tencent's self-developed Tencent266 encoder.
▲Scan the QR code in the picture or click " Read the original text " ▲
Check out more exciting topics of LiveVideoStackCon 2023 Shanghai Station