After 20 years of development, the AVS standard has achieved a breakthrough in domestic standards from following to leading in the generation of AVS3. The Boya Ruishi team has been deeply involved in the formulation of the AVS3 standard, and the developed 8K AVS3 real-time and offline codec equipment has strongly supported the landing and promotion of my country's 8K ultra-high-definition video industry.
This sharing will be divided into three parts. The first part focuses on the development history of the AVS standard and the current status and latest progress of the AVS3 standard; the second part focuses on some key technologies in the entire link of 8K live broadcast, on-demand, and content production , challenges, and some of our landing ideas and practical experience; the third part shares the technology and market direction that Boya Vision will focus on in the 8K ultra-high-definition field in the next few years, as well as the plans and thinking of AVS ecological construction.
Text/Han Wei
Edit/LiveVideoStack
Hello everyone, the topic I share today is: 8K content production and transmission application practice supported by AVS3 . As we all know, the current domestic 8K industry lacks content and content creators to enrich the content. So we made 8K super resolution and enhancement to make up for the lack of content.
I am Han Wei from Boya Vision. Boya Ruishi is a technological achievement transformation company incubated by the National Engineering Research Center for Video and Vision of Peking University. It is mainly committed to my country's video codec standards, AVS industrialization and internationalization. Under the leadership of Academician Gao Wen, the AVS standard has been developed for 20 years. There are three generations and four standards, and all of them have become national standards and industry standards. AVS2 is a military standard and an international standard in IEEE. At the same time, in July this year, AVS3 became the recommended standard for European DVB in the ultra-high-definition field.
The use of MPEG-2, H.264, and H.265 will face high patent fees. Therefore, due to the needs of national strategy and industrial security, the construction of video codec standards with independent intellectual property rights in my country has been promoted since 2002.
Today's content is divided into three parts:
-01-
The development history of AVS standard and the latest progress of AVS3
Firstly, it introduces the development process of AVS standard and the latest progress of AVS3.
Since my country joined the WTO in 2001, it has become the world's largest DVD exporter, but many DVDs will be charged high patent fees due to the use of MPEG-2, H.264 and other video coding standards when they go out to sea.
The Xiangshan Conference is a conference on China's major scientific and technological developments held in Xiangshan every year by the Ministry of Science and Technology and the Chinese Academy of Sciences. In the Xiangshan Conference in March 2002, the Ministry of Industry and Information Technology pointed out that a Chinese video coding standard must be established in response to the DVD incident, so as to compete with MPEG-2 and H.264 in the United States.
The AVS standard is proposed based on MPEG-2 and H.264 of the United States. The earliest is MPEG-2, jointly proposed by Japan's Sony, Bell Labs, Columbia University, Philips, etc. This codec standard has promoted the basic development of global audio and video. After that, South Korea and Germany invested a lot in the research and development of key technologies in this field in the late 1990s, resulting in the 264 generation standard.
After the DVD incident promoted the Xiangshan Conference, MP3 export patent incidents, set-top box patent incidents and digital TV standard incidents occurred one after another. For example, my country accounts for 70% of the world's TV panel exports. If patent fees are collected from manufacturers, HEVC has at least three patent pools, and each patent pool charges at least 1.5 US dollars. This is a very big threat to our industry.
The entire industry market related to standards was about 500 billion from 2002 to 2006, 900 billion in 2008, and more than trillion after 2018. If our country can have its own standards and even export, it will be a huge industry. Therefore, the establishment of my country's own standards is not only to ensure the safety of my country's strategic national industries, but also to promote my country's standards and industries to gain profits. This is a very meaningful thing.
The AVS standard had a very good design when the AVS working group was established. The reason is that the MPEG organization is now scattered, and there are too many companies in it, so there is no unified opinion. The collection of their patent fees is one of the reasons why the H series standards have a lot of resistance in the promotion process.
Therefore, when we were designing, the AVS working group produced standards from the very beginning. Many universities participated in the production of standards, such as Peking University, Zhejiang University, and the Institute of Computing Technology of the Chinese Academy of Sciences, as well as industries, such as Haikang, Byte, Ali, Tencent, Mi Goo wait.
In addition, the AVS Patent Pool Committee is established. When various organizations join us to contribute patents, they must sign an agreement to unify the charging standards and use a patent pool to charge. At present, it is promised that the entire industry chain will only be charged once, only terminal chips will be charged, and only RMB 1 will be charged at a time. This is more regular and cheaper than H.265, which will help the entire ecology to be promoted internationally.
At the same time, the AVS Industry Alliance was established, including nearly 200 domestic research institutes and enterprises, to jointly promote the implementation and industrialization of the entire standard, which is a very big advantage.
How is AVS made? In August 2002, the first meeting was held in Beijing to establish AVS; at the seventh meeting in December 2003, the AVS1 standard was formulated and completed; later, when CCTV wanted to make AVS a mandatory standard for radio and television, we made a version of AVS+ , AVS+ benchmarked against the compression efficiency of 264; AVS2 was launched at the 42nd meeting in Guilin in September 2012, and the benchmark was 265; in 2015, AVS2 was released at the 55th meeting, and the compression efficiency of AVS2 and H.265 is equivalent. But it was several years too late. Later, AVS2 was a strong and national standard for radio and television, and it also entered the military standard. In addition, AVS2 also entered IEEE, which is an international standard.
In 2018, the formulation of AVS3 was launched, oriented to 8K, which is similar to the current 5G and lithium batteries. In the past, we have been following and catching up. We hope that this time we can make it before the launch of the 266 standard. Therefore, in January 2019, more than ten months later, AVS3 was formulated, and a total of about 6,000 technical proposals were considered and more than 2,000 documents were output. item.
In March 2019, the AVS3 standard was released. Our laboratory was jointly established with HiSilicon and the Academy of Sciences, so in September of the same year, we released the first decoding chip Hi3796CV300 at IBC in the Netherlands. So far, we have opened up the situation from the standard to the industry, ahead of 266.
Although we are not dominant in the field below 4K high-definition, but in the field of ultra-high-definition, we stand ahead of the US standard for the first time. This is also related to my country's strategic security. my country accounts for 50%-70% of global panel exports. This kind of high power consumption and labor-intensive industry is definitely not a long-term solution. In the future, it may be transferred to Vietnam, India and other labor forces. lower cost countries. If the industrial chain is not developed along the way, the technological advantages accumulated with great difficulty will disappear.
As shown in the figure, AVS supports major applications such as domestic high-definition channels
The picture shows the process of AVS3 standard formulation and some key achievements, as well as a group photo of some key members during the closed development period, including Academician Gao Wen, Professor Ma Siwei, etc.
AVS3 took 18 months from the formulation of technical standards to the launch of the chip. It gathered various forces to formulate technical standards, chip terminals and system applications, forming a complete ecological and application system.
AVS3 is oriented to 8K, so the compression efficiency must be sufficient. The picture shows the evolution of compression efficiency in the field of video coding over the past 30 years.
Compared with AVS3 and HEVC/H.265, the comprehensive performance of ultra-high-definition video compression YUV has been improved by more than 36%.
After the integration of intelligent coding, under the same conditions, the compression performance is further improved by more than 44%. Compared with VVC/H.266, the improvement is more than 8%.
The picture shows the progress of AVS standard internationalization
China's audio and video industry has developed from backward technology to rapid development. A group of academic research forces represented by Peking University have driven the development of China's audio and video. A large number of software and hardware talents have devoted their enthusiasm to the development of the audio and video industry.
Academic breakthroughs led to the rise of businesses. In addition to AVS, it can be seen that Chinese faces account for more than 50% of the H.266/VVC proposal units, bursting out with vigorous strength.
-02-
8K application practice supported by AVS3
For the development of the ultra-high-definition video industry, all countries have formulated corresponding plans to cope with industry competition and seize the commanding heights.
The picture shows some cases in the 8K ultra-high-definition video industry after the formulation of the AVS3 standard
When AVS supported the first 8K Winter Olympics live broadcast, Boya Ruishi supported and solved the problem of 8K AVS3 encoding bit rate control. Its CPU+GPU solution helped Gehua Wangcheng 8K broadcast service, and completed the 8K decoding and broadcast task of the opening ceremony of the Film Museum Winter Olympics.
The AVS full-series real-time codec system based on the x86 architecture developed by Boya Ruishi supports the ultra-high-definition live broadcast of the CCTV Winter Olympics.
In May 2021, the Ministry of Industry and Information Technology, together with the State Administration of Radio, Film and Television, CCTV and other units, organized the "Hundred Cities, Thousand Screens" 8K ultra-high-definition video promotion campaign, and AVS3 provided assistance.
The picture shows the logical architecture diagram of the 8K large screen. According to the requirements of the State Administration of Radio, Film and Television, the outdoor large screen must consider the safety requirements for information release, and the signal cannot be broadcast live. Information such as graphics and texts will be issued in the form of encrypted files, and the front-end information sources will be managed through material, and then the information will be released to the big screen.
As shown in the figure, high-definition video files pass through the storage system, transcoded on the ultra-separation line, and then sent to the information release system. The live broadcast signal will pass through the IP matrix, be dispatched to the 8K transcoding and recording system, and then enter the information release system. The content of the information publishing system is transmitted to the 8K receiving decoder on the large screen side through the IP dedicated line for controlled playback.
In addition, the signal of the 8K large screen will be transcoded all the way down to the resolution, and feature extraction will be performed through the return encoder to prevent the playback content from being tampered with.
A large number of devices are concentrated in the broadcast control center to realize functions such as information release, signal scheduling, content and source monitoring, content recording, 8K content production, and intelligent AI review.
The picture shows some problems encountered in 8K ultra-high-definition live broadcast and corresponding solutions
GPU is suitable for parallel computing, but entropy decoding is linear, so it is more suitable to put it in CPU. Others can be parallelized in the GPU, such as inverse transformation, inter-frame prediction, Deblock, SAO, ALF, etc., using GPU multi-core parallelism. Finally, the reconstruction information is asynchronously copied to the CPU.
The picture shows two server solutions for AVS3 8K real-time encoder
Three schemes and configurations of AVS3 8K real-time decoder
-03-
Ideas for technological evolution and ecological construction in the next few years
The problem of traditional coding is that the cost of refining coding tools under the existing framework is getting higher and higher, while the average compression efficiency of a single technology has dropped from 20% in the 1990s to less than 1% today.
As shown in the figure, AVS3 and VVC use 36 and 34 tools respectively, and only 7 and 8 tools exceed 1% of them.
However, the demand for video encoding is still strong. The original data volume of 8K content is 60Gbps, and the compression bit rate is expected to be below 100Mbps. How to solve it?
Immersive interactive video has multiple dimensions of data, such as resolution, bit width, frame rate, stereo, compression ratio, etc. Traditional coding has insufficient support for intelligent analysis and processing.
The above requirements all indicate that we need more efficient and smarter coding methods.
Full neural network encoding may be a direction to explore. The demand for future encoders has several directions:
1. Optimize the code rate directly based on visual distortion; 2. Simplify codec development;
3. Computing is highly parallel; 4. Can run efficiently on deployed hardware; 5. Can download updates.
The second stage of AVS3 is developing in this direction. Integrating AI technology, the current performance of AVS3 exceeds VVC by 7%, and the future goal is to improve it by 20%.
Generally speaking, intelligent coding will develop with the increase of data richness and computing power. The future trend of intelligent coding is to define corresponding data formats for intelligent media applications and provide more efficient data interfaces.
AVS3 can be technically benchmarked against H.266, and the time is still early, but why has the ecology not been done? In some low-latency, low-power consumption scenarios, it is difficult to have in-depth development and exploration without chip integration. The current plan is to release three AVS3 codec chips in four years.
The picture shows the AVS3 ecological construction plan
The above is my sharing, thank you!
▲Scan the QR code in the picture or click " Read the original text " ▲
Check out more exciting topics of LiveVideoStackCon 2023 Shanghai Station