This article is from the PCS 2022 paper "Unified Fast Partitioning Algorithm for Intra and Inter Predictions in Versatile Video Coding"
introduce
The VVC (Versatile Video Coding) standard adopts a more flexible partition structure than the HEVC (High Efficiency Video Coding) standard. Based on the quadtree partition structure of HEVC, it introduces multi-type tree partitions, which greatly improves the coding efficiency and leads to This greatly increases the complexity of the encoding side. In this paper, based on the historical information checked by the encoder RD, we propose a unified fast division of intra and inter frames. The algorithm proposed in this paper was tested on the VVC reference software VTM-14.0. Under the configurations of All Intra (AI), Random Access (RA) and Low Delay P (LDP), the average reduction of encoder running time was 42.47%, 40.46% and 40.38%, BDBR losses were 0.45%, 1.08% and 1.18%.
VVC partition structure
The division structure comparison between VVC and HEVC is shown in the figure below. In VVC, a new QTMT CU division structure is introduced to unify the concepts of CU, PU and TU. In addition to non-partitioned (NS) and QT partitions in HEVC, VVC also supports recursive partition structures of binary tree (BT) and TT, also known as multi-type tree (MTT). Specifically, MTT includes four types of vertical binary tree (vertical binary tree, VBT), horizontal binary tree (horizontal binary tree, HBT), vertical ternary tree (vertical three tree, VTT) and horizontal ternary tree (horizontal three tree, HTT) partition, as shown in the figure below. During the partitioning process, the CU can choose any partition tree shown in the figure below. However, if the CU chooses to use the multi-type tree MTT partition structure, subsequent child nodes can only choose the MTT partition structure, and are limited by the maximum MTT depth.
proposed algorithm
1. BT-direction-based QT and TT skip
When the encoder checks BT division before TT and QT, it can choose to skip QT and TT division according to the BT division result.
Specifically, if the final partition structure in the BT partition path only contains a single partition direction, it is likely that another partition direction is not suitable for the current CU. As shown in the figure below, if the final partition structure of VBT only contains the vertical partition direction, it can be inferred that the horizontal partition is not applicable to the current CU, so that the horizontal binary tree partition and quadtree partition can be skipped. The same rule applies to HBT skipping VTT and QT.
2. BT-position-based TT skip
In addition to the direction information of the BT partition path, the location information of its partitions also shows the optimal partition structure, which can be used to skip TT.
As shown in the figure below, if the final partition structure of VBT does not produce any partitions located in the VTT position, it can be inferred that the VTT partition is not suitable for the current CU and can be skipped. Otherwise VTT cannot be skipped.
The same rule applies to HBT's skipping of HTT.
3. MTT-depth-based BT and TT skip
When a CU cannot find a good prediction by the current size, it needs to be partitioned into smaller CUs, and the partition depth information obtained from the inspection path indicates the optimal CU size. This subsection proposes that if one decides to check QT before BT and TT, the MTT depth in QT can be exploited to skip the remaining partitions early.
As mentioned before, the VVC encoder only allows the MTT split node to recursively split BT and TT, and limits the maximum split depth of MTT.
If a CU checks QT first, and a QT child reaches the maximum MTT depth, which means that the optimal CU size is much smaller than the current CU, then the current CU can skip the check of BT and TT, because BT and VT cannot Partitioning gets such small CUs. As shown in the figure below, where the maximum MTT depth is set to 3, the MTT depth of the first CU in the QT partition path through sequential VBT, HBT, and VBT is 3. However, the current CU cannot be divided into such a small sub-CU through BT and TT (limited by the MTT depth, it can only be divided twice again), so all remaining BT and TT divisions can be skipped in advance.
4. Historical-CU-based BT and TT skip
VVC adopts the recursive partition structure of QT, BT, and MT, so CUs of the same size at the same location can be checked multiple times on different partition paths, as shown in the figure below.
In this paper, a CU at the same position and the same size as the current CU is referred to as a historical CU. Although a CU has the same size as its historical CU at the same location, due to the difference in neighboring information, such as different reference pixels in intra prediction, or different motion vectors (mv) in inter prediction, it There may be different partitioning decisions. Therefore, directly reusing the optimal partition decision of the historical CU may bring a great loss to the current CU.
This subsection proposes to exploit the correlation of the optimal partition direction between the current CU and its historical CUs in different partition paths. If both BT and TT of one direction are better than those of the other direction in the historical CU, it is likely that the division of the other direction is not suitable for the current CU, so that the current CU can skip the BT and TT of the other direction, As shown below.
experiment
The algorithm proposed in this paper was implemented in VTM14.0. The test results are shown in the figure below. Table 2 shows the final effect of the four algorithms proposed, and Table 3 shows the individual effects of the four algorithms proposed under RA.
