The initIntraMip function is mainly to downsample the reference pixels and prepare input data for MIP matrix multiplication. The function structure is as follows:
MIP can be divided into the following three cases according to the block size:
| Block size | Boundary length after downsampling m_reducedBdrySize |
Matrix multiplication output boundary length m_reducedPredSize |
|
| mipSizeId = 0 | 4x4 | 2 | 4 |
| mipSizeId = 1 | 4xN, Nx4, 8x8 | 4 | 4 |
| mipSizeId = 2 | Rest block | 4 | 8 |
The initIntraMip function mainly prepares the boundary reference pixels and calls the prepareInputForPred function to prepare input data for MIP prediction
Note: The reference pixels used by MIP are unfiltered reference pixels
The initIntraMip function code is as follows:
void IntraPrediction::initIntraMip( const PredictionUnit &pu, const CompArea &area )
{
CHECK( area.width > MIP_MAX_WIDTH || area.height > MIP_MAX_HEIGHT, "Error: block size not supported for MIP" );
// prepare input (boundary) data for prediction
// 准备输入（边界）数据进行预测
// MIP使用未滤波的参考像素
CHECK( m_ipaParam.refFilterFlag, "ERROR: unfiltered refs expected for MIP" );
#if JVET_R0350_MIP_CHROMA_444_SINGLETREE
Pel *ptrSrc = getPredictorPtr(area.compID);//获取参考像素
const int srcStride = m_refBufferStride[area.compID];
const int srcHStride = 2;
m_matrixIntraPred.prepareInputForPred(CPelBuf(ptrSrc, srcStride, srcHStride), area,
pu.cu->slice->getSPS()->getBitDepth(toChannelType(area.compID)), area.compID);
#else
Pel *ptrSrc = getPredictorPtr( COMPONENT_Y );
const int srcStride = m_refBufferStride[COMPONENT_Y];
const int srcHStride = 2;
m_matrixIntraPred.prepareInputForPred( CPelBuf( ptrSrc, srcStride, srcHStride ), area, pu.cu->slice->getSPS()->getBitDepth( CHANNEL_TYPE_LUMA ) );
#endif
}The prepareInputForPred function is mainly divided into the following four steps:
- Step 1: Save the block size and calculate the MIP related parameters by calling the initPredBlockParams function
- Step 2: Obtain input data (reference pixels in the previous row and reference pixels in the left column)
- Step 3: Calculate the reduction boundary through Haar downsampling, and implement it through the boundaryDownsampling1D function
- Step 4: Derive the input vector for matrix multiplication
The prepareInputForPred function code is as follows
#if JVET_R0350_MIP_CHROMA_444_SINGLETREE
void MatrixIntraPrediction::prepareInputForPred(const CPelBuf &pSrc, const Area &block, const int bitDepth,
const ComponentID compId)
{
m_component = compId;
#else
void MatrixIntraPrediction::prepareInputForPred(const CPelBuf &pSrc, const Area& block, const int bitDepth)
{
#endif
// Step 1: Save block size and calculate dependent values
// Step 1: 保存块大小并计算MIP相关参数
initPredBlockParams(block);
// Step 2: Get the input data (left and top reference samples)
// Step 2: 获取输入数据（左上参考像素）
// 获取上一行参考像素
m_refSamplesTop.resize(block.width);
for (int x = 0; x < block.width; x++)
{
m_refSamplesTop[x] = pSrc.at(x + 1, 0);
}
// 获取左一列参考像素
m_refSamplesLeft.resize(block.height);
for (int y = 0; y < block.height; y++)
{
m_refSamplesLeft[y] = pSrc.at(y + 1, 1);
}
// Step 3: Compute the reduced boundary via Haar-downsampling (input for the prediction)
// Step 3: 通过Haar下采样计算缩减边界（预测输入）
// 下采样后输入向量的尺寸为4或者8
const int inputSize = 2 * m_reducedBdrySize;
// 不需要转置时，下采样像素的顺序：先上后左
m_reducedBoundary .resize( inputSize );
// 转置时，下采样像素的顺序：先左后上
m_reducedBoundaryTransposed.resize( inputSize );
int* const topReduced = m_reducedBoundary.data();
boundaryDownsampling1D( topReduced, m_refSamplesTop.data(), block.width, m_reducedBdrySize );
int* const leftReduced = m_reducedBoundary.data() + m_reducedBdrySize;
boundaryDownsampling1D( leftReduced, m_refSamplesLeft.data(), block.height, m_reducedBdrySize );
int* const leftReducedTransposed = m_reducedBoundaryTransposed.data();
int* const topReducedTransposed = m_reducedBoundaryTransposed.data() + m_reducedBdrySize;
for( int x = 0; x < m_reducedBdrySize; x++ )
{
topReducedTransposed[x] = topReduced[x];
}
for( int y = 0; y < m_reducedBdrySize; y++ )
{
leftReducedTransposed[y] = leftReduced[y];
}
// Step 4: Rebase the reduced boundary
// Step 4: 缩小边界
// 推导矩阵乘法输入向量p，mipSizeId=0/1和mipSizeId=2的推导方法不一样
m_inputOffset = m_reducedBoundary[0];
m_inputOffsetTransp = m_reducedBoundaryTransposed[0];
const bool hasFirstCol = (m_sizeId < 2);
m_reducedBoundary [0] = hasFirstCol ? ((1 << (bitDepth - 1)) - m_inputOffset ) : 0; // first column of matrix not needed for large blocks
m_reducedBoundaryTransposed[0] = hasFirstCol ? ((1 << (bitDepth - 1)) - m_inputOffsetTransp) : 0;
for (int i = 1; i < inputSize; i++)
{
m_reducedBoundary [i] -= m_inputOffset;
m_reducedBoundaryTransposed[i] -= m_inputOffsetTransp;
}
}1. Initialize MIP related parameters
The initPredBlockParams function is used to initialize MIP related parameters, mainly to initialize mipSizeId according to the size of the current block, and then to initialize the boundary length after downsampling, matrix multiplication output boundary length and upsampling factor according to mipSizeId
void MatrixIntraPrediction::initPredBlockParams(const Size& block)
{
//获得当前块尺寸
m_blockSize = block;
// init size index
// 根据当前块尺寸初始化sizeId
m_sizeId = getMipSizeId( m_blockSize );
// init reduced boundary size
// 初始缩减边界尺寸
// 对于4x4的块宽度和高度分别缩减为2个像素
// 对于其余尺寸的块宽度和高度分别缩减为4个像素
m_reducedBdrySize = (m_sizeId == 0) ? 2 : 4;
// init reduced prediction size
// 初始化缩减预测后的尺寸
// 对于mipSizeId = 0、1的块，MIP预测后输出4x4的块
// 对于mipSizeId = 2的块，MIP预测后输出8x8的块
m_reducedPredSize = ( m_sizeId < 2 ) ? 4 : 8;
// init upsampling factors
// 初始上采样因子
m_upsmpFactorHor = m_blockSize.width / m_reducedPredSize;
m_upsmpFactorVer = m_blockSize.height / m_reducedPredSize;
CHECKD( (m_upsmpFactorHor < 1) || ((m_upsmpFactorHor & (m_upsmpFactorHor - 1)) != 0), "Need power of two horizontal upsampling factor." );
CHECKD( (m_upsmpFactorVer < 1) || ((m_upsmpFactorVer & (m_upsmpFactorVer - 1)) != 0), "Need power of two vertical upsampling factor." );
}2. Downsampling
The down-sampling process of the boundary reference pixels is implemented by the boundaryDownsampling1D function. The down-sampling process is actually the process of averaging the boundary reference pixels. Taking an 8x8 block as an example, as shown in the figure below, there are 8 reference pixels in the previous row. After averaging two adjacent reference pixels, 4 down-sampled reference pixels are obtained. The down-sampling process of the reference pixels in the left column is the same.
/*
一维下采样
reducedDst表示下采样后的边界
fullSrc表示下采样前的边界
srcLen表示下采样前的边界长度
dstLen表示下采样后的边界长度
*/
void MatrixIntraPrediction::boundaryDownsampling1D(int* reducedDst, const int* const fullSrc, const SizeType srcLen, const SizeType dstLen)
{
if (dstLen < srcLen)
{
//当下采样后的边界尺寸小于当前块的边界尺寸时，需要进行下采样，下采样操作即相当于求平均操作
// Create reduced boundary by downsampling 通过下采样创建缩小边界
const SizeType downsmpFactor = srcLen / dstLen;
const int log2DownsmpFactor = floorLog2(downsmpFactor);
const int roundingOffset = (1 << (log2DownsmpFactor - 1));
SizeType srcIdx = 0;
for( SizeType dstIdx = 0; dstIdx < dstLen; dstIdx++ )
{
int sum = 0;
for( int k = 0; k < downsmpFactor; k++ )
{
sum += fullSrc[srcIdx++];
}
reducedDst[dstIdx] = (sum + roundingOffset) >> log2DownsmpFactor;
}
}
else
{
// Copy boundary if no downsampling is needed 如果不需要下采样，则复制边界
for (SizeType i = 0; i < dstLen; ++i)
{
reducedDst[i] = fullSrc[i];
}
}
}After the down-sampling process is over, the down-sampled upper reference pixel and left reference pixel are arranged into the vector pTemp according to the mipTransposeFlag flag, the arrangement is as follows:
- mipTransposeFlag = 0时，pTemp = [redTop,redLeft]
- mipTransposeFlag = 1时，pTemp = [redLeft,redTop]
3. Derive the input vector for matrix multiplication
The derivation method of the matrix multiplication input vector is related to mipSizeId. The construction process of the input vector p is shown below, where inSize = 2*m_reducedBdrySize
