The road to digital image processing learning (3): Don't you know how to zoom in and out of images? Image scaling using nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation

Introduction

A digital image is composed of individual pixels. To achieve zooming in and out of an image is to map the pixel values ​​of the image to a new area, thereby realizing zooming in and out of the image. For example, a 3x3 area is enlarged to 5x5, and the nearest neighbor interpolation is used as shown in the following figure:

(Note: The color here has nothing to do with the pixel value, just to better reflect the mapping relationship between pixel values ​​in the image.)

1. Nearest neighbor interpolation

Nearest neighbor interpolation, as the name implies, is to select the value of adjacent pixel points as the pixel value of the new position, which can be realized by rounding.

The steps of nearest neighbor interpolation are as follows:

1. Calculate the position (srcX, srcY) corresponding to the current point (i, j) of the zoomed image in the original image: (src represents the input image, dst represents the output image).
$$srcX=i\ast （srcImage.rows/dstImage.rows）$$

$$srcY=j\ast (srcImage.cols/dstImage.cols)$$

2. The position calculated by the first step is a floating point number, and the integer position is obtained by rounding.

3. Find the pixel value of point (srcX, srcY) in the original image and assign it to the current image.
$$dstImage(i,j)=srcImage(srcX,srcY)$$
complete implementation code is as follows:

	for (int i = 0; i < dstImage.rows; i++) {
    
    
		for (int j = 0; j < dstImage.cols; j++)
		{
    
    
            //这里加上0.5是为了实现四舍五入
			srcX = i * (srcImage.rows * 1.0 / dstImage.rows) + 0.5;
			srcY = j * (srcImage.cols * 1.0 / dstImage.cols) + 0.5;
            //防溢出
			if (srcX >= srcImage.rows)
				srcX = srcImage.rows - 1;
			if (srcY >= srcImage.cols)
				srcY = srcImage.cols - 1;
			dstImage.at<uchar>(i, j) = srcImage.at<uchar>(srcX, srcY);
		}
	}

2. Bilinear interpolation

At the beginning of this article, it has been stated that the scaling of the image is mainly based on the pixel values ​​of the original image being filled into the new image through a certain mapping relationship. Bilinear interpolation is calculated by calculating the double lines of 4 points around the pixel point (srcX, srcY) Sexual interpolation gets a weighted average of surrounding pixels, which is then filled to the current position (i,j).

Bilinear interpolation: First perform two monolinear interpolations in the X direction, and then perform a monolinear interpolation in the Y direction.

Introduction to single linear interpolation method:

由公式
$$\frac{f(x)-f(x_0)}{x-x_0}=\frac{f(x_1)-f\left(x_0\right)}{x_1-x_0}$$
可得
$$f\left(x\right)=\frac{x-x_0}{x_1-x_0}f\left(x_1\right)+\frac{x_1-x}{x_1-x_0}f\left(x_0\right)$$
It can be seen that the function value at x can be calculated by weighting the two points on the line.

Bilinear interpolation method introduction:

First perform two linear interpolations in the X direction to obtain the values ​​of R1 and R2
$$R1=\frac{x_2-srcX}{x_2-x_1}{Q}_{11}+\frac{srcX-x_1}{x_2-x_1}{Q}_{21},R2\_=\frac{x_2-srcX}{x_2-x_1}{Q}_{12}+\frac{srcX-x_1}{x_2-x_1}{Q}_{22}$$
Then perform linear interpolation on the calculated R1 and R2 in the Y direction to obtain the final P value
$$P=\frac{y_2-srcY}{y_2-y_1}{R}_1+\frac{srcY-y_1}{y_2-y_1}{R}_2,$$
the complete implementation code is as follows:

	for (int i = 0; i < dstImage.rows; i++) {
    
    
		for (int j = 0; j < dstImage.cols; j++)
		{
    
    
			//找到原始点的位置 ，选取最近的四个点的像素值进行加权计算然后赋值
			srcX = i * (srcImage.rows * 1.0 / dstImage.rows);
			srcY = j * (srcImage.cols * 1.0 / dstImage.cols);
			x1 = srcX;
			x2 = srcX + 1 > srcImage.rows - 1 ? srcX - 1 : srcX + 1;//防溢出处理
			y1 = srcY;
			y2 = srcY + 1 > srcImage.cols - 1 ? srcY - 1 : srcY + 1;
			Q11 = srcImage.at<uchar>(x1, y1);
			Q12 = srcImage.at<uchar>(x1, y2);
			Q21 = srcImage.at<uchar>(x2, y1);
			Q22 = srcImage.at<uchar>(x2, y2);
			R1 = (x2 - srcX) / (x2 - x1) * Q11 + (srcX - x1) / (x2 - x1) * Q21;
			R2 = (x2 - srcX) / (x2 - x1) * Q12 + (srcX - x1) / (x2 - x1) * Q22;
			P = (y2 - srcY) / (y2 - y1) * R1 + (srcY - y1) / (y2 - y1) * R2;
            //saturate_cast<uchar>(P) OpenCV中的函数可以防溢出
			dstImage.at<uchar>(i, j) = saturate_cast<uchar>(P);
		}
	}

Three, bicubic interpolation

Bilinear interpolation calculates the weighted average of the pixel values ​​of 4 points around the pixel point (srcX, srcY), while bicubic interpolation calculates the weighted average of the pixel values ​​of 16 surrounding points.

As shown in the figure, (ai, bi) represents the coordinate position of the surrounding 16 points, Qi represents the pixel value of the point, w1[i] calculates the weight in the X direction, and w2[i] calculates the Y The weight in the direction, and finally the weight of each point is obtained by multiplying the value in w1 and w2. Usually use the BiCubic formula to calculate the weight (BiCubic interpolation).

The value of a is usually -0.5, where |x| represents the distance. For example, to calculate the weight corresponding to (a1,b1), use |a1-srcX| in the X direction to enter the formula for calculation, and use |b1 in the Y direction -srcY|Bring into the formula calculation.

In summary, the calculation formula of the final pixel value P is as follows:
$$P=\left[\begin{array}{cccc}w1[0]& w1[1]& w1[2]& w1[3]\end{array}\right]\left[\begin{array}{cccc}Q1& Q2\_& Q3& Q4\_\\ Q5& Q6\_& Q7\_& Q8\_\\ Q9\_& Q10& Q11& Q12\\ Q13& Q& Q15& Q16\end{array}\right]{\left[\begin{array}{cccc}w2[0]& w2[1]& w2[2]& w2[3]\end{array}\right]}^{}$$
The complete implementation code$${}^{of\; T\; is\; as\; follows:}$$

void ReSize(Mat srcImage)
{
    
    
    Mat dstImage = Mat((srcImage.size()) * 4, srcImage.type());
    double srcX = 0;
	double srcY = 0;
	int a1=0, a2=0, a3=0, a4=0;
	int b1=0, b2=0, b3=0, b4=0;
	double w1[4],w2[4];//对应x,y的权重
	int P=0;
	double sum = 0;
	for (int i = 0; i < dstImage.rows; i++) {
    
    
		for (int j = 0; j < dstImage.cols; j++)
		{
    
    
			//找到原始点的位置 ，选取最近的16个点的像素值进行加权计算然后赋值
			srcX = i * (srcImage.rows * 1.0 / dstImage.rows);
			srcY = j * (srcImage.cols * 1.0 / dstImage.cols);
			a1 = (srcX - 1) < 0 ? (srcX + 1) : (srcX - 1); //用三元表达式实现边界处理，镜像像素值
			a2 = srcX < 0 ? 0 : srcX;
			a3 = (srcX + 1) > srcImage.rows - 1 ? (srcX - 1) : (srcX + 1);
			a4 = (srcX + 2) > srcImage.rows - 1 ? (srcX - 2) : (srcX + 2);
			b1 = (srcY - 1) < 0 ? (srcY + 1) : (srcY - 1);
			b2 = srcY;
			b3 = (srcY + 1) > srcImage.cols - 1 ? (srcY - 1) : (srcY + 1);
			b4 = (srcY + 2) > srcImage.cols - 1 ? (srcY - 2) : (srcY + 2);
			w1[0] = countW(-0.5, srcX - a1);
			w1[1] = countW(-0.5, srcX - a2);
			w1[2] = countW(-0.5, srcX - a3);
			w1[3] = countW(-0.5, srcX - a4);
			w2[0] = countW(-0.5, srcY - b1);
			w2[1] = countW(-0.5, srcY - b2);
			w2[2] = countW(-0.5, srcY - b3);
			w2[3] = countW(-0.5, srcY - b4);
            //注意要将权重归一化
			sum = w1[0] * w2[0] + w1[0] * w2[1] + w1[0] * w2[2] + w1[0] * w2[3]
				+ w1[1] * w2[0] + w1[1] * w2[1] + w1[1] * w2[2] + w1[1] * w2[3]
				+ w1[2] * w2[0] + w1[2] * w2[1] + w1[2] * w2[2] + w1[2] * w2[3]
				+ w1[3] * w2[0] + w1[3] * w2[1] + w1[3] * w2[2] + w1[3] * w2[3];
			P = srcImage.at<uchar>(a1, b1) * w1[0] * w2[0] / sum + srcImage.at<uchar>(a1, b2) * w1[0] * w2[1] / sum +
				srcImage.at<uchar>(a1, b3) * w1[0] * w2[2] / sum + srcImage.at<uchar>(a1, b4) * w1[0] * w2[3] / sum +
				srcImage.at<uchar>(a2, b1) * w1[1] * w2[0] / sum + srcImage.at<uchar>(a2, b2) * w1[1] * w2[1] / sum +
				srcImage.at<uchar>(a2, b3) * w1[1] * w2[2] / sum + srcImage.at<uchar>(a2, b4) * w1[1] * w2[3] / sum +
				srcImage.at<uchar>(a3, b1) * w1[2] * w2[0] / sum + srcImage.at<uchar>(a3, b2) * w1[2] * w2[1] / sum +
				srcImage.at<uchar>(a3, b3) * w1[2] * w2[2] / sum + srcImage.at<uchar>(a2, b4) * w1[2] * w2[3] / sum +
				srcImage.at<uchar>(a4, b1) * w1[3] * w2[0] / sum + srcImage.at<uchar>(a4, b2) * w1[3] * w2[1] / sum +
				srcImage.at<uchar>(a4, b3) * w1[3] * w2[2] / sum + srcImage.at<uchar>(a4, b4) * w1[3] * w2[3] / sum;
			dstImage.at<uchar>(i, j) = saturate_cast<uchar>(P);
		}
	}
}
double countW(double a ,double x) {
    
    
	x = abs(x);//取绝对值
	if (x <= 1)
		return (a + 2) * pow(x,3)- ( a + 3 ) * pow(x,2) + 1;
	else if (x > 1 && x < 2)
		return a * pow(x,3) - 5 * a * pow(x,2) + 8 * a * x - 4 * a;
	else
		return 0;
}

4. Special processing on boundary pixels

If the calculated (srcX, srcY) is a point on the boundary, then there will be null values ​​around, and the corresponding position is filled with the pixel value of the symmetrical point, as shown in the figure, the corresponding position has been filled with the corresponding pixel value.

			 //用三元表达式实现边界处理 若(srcX - 1)<0则取它关于当前点的对称位置(srcX + 1)，否则则取(srcX - 1)
			a1 = (srcX - 1) < 0 ? (srcX + 1) : (srcX - 1);

5. Effect comparison of three interpolation methods

Enlarge Image Using Nearest Neighbor Interpolation

Enlarge image using bilinear interpolation

Enlarge image using bicubic interpolation

It can be seen that the image enlarged by nearest neighbor interpolation has a mosaic, and the enlargement effect of the latter two interpolations is obviously better. This is because the latter two interpolation methods perform a weighted average of the surrounding pixel values, making the overall image smoother. The bicubic interpolation also takes into account the different weights of different distances on the pixels, so the zoom effect of the bicubic interpolation is the best. , and the calculation amount is also the largest.

Reference blog:
Bilinear interpolation: https://blog.csdn.net/eurus_/article/details/102755898
Bi-cubic interpolation: https://blog.csdn.net/kill2013110/article/details/108125738

It is not easy to create, please mark the source when using pictures, thank you!