Principle analysis
Usually, conventional image algorithms are used for detection algorithms to enhance image features, which requires filtering to remove noise. If filtering is used directly, such as Gaussian filtering, median filtering, mean filtering, etc., it will not only filter out noise, but also blur some details of the image, including edges, resulting in loss of image information. Sometimes in order to preserve edges and some detail information, an edge-preserving filtering algorithm is used. The edge-preserving filtering used before includes algorithms such as bilateral filtering and surface blurring. These two algorithms are relatively slow, and they are not cost-effective for detection algorithms.
The traditional filtering algorithm calculates by aligning the kernel center of the filter with the pixel to be processed, regardless of whether the pixel to be processed is in the edge area, causing the filter window to introduce a weighted calculation of pixel values from both sides of the edge when it crosses the edge. Once such calculation is performed, the edge information will diffuse in the direction along the edge normal, resulting in weakening or loss of edge information. For example, for mean filtering:
After the center pixel is filtered by the mean value at the edge, the pixel value will be pulled up
In order to avoid the weakening of the edge by traditional filtering algorithms, it is now considered that the pixels to be processed do not necessarily have to be aligned with the core of the filter.
Let's take the mean filter as an example. For flat areas, no matter what shape of the filter kernel is used by the mean filter, the calculation result is not much different from the original pixel. For the pixels on the edge, if we want to preserve the edge as much as possible, we need to keep as much as possible the pixels in the block similar to the current pixel to be calculated for calculation, and the pixels in the block that are very different from the current pixel to be calculated should be discarded as much as possible. This idea is very similar to the surface blur algorithm. Here is a little introduction to the surface blur algorithm:
It can be seen from the above formula that the main idea of the surface blur algorithm is to calculate the weighted sum of different pixels in the neighborhood of the current pixel X. If the difference from the center pixel is greater, the weight will be smaller, and the difference from the center pixel will be smaller. The weight will be larger, so as to preserve the edge information. If the Y value in the above formula is very large, then the surface blur algorithm is the same as the mean filter algorithm.
The following figure is a typical ideal edge model, which are step edge, slope edge and roof edge. When we do mean filtering and calculate the new pixel value of the pixel where the red dot is located in the figure below, we definitely hope to use the block pixels on the same plane as the red dot as much as possible, and discard the block pixels that are not on the same plane as the red dot, so that we can keep the edge where the red dot is as much as possible instead of weakening the edge where the red dot is.
In order to achieve the above purpose, we can preset some filters, which have edges inside, and the edge type is the step edge type in the above figure, which is divided into 2 blocks. In this way, when calculating the pixels of the edge block of the image, the filter model can be matched with the edge model of the actual pixel block to be calculated. If it can be matched, the pixel value calculated at the red point will not change much. If the matching is reversed, the new value calculated at the red point will be much different from the original value.
There are many kinds of edge models in the filter, and for the convenience of calculation, only 8 of them are taken. Filters of other shapes are not easy to traverse and calculate due to their sawtooth shape (especially when the filter kernel radius is relatively large, then if there is a sawtooth shape, it can only be traversed line by line and cannot be for looped), so it has not been adopted.
Each filter in the 8 types of filters in the above figure is clearly divided into two blocks, which have edges and have different shapes. The most important thing is that the value distribution in it is square or rectangular, which is very convenient for traversal calculations. We only need to use the above-mentioned 8 kinds of filters to match the pixels to be calculated, and then calculate the corresponding new pixel value. When the difference between the new pixel value and the original pixel value is the smallest, it means that the edge shape of the corresponding filter is closest to the edge shape of the block where the pixel to be calculated is located. At this time, the average filter adopted will minimize the weakening of the pixels at the edge.
The above is my introduction to the side window filtering algorithm based on my own understanding. There are formula derivations in the original paper, and there are also formulas for analyzing the shape of the side window of the filter. It is more rigorous, but it does not feel intuitive. I will provide a way of understanding according to my own thinking. If you have any questions, or think that my understanding is wrong, please correct me, thank you!
#include <fstream>
#include <opencv2/opencv.hpp>
#include <string>
using namespace std;
using namespace cv;
#include <math.h>
#include <cstdio>
#include <cstdlib>
enum class SIDE_DIRECTIONS : int {
L = 0,
R = 1,
U = 2,
D = 3,
NW = 4,
NE = 5,
SW = 6,
SE = 7,
SIDE_DIRECTIONS_COUNT = 8
};
const int side_directions_count = 8;
enum class XY_START_END : int {
WIDTH_START = 0,
WIDTH_END = 1,
HEIGHT_START = 2,
HEIGHT_END = 3,
XY_START_END_COUNT = 4
};
//表面模糊处理
void SurfaceBlur(float *data, int height, int width, int r, int threshold,
float *result) {
for (int i = 0; i < height; ++i) {
for (int j = 0; j < width; ++j) {
float sumx = 0.f;
float sumy = 0.f;
for (int m = i - r; m < i + r; ++m) {
for (int n = j - r; n < j + r; ++n) {
sumx = sumx + (1 - fabs(data[i * width + j] - data[m * width + n]) /
(2.5 * threshold)) *
data[m * width + n];
sumy = sumy + (1 - fabs(data[i * width + j] - data[m * width + n]) /
(2.5 * threshold));
}
}
result[i * width + j] = sumx / sumy;
}
}
}
std::vector<std::vector<int> > m_side_window_params;
std::vector<std::vector<float> > m_side_window_filter_results;
void init(int height, int width, int radius) {
int panel_size = height * width;
m_side_window_params.resize(side_directions_count);
m_side_window_params[static_cast<int>(SIDE_DIRECTIONS::L)] = {
-radius, 0, -radius, radius};
m_side_window_params[static_cast<int>(SIDE_DIRECTIONS::R)] = {
0, radius, -radius, radius};
m_side_window_params[static_cast<int>(SIDE_DIRECTIONS::U)] = {-radius, radius,
-radius, 0};
m_side_window_params[static_cast<int>(SIDE_DIRECTIONS::D)] = {-radius, radius,
0, radius};
m_side_window_params[static_cast<int>(SIDE_DIRECTIONS::NW)] = {-radius, 0,
-radius, 0};
m_side_window_params[static_cast<int>(SIDE_DIRECTIONS::NE)] = {0, radius,
-radius, 0};
m_side_window_params[static_cast<int>(SIDE_DIRECTIONS::SW)] = {-radius, 0, 0,
radius};
m_side_window_params[static_cast<int>(SIDE_DIRECTIONS::SE)] = {0, radius, 0,
radius};
m_side_window_filter_results.resize(side_directions_count);
for (int i = 0; i < side_directions_count; ++i) {
m_side_window_filter_results[i].resize(panel_size);
}
}
void sideWindowFilterImpl(const float *input, const int radius,
const int height, const int width,
const SIDE_DIRECTIONS direction, float *output) {
const int w_start =
m_side_window_params[static_cast<int>(direction)]
[static_cast<int>(XY_START_END::WIDTH_START)];
const int w_end =
m_side_window_params[static_cast<int>(direction)]
[static_cast<int>(XY_START_END::WIDTH_END)];
const int h_start =
m_side_window_params[static_cast<int>(direction)]
[static_cast<int>(XY_START_END::HEIGHT_START)];
const int h_end =
m_side_window_params[static_cast<int>(direction)]
[static_cast<int>(XY_START_END::HEIGHT_END)];
const int w_first_loop_end = std::max(0, -w_start);
const int w_second_loop_start = w_first_loop_end;
const int w_second_loop_end = width - 1 - w_end;
const int w_third_loop_start = width - w_end;
int nn = 0;
int remain_start = (nn << 2) + w_second_loop_start;
int out_idx = 0;
int threshold = 10000;
for (int h = 0; h < height; ++h) {
const int h_lower_bound = std::max(0, h + h_start);
const int h_upper_bound = std::min(height - 1, h + h_end);
const int h_interval = h_upper_bound - h_lower_bound + 1;
//第一个for循环是处理图像边缘
for (int w = 0; w < w_first_loop_end; ++w) {
const int w_left_bound = std::max(0, w + w_start);
const int w_right_bound = std::min(width - 1, w + w_end);
const int arr_len = h_interval * (w_right_bound - w_left_bound + 1);
int idx = 0;
float sumx = 0.f;
float sumy = 0.f;
float center_pixel = input[h * width + w];
for (int i = h_lower_bound; i <= h_upper_bound; ++i) {
const int h_idx = i * width;
for (int j = w_left_bound; j <= w_right_bound; ++j) {
sumx = sumx + (1 - fabs(center_pixel - input[h_idx + j]) /
(2.5 * threshold)) *
input[h_idx + j];
sumy = sumy + (1 - fabs(center_pixel - input[h_idx + j]) /
(2.5 * threshold));
}
}
output[out_idx++] = sumx / sumy;
}
//这里是处理图像中间的任意像素
for (int w = remain_start; w <= w_second_loop_end; ++w) {
const int w_left_bound = std::max(0, w + w_start);
const int w_right_bound = std::min(width - 1, w + w_end);
int idx = 0;
float sumx = 0.f;
float sumy = 0.f;
float center_pixel = input[h * width + w];
for (int i = h_lower_bound; i <= h_upper_bound; ++i) {
const int h_idx = i * width;
for (int j = w_left_bound; j <= w_right_bound; ++j) {
sumx = sumx + (1 - fabs(center_pixel - input[h_idx + j]) /
(2.5 * threshold)) *
input[h_idx + j];
sumy = sumy + (1 - fabs(center_pixel - input[h_idx + j]) /
(2.5 * threshold));
}
}
output[out_idx++] = sumx / sumy;//这里是基于表面模糊的侧窗滤波算法，
//如果你想改成基于均值滤波的侧窗滤波算法，那么就统计下上面那两个for循环的
//像素个数，及累加的像素值的和，然后在这里求下比值赋值给output[out_idx++]就行了
}
for (int w = w_third_loop_start; w < width; ++w) {
const int w_left_bound = std::max(0, w + w_start);
const int w_right_bound = std::min(width - 1, w + w_end);
const int arr_len = h_interval * (w_right_bound - w_left_bound + 1);
int idx = 0;
float sumx = 0.f;
float sumy = 0.f;
float center_pixel = input[h * width + w];
for (int i = h_lower_bound; i <= h_upper_bound; ++i) {
const int h_idx = i * width;
for (int j = w_left_bound; j <= w_right_bound; ++j) {
sumx = sumx + (1 - fabs(center_pixel - input[h_idx + j]) /
(2.5 * threshold)) *
input[h_idx + j];
sumy = sumy + (1 - fabs(center_pixel - input[h_idx + j]) /
(2.5 * threshold));
}
}
output[out_idx++] = sumx / sumy;
}
}
}
void SideWindowFilter(const float *input, const int radius, const int height,
const int width, float *output) {
int panel_size = height * width;
for (int i = 0; i < side_directions_count; ++i) {
sideWindowFilterImpl(input, radius, height, width,
static_cast<SIDE_DIRECTIONS>(i),
m_side_window_filter_results[i].data());
}
for (int i = 0; i < panel_size; ++i) {
float tmp = m_side_window_filter_results[0][i] - input[i];
float min = tmp * tmp;
int min_idx = 0;
for (int j = 1; j < side_directions_count; ++j) {
tmp = m_side_window_filter_results[j][i] - input[i];
tmp = tmp * tmp;
if (min > tmp) {
min_idx = j;
min = tmp;
}
}
output[i] = m_side_window_filter_results[min_idx][i];
}
}
int main() {
cv::Mat sourceimg = imread("src_img/降噪test.jpg", 1);
cv::Mat resultimg = imread("src_img/降噪test.jpg", 1);
cv::imwrite("0_sourceimg.jpg", sourceimg);
int img_h = sourceimg.rows;
int img_w = sourceimg.cols;
cv::Mat temp_img;
temp_img.create(img_h, img_w, CV_32FC3);
sourceimg.convertTo(temp_img, CV_32FC3);
float *input_r = new float[img_h * img_w];
float *input_g = new float[img_h * img_w];
float *input_b = new float[img_h * img_w];
int radius = 10;
int iteration = 10;
for (int ite = 10; ite <= 10; ++ite) {
iteration = 5;
for (int rad = 3; rad <= 3; rad +=1) {
radius = 2;
for (int i = 0; i < img_h; ++i) {
for (int j = 0; j < img_w; ++j) {
input_r[i * img_w + j] = sourceimg.data[i * img_w * 3 + j * 3 + 0];
input_g[i * img_w + j] = sourceimg.data[i * img_w * 3 + j * 3 + 1];
input_b[i * img_w + j] = sourceimg.data[i * img_w * 3 + j * 3 + 2];
}
}
// SurfaceBlur(input,img_h,img_w, 1, 250, result);
init(img_h, img_w, radius);
for (int m = 0; m < iteration; ++m) {
float *temp_result_r = new float[img_h * img_w];
float *temp_result_g = new float[img_h * img_w];
float *temp_result_b = new float[img_h * img_w];
SideWindowFilter(input_r, radius, img_h, img_w, temp_result_r);
SideWindowFilter(input_g, radius, img_h, img_w, temp_result_g);
SideWindowFilter(input_b, radius, img_h, img_w, temp_result_b);
for (int i = 0; i < img_h; ++i) {
for (int j = 0; j < img_w; ++j) {
input_r[i * img_w + j] = temp_result_r[i * img_w + j];
input_g[i * img_w + j] = temp_result_g[i * img_w + j];
input_b[i * img_w + j] = temp_result_b[i * img_w + j];
}
}
delete (temp_result_r);
delete (temp_result_g);
delete (temp_result_b);
}
for (int i = 0; i < img_h; ++i) {
for (int j = 0; j < img_w; ++j) {
resultimg.data[i * img_w * 3 + j * 3 + 0] = input_r[i * img_w + j];
resultimg.data[i * img_w * 3 + j * 3 + 1] = input_g[i * img_w + j];
resultimg.data[i * img_w * 3 + j * 3 + 2] = input_b[i * img_w + j];
}
}
char str[100];
sprintf(str, "iteration_%d_radius_%d_threshold_50.jpg", iteration,
radius);
cv::imwrite(str, resultimg);
}
}
// cv::imwrite("SurfaceBlur.jpg", resultimg);
delete (input_r);
delete (input_g);
delete (input_b);
waitKey(-1);
}