For various reasons, it is inevitable there will be noise in the image, it needs to be removed. Random noise can be understood as a change in gradation values, i.e. number of pixels unwanted introduced during photographing. Noise can be divided salt and pepper noise, Gaussian noise, additive noise and multiplicative noise, see: https: //zhuanlan.zhihu.com/p/52889476
And removing the main noise suppression by smoothing, comprising a two-dimensional discrete convolution based on Gaussian smoothing, the smoothing average, based on the statistical median smoothed, and capable of holding bilateral filtering image edge, the guide filtering algorithm. The following describes the use of specific
1. The two-dimensional discrete convolution
Understand convolution: https: //www.zhihu.com/question/22298352/answer/637156871
https://www.zhihu.com/question/22298352/answer/228543288
Before the learning image smoothing, it is necessary knowledge of the convolution, reading the connection, the image processing should be appreciated that several convolution Keywords: convolution kernel, anchor, step, inner product, convolution mode
Convolution kernel (Kernel) : used for smoothing matrix image matrix, also referred to as a filter (filter)
Anchor : overlapping image matrix convolution kernel and performs the inner product calculation, the pixels from the anchor position is substituted Calcd. Generally selected convolution kernel is odd, the center point as an anchor
Step : convolution moving along the length of each image matrix
Inner product : convolution kernel and the image matrix is multiplied by the corresponding pixel, and a sum obtained by adding, as shown in FIG. (Not to be confused matrix multiplication)
Convolution mode : convolution has three modes, FULL, SAME, VALID, a distinction that the actual mode of use. (Reference: https: //zhuanlan.zhihu.com/p/62760780)
Full: full convolution, Full mode means that the filter intersects just started, and image convolution, the white portion is filled 0, orange part of image, part of the blue filter, the range of motion filter shown in FIG.
Same convolution: when the filter coincides with the corners of the anchor (K) of the image, convolution operation started , the visible range of motion is smaller than the full mode filter circle, same mode MODE is a subset of the full, i.e., full the result of the convolution mode includes same mode.
Convolution valid: When all the image inside the filter when the convolution operation , the movement range of the visible filter Same smaller than, the same is valid MODE subset of the same mode. The convolution calculation valid mode, a pixel value is not involved in filling boundary calculation, that does not affect the invalid padded boundary convolution, so called valid mode.
the python scipy package provided convolve2d () function to achieve convolution with the following parameters:
from SciPy Import Signal signal.convolve2d (the src, Kernel, MODE, boundary, fillvalue) the src: input image matrix, only supports single-pass (i.e., two-dimensional matrix) Kernel: convolution kernel mode: Convolution Type: full, same , Valid boundary: boundary fill mode: fill, wrap, Symm fillvalue: when the boundary is fill, fill boundary value defaults to 0
opencv中提供了flip()函数翻转卷积核，filter2D进行same 卷积, 其参数如下：
dst = cv2.flip(src,flipCode) src: 输入矩阵 flipCode:0表示沿着x轴翻转，1表示沿着y轴翻转，-1表示分别沿着x轴，y轴翻转 dst:输出矩阵（和src的shape一样） cv2.filter2D(src,dst,ddepth,kernel,anchor=(-1,-1),delta=0,borderType=cv2.BORDER_DEFAULT) src: 输入图像对象矩阵 dst:输出图像矩阵 ddepth:输出矩阵的数值类型 kernel:卷积核 anchor：卷积核锚点，默认(-1,-1)表示卷积核的中心位置 delat:卷积完后相加的常数 borderType:填充边界类型
2 图像平滑
2.1 高斯平滑
高斯平滑即采用高斯卷积核对图像矩阵进行卷积操作。高斯卷积核是一个近似服从高斯分布的矩阵，随着距离中心点的距离增加，其值变小。这样进行平滑处理时，图像矩阵中锚点处像素值权重大，边缘处像素值权重小，下为一个3*3的高斯卷积核：
opencv中提供了GaussianBlur()函数来进行高斯平滑，其对应参数如下：
dst = cv2.GaussianBlur(src,ksize,sigmaX,sigmay,borderType)
src: 输入图像矩阵,可为单通道或多通道，多通道时分别对每个通道进行卷积
dst:输出图像矩阵,大小和数据类型都与src相同
ksize:高斯卷积核的大小，宽，高都为奇数，且可以不相同
sigmaX: 一维水平方向高斯卷积核的标准差
sigmaY: 一维垂直方向高斯卷积核的标准差，默认值为0，表示与sigmaX相同
borderType:填充边界类型
代码使用示例和效果如下：(相比于原图，平滑后图片变模糊)
#coding:utf-8 import cv2 as cv import matplotlib.pyplot as plt import numpy as np img = cv.imread(r"C:\Users\Administrator\Desktop\timg.jpg") img_gauss = cv.GaussianBlur(img,(3,3),1) cv.imshow("img",img) cv.imshow("img_gauss",img_gauss) cv.waitKey(0) cv.destroyAllWindows()
对于上面的高斯卷积核，可以由如下两个矩阵相乘进行构建，说明高斯核是可分离卷积核，因此高斯卷积操作可以分成先进行垂直方向的一维卷积，再进行一维水平方向卷积。
opencv中getGaussianKernel()能用来产生一维的高斯核，分别获得水平和垂直的高斯核，分两步也能完成高斯卷积，获得和GaussianBlur一样的结果。其参数如下：
cv2.getGaussianKernel(ksize,sigma,ktype)
ksize:奇数，一维核长度
sigma:标准差
ktype:数据格式，应该为CV_32F 或者 CV_64F
返回矩阵如下：垂直的矩阵
[[ 0.27406862]
[ 0.45186276]
[ 0.27406862]
#coding:utf-8 import cv2 as cv import matplotlib.pyplot as plt import numpy as np from scipy import signal #convolve2d只是对单通道进行卷积，若要实现cv.GaussianBlur()多通道高斯卷积，需要拆分三个通道进行，再合并 def gaussianBlur(img,h,w,sigma,boundary="fill",fillvalue=0): kernel_x = cv.getGaussianKernel(w,sigma,cv.CV_64F) #默认得到的为垂直矩阵 kernel_x = np.transpose(kernel_x) #转置操作，得到水平矩阵 #水平方向卷积 gaussian_x = signal.convolve2d(img,kernel_x,mode="same",boundary=boundary,fillvalue=fillvalue) #垂直方向卷积 kernel_y = cv.getGaussianKernel(h,sigma,cv.CV_64F) gaussian_xy = signal.convolve2d(gaussian_x,kernel_y,mode="same",boundary=boundary,fillvalue=fillvalue) #cv.CV_64F数据转换为uint8 gaussian_xy = np.round(gaussian_xy) gaussian_xy = gaussian_xy.astype(np.uint8) return gaussian_xy if __name__=="__main__": img = cv.imread(r"C:\Users\Administrator\Desktop\timg.jpg",0) img_gauss = gaussianBlur(img,3,3,1) cv.imshow("img",img) cv.imshow("img_gauss",img_gauss) cv.waitKey(0) cv.destroyAllWindows()
2.2 均值平滑
高斯卷积核，对卷积框中像素值赋予不同权重，而均值平滑赋予相同权重，一个3*5的均值卷积核如下，均值卷积核也是可分离的。
opencv的boxFilter()函数和blur()函数都能用来进行均值平滑，其参数如下：
cv2.boxFilter(src,ddepth,ksize,dst,anchor,normalize,borderType) src: 输入图像对象矩阵, ddepth:数据格式,位深度 ksize:高斯卷积核的大小，格式为(宽，高) dst:输出图像矩阵,大小和数据类型都与src相同 anchor：卷积核锚点，默认(-1,-1)表示卷积核的中心位置 normalize:是否归一化 （若卷积核3*5，归一化卷积核需要除以15） borderType:填充边界类型 cv2.blur(src,ksize,dst,anchor,borderType) src: 输入图像对象矩阵,可以为单通道或多通道 ksize:高斯卷积核的大小，格式为(宽，高) dst:输出图像矩阵,大小和数据类型都与src相同 anchor：卷积核锚点，默认(-1,-1)表示卷积核的中心位置 borderType:填充边界类型
示例代码和使用效果如下：
#coding:utf-8 import cv2 as cv import matplotlib.pyplot as plt import numpy as np img = cv.imread(r"C:\Users\Administrator\Desktop\timg.jpg") img_blur = cv.blur(img,(3,5)) # img_blur = cv.boxFilter(img,-1,(3,5)) cv.imshow("img",img) cv.imshow("img_blur",img_blur) cv.waitKey(0) cv.destroyAllWindows()
2.3 中值平滑
中值平滑也有核，但并不进行卷积计算，而是对核中所有像素值排序得到中间值，用该中间值来代替锚点值。opencv中利用medianBlur()来进行中值平滑，中值平滑特别适合用来去除椒盐噪声，其参数如下：
cv2.medianBlur(src,ksize,dst) src: 输入图像对象矩阵,可以为单通道或多通道 ksize:核的大小，格式为 3 #注意不是（3,3） dst:输出图像矩阵,大小和数据类型都与src相同
其使用代码及效果如下：（加上的白点噪声都被平滑掉了）
#coding:utf-8 import cv2 as cv import matplotlib.pyplot as plt import numpy as np import random img = cv.imread(r"C:\Users\Administrator\Desktop\timg.jpg") rows,cols = img.shape[:2] #加入椒盐噪声 for i in range(100): r = random.randint(0,rows-1) c = random.randint(0,cols-1) img[r,c]=255 img_medianblur = cv.medianBlur(img,5) cv.imshow("img",img) cv.imshow("img_medianblur",img_medianblur) cv.waitKey(0) cv.destroyAllWindows()
2.4 双边滤波
相比于上面几种平滑算法，双边滤波在平滑的同时还能保持图像中物体的轮廓信息。双边滤波在高斯平滑的基础上引入了灰度值相似性权重因子，所以在构建其卷积核核时，要同时考虑空间距离权重和灰度值相似性权重。在进行卷积时，每个位置的邻域内，根据和锚点的距离d构建距离权重模板，根据和锚点灰度值差异r构建灰度值权重模板，结合两个模板生成该位置的卷积核。opencv中的bilateralFilter()函数实现了双边滤波，其参数对应如下：
dst = cv2.bilateralFilter(src,d,sigmaColor,sigmaSpace,borderType) src: 输入图像对象矩阵,可以为单通道或多通道 d:用来计算卷积核的领域直径，如果d<=0，从sigmaSpace计算d sigmaColor：颜色空间滤波器标准偏差值，决定多少差值之内的像素会被计算（构建灰度值模板） sigmaSpace:坐标空间中滤波器标准偏差值。如果d>0，设置不起作用，否则根据它来计算d值（构建距离权重模板）
其使用代码及效果如下：
#coding:utf-8 import cv2 as cv import matplotlib.pyplot as plt import numpy as np import random import math img = cv.imread(r"C:\Users\Administrator\Desktop\timg.jpg") img_bilateral = cv.bilateralFilter(img,0,0.2,40) cv.imshow("img",img) cv.imshow("img_bilateral",img_bilateral) cv.waitKey(0) cv.destroyAllWindows()
同样，利用numpy也可以自己实现双边滤波算法，同样需要对每个通道进行双边滤波，最后进行合并，下面代码只对单通道进行了双边滤波，代码和效果如下图：
#coding:utf-8 import cv2 as cv import matplotlib.pyplot as plt import numpy as np import random import math def getDistanceWeight(sigmaSpace,H,W): r,c = np.mgrid[0:H:1,0:W:1] r = r-(H-1)/2 c =c-(W-1)/2 distanceWeight = np.exp(-0.5*(np.power(r,2)+np.power(c,2))/math.pow(sigmaSpace,2)) return distanceWeight def bilateralFilter(img,H,W,sigmaColor,sigmaSpace): distanceWeight = getDistanceWeight(sigmaSpace,H,W) cH = (H-1)/2 cW = (W-1)/2 rows,cols = img.shape[:2] bilateralImg = np.zeros((rows,cols),np.float32) for r in range(rows): for c in range(cols): pixel = img[r,c] rTop = 0 if r-cH<0 else r-cH rBottom = rows-1 if r+cH>rows-1 else r+cH cLeft = 0 if c-cW<0 else c-cW cRight = cols-1 if c+cW>cols-1 else c+cW #权重模板作用区域 region=img[rTop:rBottom+1,cLeft:cRight+1] #灰度值差异权重 colorWeight = np.exp(0.5*np.power(region-pixel,2.0)/math.pow(sigmaColor,2)) print(colorWeight.shape) #距离权重 distanceWeightTemp = distanceWeight[cH-(r-rTop):rBottom-r+cH+1,cW-(c-cLeft):cRight-c+cW+1] print(distanceWeightTemp.shape) #权重相乘并归一化 weightTemp = colorWeight*distanceWeightTemp weightTemp = weightTemp/np.sum(weightTemp) bilateralImg[r][c]=np.sum(region*weightTemp) return bilateralImg if __name__=="__main__": img = cv.imread(r"C:\Users\Administrator\Desktop\timg.jpg",0) img_temp = img/255.0 img_bilateral = bilateralFilter(img_temp,3,3,0.2,19)*255 img_bilateral[img_bilateral>255] = 255 img_bilateral = img_bilateral.astype(np.uint8) cv.imshow("img",img) cv.imshow("img_bilateral",img_bilateral) cv.waitKey(0) cv.destroyAllWindows()
2.5 联合双边滤波
双边滤波是根据原图中不同位置灰度相似性来构建相似性权重模板，而联合滤波是先对原图进行高斯平滑，然后根据平滑后的图像灰度值差异建立相似性模板，再与距离权重模板相乘得到最终的卷积核，最后再对原图进行处理。所以相比于双边滤波，联合双边滤波只是建立灰度值相似性模板的方法不一样。
联合双边滤波作为边缘保留滤波算法时，进行joint的图片即为自身原图片，如果将joint换为其他引导图片，联合双边滤波算法还可以用来实现其他功能。opencv 2中不支持联合双边滤波，opencv 3中除了主模块，还引入了contrib，其中的ximgproc模块包括了联合双边滤波的算法。因此如果需要使用opencv的联合双边滤波，需要安装opencv-contrib-python包。
安装opencv主模块和contrib附加模块步骤： pip uninstall opencv-python （如果已经安装opencv-python包，先卸载） pip install opencv-contrib-python
联合双边滤波: cv2.xmingproc.jointBilateralFilter(), 其相关参数如下：
dst = cv2.xmingproc.jointBilateralFilter(joint,src,d,sigmaColor,sigmaSpace,borderType) joint: 进行联合滤波的导向图像，可以为单通道或多通道，保持边缘的滤波算法时常采用src src: 输入图像对象矩阵,可以为单通道或多通道 d:用来计算卷积核的领域直径，如果d<0，从sigmaSpace计算d sigmaColor：颜色空间滤波器标准偏差值，决定多少差值之内的像素会被计算（构建灰度值模板） sigmaSpace:坐标空间中滤波器标准偏差值。如果d>0，设置不起作用，否则根据它来计算d值（构建距离权重模板）
下面是联合双边滤波的使用代码和效果：（采用src的高斯平滑图片作为joint）
#coding:utf-8 import cv2 as cv import matplotlib.pyplot as plt import numpy as np import random import math src = cv.imread(r"C:\Users\Administrator\Desktop\timg.jpg") joint = cv.GaussianBlur(src,(7,7),1,0) dst = cv.ximgproc.jointBilateralFilter(joint,src,33,2,0) # dst = cv.ximgproc.jointBilateralFilter(src,src,33,2,0) #采用src作为joint cv.imshow("img",src) cv.imshow("joint",joint) cv.imshow("dst",dst) cv.waitKey(0) cv.destroyAllWindows()
2.6 导向滤波
导向滤波也是需要一张图片作为引导图片，来表明边缘，物体等信息，作为保持边缘滤波算法，可以采用自身作为导向图片。opencv 2中也暂不支持导向滤波, 同样在opencv-contrib-python包的ximgproc模块提供了导向滤波函。
导向滤波具体原理可以参考：https://blog.csdn.net/baimafujinji/article/details/74750283
opencv中导向滤波cv2.ximgproc.guidedFilter()的参数如下：
导向滤波 cv2.ximgproc.guidedFilter(guide,src,radius,eps,dDepth) guide: 导向图片，单通道或三通道 src: 输入图像对象矩阵,可以为单通道或多通道 radius:用来计算卷积核的领域直径 eps:规范化参数， eps的平方类似于双边滤波中的sigmaColor（颜色空间滤波器标准偏差值） (regularization term of Guided Filter. eps2 is similar to the sigma in the color space into bilateralFilter.) dDepth: 输出图片的数据深度
其代码使用和效果如下：
#coding:utf-8 import cv2 as cv src = cv.imread(r"C:\Users\Administrator\Desktop\timg.jpg") dst = cv.ximgproc.guidedFilter(src,src,33,2,-1) cv.imshow("img",src) cv.imshow("dst",dst) cv.waitKey(0) cv.destroyAllWindows()
