guide
Sometimes we want to calculate whether two pictures are similar, and there are many algorithms used to measure the similarity of two pictures, such as: RMSE, PSNR, SSIM, UQI, SIFTand 深度学习so on. This article mainly introduces, RMSE, PSNR, SSIM, UQIthe calculation and application of these indicators, and the algorithm SIFTto calculate the similarity of images will be introduced in a later article
There are also many factors that affect the results of these algorithms, such as pictures 噪声, , 平移, 缩放, 旋转, 裁剪etc. 透视变换, will affect the final result of the algorithm, so we need to choose different algorithms according to different application scenarios
MSE and RMSE
MSE(mean squared error): the square error of the image pixel value
RMSE(root mean squared error): the square root error of the image pixel value
- shortcoming
- Compare the size of the two pictures must be exactly the same
- Sensitive to scaling, rotation, cropping
- advantage
- calculation speed
- simple algorithm
- adapt to the scene
- Compare the difference between the two frames before and after in the video
- Calculate the similarity of pictures
- Code
def mean_squared_error(img1,img2):
#判断两张图片的shape是否一致
assert img1.shape == img2.shape,"images have different shape " \
"{} and {}".format(img1.shape,img2.shape)
#计算两张图像的MSE
img_mse = np.mean((img1.astype(np.float64) - img2.astype(np.float64))**2)
#将图像的MSE量化为相似度
img_sim = img_mse / (255**2)
return img_mse,1 - img_sim
def root_mean_squared_error(img1,img2):
#判断两张图片的shape是否一致
assert img1.shape == img2.shape,"images have different shape " \
"{} and {}".format(img1.shape,img2.shape)
#计算两张图像的MSE
img_rmse = np.sqrt(np.mean((img1.astype(np.float64) - img2.astype(np.float64))**2))
#将图像的MSE量化为相似度
img_sim = img_rmse / 255
return img_rmse,1 - img_sim
MSECalculate the sum of two imagesRMSE
print(mean_squared_error(img1,img2))
#(10826.879833697936, 0.8334966576901509)
print(root_mean_squared_error(img1,img2))
#(104.05229374549096, 0.5919517892333688)
PSNR
PSNR(Peak Signal Noise Ratio) is also called 峰值信噪比: In order to measure the quality of the processed image, we often use PSNR to measure whether the processing result of the program is satisfactory. It is the mean square error between the original image and the processed image relative to ( 2 n − 1 ) 2 (2^n-1)^2(2n−1)The logarithmic value of 2 (the square of the maximum value of the signal, n is the number of bits per sample value), the calculation formula is as follows:
PSNR = 10 ∗ log 10 ( 2 n − 1 ) 2 MSE PSNR = 10 * log_{10}\frac {(2^n-1)^2}{MSE}PSNR=10∗log10MSE(2n−1)2
-
Disadvantages
PSNR scores cannot be completely consistent with the quality seen by the human eye -
Advantages
The algorithm is simple and widely used -
Applicable scene
The measurement method used to evaluate the objective quantity of image quality can also be used for the calculation of image similarity -
Code
def peak_signal_noise_ratio(img1,img2):
#判断两张图片的size是否一致
assert img1.shape == img2.shape,"images hava different shape " \
"{} and {}".format(img1.shape,img2.shape)
#获取数据类型所表示的最大值
MAX = np.iinfo(img1.dtype).max
#计算两张图片的MSE
mse,_ = mean_squared_error(img1,img2)
#计算两张图片的PSNR
psnr = 10 * np.log10(MAX**2 / mse)
return psnr
print(peak_signal_noise_ratio(img1,img2))
#49.24985796979349
注: Since PSNRthe value range is (0,inf), PSNR的值越大表示图像越相似. If you want to quantify the PSNR index into similarity, you can set a maximum PSNR value based on the test picture, and perform the maximum operation to convert it into similarity
SSIM
SSIM(structural similarity index) Structural similarity: It is an index used to measure the similarity of two images. When one of the two images is an undistorted image and the other is a distorted image, the quality of the two images 结构相似性can be regarded as a quality indicator of the distorted image. Structural similarity relative to peak signal-to-noise ratio,The structural similarity index is more in line with the human eye's judgment on image quality in the measurement of image quality。
with xxx和yyFor each case of y
, we have the following functional coordinates: SSIM ( x , y ) = ( l ( x , y ) ) α ( c ( x , y ) ) β ( s ( x , y ) ) γ l ( x , y ) = 2 μ x μ y + C 1 μ x 2 + μ y 2 + C 1 c ( x , y ) = 2 δ x δ y + C 2 δ x 2 + δ y 2 + C 2 s ( x , y ) = δ xy + C 3 δ x δ y + C 3 \begin{aligned} SSIM(x,y) &= (l(x,y))^\alpha(c(x,y)) ^\beta(s(x,y))^\gamma \\ l(x,y) &= \frac{2\in_x\in_y + C_1}{\in_x^2+\in_y^2+C_1} \\ c(x,y) &= \frac{2\delta_x\delta_y + C_2}{\delta_x^2+\delta_y^2+C_2}\\ s(x,y) &= \frac{\delta_{xy} +C_3}{\delta_x\delta_y+C_3} \end{aligned}S S I M ( x ,y)l(x,y)c(x,y)s(x,y)=(l(x,y))a (c(x,y))b (s(x,y))c=mx2+my2+C12 mxmy+C1=dx2+dy2+C22 dxdy+C2=dxdy+C3dxy+C3
l ( x , y ) l(x,y) in the above formulal(x,y ) compares the brightness between two signals,c ( x , y ) c(x,y)c(x,y ) compares the contrast of two signals,s ( x , y ) s(x,y)s(x,y ) compares the structure of two signals, whereα > 0 , β > 0 , γ > 0 \alpha > 0, \beta > 0, \gamma > 0a>0 ，b>0 ，c>0 is mainly used to adjustl ( x , y ) , c ( x , y ) , s ( x , y ) l(x,y), c(x,y), s(x,y)l(x,y)、c(x,y)、s(x,y ) ,μ x and μ y \mu_x and \mu_ymxand μyRepresents the average value of the signal, δ x and δ y \delta_x and \delta_ydxand δyRepresents the standard deviation of the signal, and δ xy \delta_{xy}dxyis the covariance of the signal, C 1 , C 2 , C 3 C_1, C_2, C_3C1、C2、C3All are constants, used to maintain l ( x , y ) , c ( x , y ) , s ( x , y ) l(x,y), c(x,y), s(x,y)l(x,y)、c(x,y)、s(x,y ) stability,SSIM的值越大，表示两个信号之间的相似度越高
When calculating the similarity of image structure, we generally set the parameters to α = β = γ = 1 and C 3 = C 2 / 2 \alpha=\beta=\gamma=1 and C_3=C_2/2a=b=c=1 and C3=C2/ 2 soSSIM SSIMS S I M smooth scattering and smoothing
the SSIM ( x , y ) = ( 2 µ x µ y + C 1 ) ( 2 δ xy + C 2 ) ) ( µ x 2 + µ y 2 + C 1 ). ) ( δ x 2 + δ y 2 + C 2 ) SSIM(x,y) = \frac{(2\mu_x\mu_y+C_1)(2\delta_{xy}+C_2))}{(\mu_x^2 +\mu_y^2+C_1)(\delta_x^2+\delta_y^2+C_2)}S S I M ( x ,y)=( mx2+my2+C1) ( dx2+dy2+C2)( 2 mxmy+C1) ( 2 dxy+C2))
Usually when computing the SSIM of an image SSIMWhen S S I M , we will use mobilefilterto calculate
- Code
from scipy import signal
from enum import Enum
def filter2(img,fltr,mode='same'):
return signal.convolve2d(img, np.rot90(fltr,2), mode=mode)
def _get_sums(GT,P,win,mode='same'):
mu1,mu2 = (filter2(GT,win,mode),filter2(P,win,mode))
return mu1*mu1, mu2*mu2, mu1*mu2
def _get_sigmas(GT,P,win,mode='same',**kwargs):
if 'sums' in kwargs:
GT_sum_sq,P_sum_sq,GT_P_sum_mul = kwargs['sums']
else:
GT_sum_sq,P_sum_sq,GT_P_sum_mul = _get_sums(GT,P,win,mode)
return filter2(GT*GT,win,mode) - GT_sum_sq,\
filter2(P*P,win,mode) - P_sum_sq, \
filter2(GT*P,win,mode) - GT_P_sum_mul
class Filter(Enum):
UNIFORM = 0
GAUSSIAN = 1
def fspecial(fltr,ws,**kwargs):
if fltr == Filter.UNIFORM:
return np.ones((ws,ws))/ ws**2
elif fltr == Filter.GAUSSIAN:
x, y = np.mgrid[-ws//2 + 1:ws//2 + 1, -ws//2 + 1:ws//2 + 1]
g = np.exp(-((x**2 + y**2)/(2.0*kwargs['sigma']**2)))
g[ g < np.finfo(g.dtype).eps*g.max() ] = 0
assert g.shape == (ws,ws)
den = g.sum()
if den !=0:
g/=den
return g
return None
def _ssim_single(GT, P, ws, C1, C2, fltr_specs, mode):
win = fspecial(**fltr_specs)
GT_sum_sq, P_sum_sq, GT_P_sum_mul = _get_sums(GT, P, win, mode)
sigmaGT_sq, sigmaP_sq, sigmaGT_P = _get_sigmas(GT, P, win, mode, sums=(GT_sum_sq, P_sum_sq, GT_P_sum_mul))
assert C1 > 0
assert C2 > 0
ssim_map = ((2 * GT_P_sum_mul + C1) * (2 * sigmaGT_P + C2)) / (
(GT_sum_sq + P_sum_sq + C1) * (sigmaGT_sq + sigmaP_sq + C2))
cs_map = (2 * sigmaGT_P + C2) / (sigmaGT_sq + sigmaP_sq + C2)
return np.mean(ssim_map), np.mean(cs_map)
def ssim(GT,P,ws=11,K1=0.01,K2=0.03,MAX=None,fltr_specs=None,mode='valid'):
if MAX is None:
MAX = np.iinfo(GT.dtype).max
assert GT.shape == P.shape, "Supplied images have different sizes " + \
str(GT.shape) + " and " + str(P.shape)
if fltr_specs is None:
fltr_specs=dict(fltr=Filter.UNIFORM,ws=ws)
C1 = (K1*MAX)**2
C2 = (K2*MAX)**2
ssims = []
css = []
for i in range(GT.shape[2]):
ssim,cs = _ssim_single(GT[:,:,i],P[:,:,i],ws,C1,C2,fltr_specs,mode)
ssims.append(ssim)
css.append(cs)
return np.mean(ssims),np.mean(css)
img_sim,_ = ssim(img1,img2)
#0.9459787655432684
QI
UQI(Universal Quality Image Index) is also called 图像通用质量指标: it can be easily calculated and applied to various image processing applications, mainly combined with three factors to calculate, 相关性的损失, 亮度失真, 对比度失真. Although UQIthe metrics are defined mathematically it does not take advantage of the human visual system, but manifests itself through various image distortions UQI与主观质量测量有着惊人的一致性.
UQIGive the equation as follows:
Q = 4 δ xyx ^ y ^ ( δ x 2 + δ y 2 ) [ ( x ^ ) 2 + ( y ^ ) 2 ] x ^ = 1 N ∑ i = 1 N xiy ^ = N ∑ i = 1 N yi δ x 2 = 1 N − 1 ∑ i = 1 N ( xi − x ^ ) 2 δ y 2 = 1 N − 1 ∑ i = 1 N ( yi − y ^ ) 2 \begin{ aligned} Q &= \frac{4\delta_{xy}\hat{x}\hat{y}}{(\delta^2_x+\delta^2_y)[(\hat{x})^2+(\hat {y})^2]} \\\that{x} &= \frac{1}{N}\sum_{i=1}^{N}x_i \\that{y} &= \frac{1 }{N}\sum_{i=1}^{N}y_i \\\delta_{x}^2 &= \frac{1}{N - 1}\sum_{i=1}^{N}(x_i -\hat{x})^2\\\delta_{y}^2 &= \frac{1}{N-1}\sum_{i=1}^{N}(y_i-\hat{y}) ^2 \end{aligned}Qx^y^dx2dy2=( dx2+dy2)[(x^)2+(y^)2]4 dxyx^y^=N1i=1∑Nxi=N1i=1∑Nyi=N−11i=1∑N(xi−x^)2=N−11i=1∑N(yi−y^)2
QThe value range is between [-1,1]
- Code
import numpy as np
from scipy.ndimage import uniform_filter
def _uqi_single(GT,P,ws):
N = ws**2
window = np.ones((ws,ws))
GT_sq = GT*GT
P_sq = P*P
GT_P = GT*P
GT_sum = uniform_filter(GT, ws)
P_sum = uniform_filter(P, ws)
GT_sq_sum = uniform_filter(GT_sq, ws)
P_sq_sum = uniform_filter(P_sq, ws)
GT_P_sum = uniform_filter(GT_P, ws)
GT_P_sum_mul = GT_sum*P_sum
GT_P_sum_sq_sum_mul = GT_sum*GT_sum + P_sum*P_sum
numerator = 4*(N*GT_P_sum - GT_P_sum_mul)*GT_P_sum_mul
denominator1 = N*(GT_sq_sum + P_sq_sum) - GT_P_sum_sq_sum_mul
denominator = denominator1*GT_P_sum_sq_sum_mul
q_map = np.ones(denominator.shape)
index = np.logical_and((denominator1 == 0) , (GT_P_sum_sq_sum_mul != 0))
q_map[index] = 2*GT_P_sum_mul[index]/GT_P_sum_sq_sum_mul[index]
index = (denominator != 0)
q_map[index] = numerator[index]/denominator[index]
s = int(np.round(ws/2))
return np.mean(q_map[s:-s,s:-s])
def uqi (GT,P,ws=8):
if len(GT.shape) == 2:
GT = GT[:, :, np.newaxis]
P = P[:, :, np.newaxis]
GT = GT.astype(np.float64)
P = P.astype(np.float64)
return np.mean([_uqi_single(GT[:,:,i],P[:,:,i],ws) for i in range(GT.shape[2])])
Summarize
In addition to the above image similarity evaluation index, Sewar also provides many other image quality evaluation index algorithms, such as: MS-SSIM, ERGAS, , SCC, RASE, SAM, D_lambda, D_S, QNRand .VIFPSNR-B
Let's compare the difference between the calculated similarity results of some commonly used indicators when different noises are added. To
add noise to pictures, you can refer to my article on image addition (Gaussian / salt and pepper / Poisson / spots) noise
| evaluation algorithm | original image | Gaussian noise | Gaussian blur | Poisson noise | salt and pepper noise | speckle noise |
|---|---|---|---|---|---|---|
| MSE | 0.00 | 548.66 | 79.78 | 0.77 | 885.12 | 10826.88 |
| RMSE | 0.00 | 23.42 | 8.93 | 0.88 | 29.75 | 104.05 |
| PSNR | inf | 20.74 | 29.11 | 49.25 | 18.66 | 7.79 |
| SSIM | 1.00 | 0.25 | 0.94 | 0.99 | 0.45 | 0.06 |
| QI | 1.00 | 0.98 | 0.99 | 1.00 | 0.98 | 0.80 |
| MS-SSIM | 1.00+0.00u | 0.76+0.00u | 0.98+0.00u | 1.00+0.00u | 0.86+0.00u | 0.32+0.00u |
| LIVELY | 0.00 | 8031.33 | 2322.77 | 225.79 | 11187.33 | 26741.73 |
| SCC | 0.89 | 0.08 | 0.17 | 0.48 | 0.35 | 0.02 |
| RASE | 0.00 | 1158.13 | 333.25 | 32.21 | 1600.90 | 3814.14 |
| SAM | 0.00 | 0.13 | 0.05 | 0.00 | 0.13 | 0.58 |
| D_lambda | 0.00 | 0.02 | 0.00 | 0.00 | 0.03 | 0.20 |
| VIFP | 1.00 | 0.23 | 0.49 | 0.93 | 0.44 | 0.06 |
| PSNR-B | inf | 20.70 | 28.93 | 49.59 | inf | 8.10 |