[3D Reconstruction] [Deep Learning] NeRF image quality objective evaluation indicators: PSNR and SSIM
Tips: Recently, I started researching on [3D reconstruction] and recorded NeRF-related knowledge points. At present, there are three main image quality evaluation indicators for real references: artificially designed SSIM and PSNR, and LPIPS for comparing neural network extraction features. Here we mainly introduce the principles of SSIM and PSNR.
Article Directory
PSNR (peak signal-to-noise ratio)
Peak Signal to Noise Ratio (PSNR) is a measure of image quality. Because the PSNR value has limitations, it is only a reference value for measuring the image quality between the maximum signal and the background noise.
PSNR is defined based on MSE (MeanSquare Error, mean square error), where MSE is defined as:
MSE = 1 H × W ∑ i = 1 H ∑ j = 1 W ( X ( i , j ) − Y ( i , j ) ) 2 MSE = \frac{1}{
{H \times W}}\sum\nolimits_{i = 1}^H {\sum\nolimits_{j = 1}^W { { {\
left
( {X\ left( {i,j} \right) - Y\left( {i,j} \right)} \right)}^2}} }MSE=H×W1∑i=1H∑j=1W(X(i,j)−Y(i,j))2
MSE represents the mean square error of the current image X and the reference image Y, H and W are the height and width of the image respectively
PSNR定义为：
P S N R = 10 log 10 ( M A X x 2 M S E ) = 10 log 10 ( ( 2 n − 1 ) 2 M S E ) PSNR = 10{\log _{10}}\left( {\frac{
{MAX_x^2}}{
{MSE}}} \right) = 10{\log _{10}}\left( {\frac{
{
{
{\left( {
{2^n} - 1} \right)}^2}}}{
{MSE}}} \right) PSNR=10log10(MSEMAXx2)=10log10(MSE(2n−1)2)
MAX x MA{X_x}MAXxis the maximum pixel value of image X: n is the number of bits per pixel, generally 8, that is, the number of grayscale pixels is 256
The unit of PSNR is dB, the smaller the MSE, the larger the PSNR; the larger the PSNR, the better the image quality.
The larger the PSNR value, the better the image quality, generally speaking:
- Greater than 40dB: very close to the original image, the image quality is excellent
- 30 to 40dB: Perceptible but acceptable distortion, good image quality
- 20 to 30dB: Poor image quality
- Below 20dB: Image quality is terrible
SSIM (Structural Similarity Index)
Structural similarity index (SSIM) is an index used to quantify the structural similarity between two images. Compared with PSNR, SSIM is more in line with human visual characteristics in evaluating image quality. SSIM quantifies the properties of an image in terms of brightness, contrast, and structure.
1. Use the mean value as an estimate of brightness:
mean value:
μ x = 1 N ∑ i = 1 N xi {\mu _{\rm{x}}} = \frac{1}{N}\sum\nolimits_{ {
\ rm{i}} = 1}^N {
{x_i}}mx=N1∑i=1Nxi
亮度：
l ( X , Y ) = 2 μ x μ y + c 1 μ x 2 + μ y 2 + c 1 l\left( {X,Y} \right) = \frac{
{2{\mu _{\rm{x}}}{\mu _y} + {c_1}}}{
{\mu _x^2 + \mu _y^2 + {c_1}}} l(X,Y)=mx2+my2+c12 mxmy+c1
N represents the number of pixels, xi {x_i}xiRepresents the pixel value, X and Y represent two different images, c represents a constant to prevent the denominator from being zero
2. Standard deviation as an estimate of contrast:
standard deviation:
σ x = ( 1 N − 1 ∑ i = 1 N ( xi − μ i ) 2 ) 1 2 {\sigma _x} = {\left( {\frac{1 }{
{N - 1}}{
{\sum\nolimits_{
{\rm{i}} = 1}^N {\left( {
{x_i} - {\mu _i}} \right)} }^2} } \right)^{\frac{1}{2}}}px=(N−11∑i=1N(xi−mi)2)21
Formula:
c ( X , Y ) = 2 σ x σ y + c 2 σ x 2 + σ y 2 + c 2 c\left( {X,Y} \right) = \frac{ {2{\sigma_
{ \rm{x}}}{\sigma_y} + {c_2}}}{{
\sigma_x^2+\sigma_y^2+{c_2}}}c(X,Y)=px2+py2+c22 pxpy+c2
3. Covariance as a measure of structural similarity:
Covariance:
σ xy = 1 N − 1 ∑ i = 1 N ( xi − μ ix ) ( yi − μ iy ) {\sigma _{xy}} = \frac{ 1}{
{N - 1}}\sum\nolimits_{
{\rm{i}} = 1}^N {\left( {
{x_i} - \mu _i^x} \right)} \left( {
{ y_i} - \mu _i^y} \right)pxy=N−11∑i=1N(xi−mix)(yi−miy)
Specific function:
s ( X , Y ) = σ xy + c 3 σ x σ y + c 3 s\left( {X,Y} \right) = \frac{ { {\sigma _{
xy
}} + {c_3}}}{
{
{\sigma_x}{\sigma_y} + {c_3}}}s(X,Y)=pxpy+c3pxy+c3
4.The SSIM is one:
SSIM ( X , Y ) = [ l ( X , Y ) α c ( X , Y ) β s ( X , Y ) γ ] SSIM\left( {X,Y} \right) = \ left[ {l{
{\left( {X,Y} \right)}^\alpha }c{ {
\left( {X,Y}\right)}^\beta }s{ {
\left( {X, Y}\right)}^\gamma}}\right]SS I M(X,Y)=[l(X,Y)αc(X,Y)βs(X,Y)γ ]
in β and γ in 1 :
SSIM ( X , Y ) = ( 2 µ x µ y + c 1 ) ( 2 σ xy + c 2 ) ( µ x 2 + µ y 2 + c 1 ) ( σ x 2 + σ y 2 + c 2 ) SSIM\left( {X,Y} \right) = \frac{ {\left( {
2{\in _{\rm{x}}}{\in _y } + {c_1}} \right)\left( {2{\sigma_{xy}} + {c_2}} \right)}}{ {\left( {\mu _x^2 + \mu _y^2
+ {c_1}}\right)\left({\sigma_x^2+\sigma_y^2+{c_2}}\right)}}SS I M(X,Y)=( mx2+my2+c1)( px2+py2+c2)( 2 mxmy+c1)( 2 pxy+c2)
The SSIM value ranges from 0 to 1, and the larger the image, the more similar it is.
Summarize
Introduce the principles of PSNR and SSIM evaluation indicators as simply and in detail as possible.