SSIM study

SSIM original link:https://www.researchgate.net/profile/Eero-Simoncelli/publication/3327793_Image_Quality_Assessment_From_Error_Visibility_to_Structural_Similarity/links/542173b20cf203f155c6bf1a/Image-Quality-Assessment-From-Error-Visibility-to-Structural-Similarity.pdf

---

SSIM purpose

SSIM is a structural similarity index (Structural Similarity Index), which is an index to measure the similarity of two images. In deep learning, SSIM is often used to evaluate the similarity between images generated by generative models (such as GAN) and real images.

SSIM calculates a similarity score between two images by comparing their similarity in terms of brightness, contrast, and structure. It is based on the human eye's perception of the image, which can better reflect the image quality, and is more robust and accurate than traditional error measurement methods such as mean square error.

With the naked eye, we can observe the difference between the pictures under the same MSE and different SSIM:

SSIM calculation formula

The calculation of SSIM is mainly determined by three key features of the picture, namely: brightness (Luminance), contrast (Contrast), and structure (Structure).

1. Brightness

The brightness is described by the gray value of a single pixel. x represents the gray value of a pixel in a certain channel of the predict image, and y represents the gray value of the corresponding pixel in the corresponding channel of the label image. We calculate x, y Sample mean to get the term corresponding to brightness:

We can find that $m_x=m_y$When $l=1$ , the introduction of C1 is to prevent the mean value from being 0.

2. Contrast

Contrast is described by the degree of deviation of the pixel from the mean value. You can imagine that if the contrast of a picture is greater, the more pixels it is away from the mean value of the gray scale. We obtain the term corresponding to the contrast by calculating the sample standard deviation:

3. Structure

The structure is described in terms of linear dependencies of pixels. We obtain the terms of the corresponding structure by finding the covariance:

4. SSIM final formula

The final calculation formula of SSIM is:

$SSIM(x,y)=[l(x,y){]}^a[c(x,y){]}^{\beta }\left[s\right(x,y){]}^c$

Among them, x and y represent the two images to be compared, l(x, y), c(x, y), s(x, y) are brightness similarity, contrast similarity and structure similarity respectively, α, β and γ are weighting coefficients, generally 1.

We will calculate $l,c,s$ into the formula to get the following:

In practical applications, since pixels are always positive values, such as true color RGB 24bit, the value of SSIM is usually between 0 and 1, and the closer to 1, the more similar the two images are.