[Image evaluation indicators and codes] PSNR, SSIM, FID, NMSE explanations and codes

Evaluation indicators and calculations

Article directory

Peak Signal to Noise Ratio (PSNR):

"Peak Signal-to-Noise Ratio", which means peak signal-to-noise ratio in Chinese, is one of the indicators for measuring image quality. It is used to measure the similarity and distortion between the reconstructed image and the original image. PSNR can only provide information about the overall quality of the image and cannot capture details and perceptual differences.

According to the formula, it can be seen that when the MSE is smaller, the quality is better, and the PSNR is larger, the better.
Insert image description here

Obtained from the mean square error, MAX1 is the largest value in the pixel, usually 1 or 255.

def psnr(res,gt):
    mse = np.mean((res - gt) ** 2)
    if(mse == 0):
        return 100
    max_pixel = 1
    psnr = 20 * log10(max_pixel / sqrt(mse))
    return psnr

Structural Similarity (SSIM):

This method is also called (Structural Similarity) and is an indicator used to measure the similarity of two images. The SSIM method is evaluated by viewing an image as multiple regions and comparing statistics such as structure, brightness, and contrast within these regions. The SSIM algorithm takes into account the characteristics of human image perception and is more in line with the visual characteristics of the human eye. The SSIM algorithm is relatively complex and requires comparison of local structure information and global structure information of the image. Given two images x and y, the formula for the structural similarity of the two images can be expressed as Equation 2-7.

Insert image description here

$SSIM" (x,y)=(2μ_x μ_y+c_1 )(2σ_xy+c_2 )/(μ_x^2+μ_y^2+c_1 )(σ_x^2+σ_y^2+c_2 )$

The returned value is: SSIM score in the range [0, 1]. The closer to 1, the more similar the two images are.

from skimage.metrics import structural_similarity as ssim

def calculate_ssim(img1, img2):
    # 计算 SSIM
    ssim_score = ssim(img1, img2, multichannel=True)
    
    return ssim_score

# 示例用法
img1 = ...  # 第一幅图像数据，类型为numpy数组，范围为0-1
img2 = ...  # 第二幅图像数据，类型为numpy数组，范围为0-1

score = calculate_ssim(img1, img2)
print("SSIM score:", score)

# 或者使用下面代码也可以
#-*- codeing = utf-8 -*-
#@Time ： 2023/4/27 0027 14:43
#@Author : Tom
#@File : SSIM.py
#@Software : PyCharm

import torch
import torch.nn.functional as F

def ssim(img1, img2, window_size=11, size_average=True, full=False):
    # 常量，用于计算均值和方差
    C1 = (0.01 * 255) ** 2
    C2 = (0.03 * 255) ** 2

    # 平均池化滤波器
    window = torch.ones(3, 1, window_size, window_size).float().cuda()
    mu1 = F.conv2d(img1, window, padding=window_size//2, groups=3)
    mu2 = F.conv2d(img2, window, padding=window_size//2, groups=3)

    # 平均值和方差
    mu1_sq, mu2_sq, mu1_mu2 = mu1 ** 2, mu2 ** 2, mu1 * mu2
    sigma1_sq = F.conv2d(img1 ** 2, window, padding=window_size//2, groups=3) - mu1_sq
    sigma2_sq = F.conv2d(img2 ** 2, window, padding=window_size//2, groups=3) - mu2_sq
    sigma12 = F.conv2d(img1 * img2, window, padding=window_size//2, groups=3) - mu1_mu2

    # SSIM 的计算公式
    ssim_map = ((2 * mu1_mu2 + C1) * (2 * sigma12 + C2)) / ((mu1_sq + mu2_sq + C1) * (sigma1_sq + sigma2_sq + C2))
    if size_average:
        ssim_value = torch.mean(ssim_map)
    else:
        ssim_value = torch.mean(ssim_map, dim=(1, 2, 3))

    # 如果设置了 full=True，返回 SSIM 的值和 SSIM 图像的各通道值
    if full:
        return ssim_value, ssim_map
    else:
        return ssim_value


img1 = torch.ones(1, 3, 256, 256).cuda()
img2 = torch.ones(1, 3, 256, 256).cuda()

ssim_value = ssim(img1, img2, window_size=11)
print(ssim_value)

Normalized mean square error (NMSE):

Normalized Mean Squared Error

The value of NMSE is always between 0 and 1 (inclusive). The smaller the value, the better the performance of the model.

def nmse(res,gt):
    Norm = np.linalg.norm((gt * gt),ord=2) # 计算 'gt' 的平方的2-范数（也就是欧几里得长度），存储在变量 'Norm' 中。这一步是为了避免在分母中出现零，从而导致除零错误
    if np.all(Norm == 0): # 如果 'Norm' 等于零（也就是说，如果 'gt' 是全零向量），那么函数直接返回0。这是因为对于全零向量，任何误差都会得到无穷大的NMSE。
        return 0 
    else:
        nmse = np.linalg.norm(((res - gt) * (res - gt)),ord=2) / Norm # 算 'res' 和 'gt' 的差的平方的2-范数，然后除以 'Norm'。这就是NMSE的定义：误差的平方和除以目标值的平方和。
    return nmse

Frecher distance (FID)

Fréchet Inception Distance (FID) is an indicator used to measure the difference between the image generated by the generative model and the real image. It is calculated by comparing their distribution distance in the feature space of the Inception v3 [32] model. The feature vector used by FID is the high-dimensional vector output by the penultimate fully connected layer of the Inception v3 model. This vector can capture the visual feature information of the image. The FID value can be used to measure the similarity between the image generated by the generative model and the real image. The smaller the FID value, the closer the distribution of the two images in the feature space is, and the higher the similarity. When two images are identical, their FID value is 0.

Insert image description here

$FID=∣∣μ_r-μ_g∣∣^2+Tr (Σ_r+Σ_g-2〖(Σ_r Σ_g)〗^(1/2))$
In the formula, is the feature mean of the real picture, is the feature mean of the generated picture, is the covariance matrix of the real picture, is the covariance matrix of the generated picture, is the trace.

import torch
import torchvision
import torchvision.transforms as transforms
from pytorch_fid import fid_score

# 准备真实数据分布和生成模型的图像数据
real_images_folder = '/path/to/real/images/folder'
generated_images_folder = '/path/to/generated/images/folder'

# 加载预训练的Inception-v3模型
inception_model = torchvision.models.inception_v3(pretrained=True)

# 定义图像变换
transform = transforms.Compose([
    transforms.Resize(299),
    transforms.CenterCrop(299),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
])

# 计算FID距离值
fid_value = fid_score.calculate_fid_given_paths([real_images_folder, generated_images_folder],
                                                 inception_model,
                                                 transform=transform)
print('FID value:', fid_value)

Different versions of pytorch_fid have different usage methods, so you need to pay attention to it.
The number of pictures in the two folders needs to be the same, the sizes should be the same as much as possible, and the names should preferably correspond, so that the corresponding pictures can be calculated.

last use

    PSNR = []
    NMSE = []
    SSIM = []

    for i, (img, gt) in enumerate(data_loader):
        batch_size = img.size()[0]
        img = img.to(device, dtype=torch.float)
        gt = gt.to(device, dtype=torch.float)

        with torch.no_grad():
            pred = model(img)

        for j in range(batch_size):
            a = pred[j]
            # save_image([pred[j]], images_save + str(i * batch_size + j + 1) + '.png', normalize=False)
            print(images_save + str(i * batch_size + j + 1) + '.png')

        pred, gt = pred.cpu().detach().numpy().squeeze(), gt.cpu().detach().numpy().squeeze()

        for j in range(batch_size):
            PSNR.append(psnr(pred[j], gt[j]))
            NMSE.append(nmse(pred[j], gt[j]))
            SSIM.append(ssim(pred[j], gt[j]))

    PSNR = np.asarray(PSNR)
    NMSE = np.asarray(NMSE)
    SSIM = np.asarray(SSIM)

    PSNR = PSNR.reshape(-1, slice_num)
    NMSE = NMSE.reshape(-1, slice_num)
    SSIM = SSIM.reshape(-1, slice_num)

    PSNR = np.mean(PSNR, axis=1)
    NMSE = np.mean(NMSE, axis=1)
    SSIM = np.mean(SSIM, axis=1) 

    print("PSNR mean:", np.mean(PSNR), "PSNR std:", np.std(PSNR))
    print("NMSE mean:", np.mean(NMSE), "NMSE std:", np.std(NMSE))
    print("SSIM mean:", np.mean(SSIM), "SSIM std:", np.std(SSIM))

By averaging each slice, a more detailed evaluation can be obtained and differences between different slices can be observed. If you average the entire data directly, you may not be able to obtain slice-level information. Therefore, it is a common practice to take slices and calculate the average of the metric on each slice separately.