[Image Enhancement] Common image enhancement algorithms

Collected and sorted by major platforms

Project address: Image-Enhancement

1. Histogram equalization

① Algorithm principle
Histogram equalization can generally be used for contrast enhancement of grayscale images (such as enhancement of shadow parts of faces);

② Advantages and disadvantages of the algorithm
If you directly equalize the three channels of R, G, and B of the color image and then merge them, it is very easy to have problems such as color unevenness and distortion. Therefore, the RGB image is generally converted to the YCrCb space, and the Y channel is Equalization (Y channel represents luminance component)

2. Gray world algorithm

① Algorithm principle
The gray world algorithm is based on the gray world hypothesis, which holds that: for an image with a large number of color changes, the average value of the three components of R, G, and B tends to the same gray value Gray. In a physical sense, the gray world method assumes that the mean value of the average reflection of light by natural scenes is a constant value on the whole, and this constant value is approximately "gray". The color balance algorithm enforces this assumption to the image to be processed, which can remove the influence of ambient light from the image and obtain the original scene image.

There are generally two ways to determine the Gray value

1) Use a fixed value, for an 8-bit image (0~255), usually take 128 as the gray value

2) Calculate the gain coefficient, calculate the average values ​​of avgR, avgG, and avgB of the three channels respectively, then:
$$\begin{gathered} Avg=(byR\_+byG\_+avgB)/3 \\ kr=Avg/avgR \\ kg=Avg/avgG \\ kb=Avg/avgB \end{gathered}$$

Using the calculated gain coefficient, recalculate each pixel value to form a new picture

② Advantages and disadvantages of the algorithm
This algorithm is simple and fast, but when the color of the image scene is not rich, especially when there are large monochrome objects, the algorithm often fails.
The effect of vegetable.png is obvious, but the effect of sky.png is not obvious

3. Retinex Algorithm

① Algorithm principle
The retina-cerebral cortex (Retinex) theory believes that the world is colorless, and the world seen by the human eye is the result of the interaction between light and matter, that is, the image mapped to the human eye and the long-wave (R) and medium wave (G), short wave (B) and the reflective properties of objects

$$I(x,y)=R(x,y)L(x,y)$$
where I is the image seen by the human eye, R is the reflection component of the object, L is the ambient light illumination component, and (x, y) is the corresponding position of the two-dimensional image

It calculates R by estimating L. Specifically, L can be obtained by convolution of Gaussian blur and I, expressed as:
$$\begin{gathered} log(R)=log(I)-log(L) \\ L=F\ast I \end{gathered}$$
where$F$ is a Gaussian blur filter,$\ast$ Determine the functional function
$$F=\frac{1}{\sqrt{2p.m}p}exp(\frac{-r^2}{p^2})$$
where$\sigma$ is called the Gaussian Surround Space Constant (Gaussian Surround Space Constant), which is the so-called scale in the algorithm, which has a relatively large impact on image processing. For two-dimensional images, r 2 r^$r^2$ is equal to the corresponding position:$x^2+y^2$ , that is, it is generally believed that the illumination component is the result of the original image after Gaussian filtering

② Algorithm advantages and disadvantages
Retinex algorithm, from SSR (Single-scale Retinex) to MSR (Multi-scale Retinex) and to the most commonly used MSRCR (Multi-scale Retinex with color restoration); the main purpose of color restoration is to adjust the contrast due to the local area of ​​​​the image If it is a grayscale
image, it only needs to be calculated once. If it is a color image, such as RGB three-channel, each channel needs to be calculated as above. First look at
a set of formulas:

$$\begin{gathered} RMSRCR(x,y{)}^{\prime }=G\cdot RMSRCR(x,y)+b \\ RMSRCR(x,y)=C(x,y)RMSR(x,y) \\ C(x,y)=f[I^{\prime }(x,y)]=f[\frac{I(x,y)}{\sum I(x,y)}]Ci(x,y)=f[Ii\prime (x,y)]=f[Ii(x,y)\sum j=1NIj(x,y)] \\ f[I^{\prime }(x,y)]=\beta log[\alpha I^{\prime }(x,y)]=\beta log[\alpha I^{\prime }(x,y)]-log[\sum I(x,y)] \end{gathered}$$

G表示增益Gain（一般取值：5）

b表示偏差Offset（一般取值：25）

I (x, y)表示某个通道的图像

C表示某个通道的彩色回复因子，用来调节3个通道颜色的比例；

f(·)表示颜色空间的映射函数；

β是增益常数（一般取值:46）；

α是受控制的非线性强度(一般取值：125)

MSRCR算法利用彩色恢复因子C，调节原始图像中3个颜色通道之间的比例关系，从而把相对较暗区域的信息凸显出来，达到了消除图像色彩失真的缺陷。 处理后的图像局部对比度提高，亮度与真实场景相似，在人们视觉感知下，图像显得更加逼真；但是MSRCR算法处理图像后，像素值一般会出现负值。所以从对数域r(x, y)转换为实数域R(x, y)后，需要通过改变增益Gain，偏差Offset对图像进行修正

四、自动白平衡(AWB)

① 算法原理
用一个简单的概念来解释什么是白平衡：假设，图像中R、G、B最高灰度值对应于图像中的白点，最低灰度值的对应于图像中最暗的点；其余像素点利用(ax+b)映射函数把彩色图像中R、G、B三个通道内的像素灰度值映射到[0.255]的范围内.

白平衡的本质是让白色的物体在任何颜色的光源下都显示为白色，这一点对人眼来说很容易办到，因为人眼有自适应的能力，只要光源的色彩不超出一定的限度，就可以自动还原白色。但相机就不同了，无论是图像传感器还是胶卷都会记录光源的颜色，白色的物体就会带上光源的颜色，白平衡所要做的就是把这个偏色去掉。

② 算法优缺点
自动白平衡是一个很复杂的问题，目前还没有一个万能的方法可以解决所有场景的白平衡问题

五、自动色彩均衡(ACE)

① 算法原理
ACE算法源自retinex算法，可以调整图像的对比度，实现人眼色彩恒常性和亮度恒常性，该算法考虑了图像中颜色和亮度的空间位置关系，进行局部特性的自适应滤波，实现具有局部和非线性特征的图像亮度与色彩调整和对比度调整，同时满足灰色世界理论假设和白色斑点假设。

第一步：对图像进行色彩／空域调整，完成图像的色差校正，得到空域重构图像；
$$R_c(p)=\sum _{j\in Subset,j\ne p}\frac{r(I_c(p)-I_c(j))}{d(p,j)}$$
式中，$Rc$ 是中间结果，$I_c(p)-I_c(j)$为两个不同点的亮度差，$d(p,j)$ 表示距离度量函数，$r(\ast )$为亮度表现函数，需是奇函数；这一步可以适应局部图像对比度，$r(\ast )$能够放大较小的差异，并丰富大的差异，根据局部内容扩展或者压缩动态范围。一般得，$r(\ast )$为：
$$r(x)=\{\begin{array}{ll}1& ,x<-T\\ x/T& ,-T⩽x⩽T\\ -1& ,x>T\end{array}$$
第二步：对校正后的图像进行动态扩展。ACE算法是对单一色道进行的，对于彩色图片需要对每一个色道分别处理
其中存在一种简单的线性扩展：

$R(x)=round[127.5+w\ast R_c(p)]$,其中，$w$表示线段$[(0,m_c),(255,M_c)]$的斜率，且有:
$M_c=min[R_c(p)]，M_c=max[R_c(p)]$

第三步：利用下面的公式将$R\left(x\right)$ expands to$[0,1]$ , the enhanced channel
$$L(x)=\frac{R(x)-my}{maxR-IR}$$
②Advantages and disadvantages of the algorithm
The enhancement effect of ACE is generally better than that of retinex. It should be noted that the current pixel in ACE is differentially compared with other pixels in the entire image, and the computational complexity is very, very high, which is the main reason for limiting its application.

Therefore, in the general algorithm, the specified number of samples is used to replace the pixel information of the entire image for differential calculation, reducing the amount of calculation and improving efficiency.

Summary: Looking at the renderings of various traditional algorithms, the ACE automatic color equalization algorithm has relatively good universality and effect. Of course, ACE cannot get good results for some pictures.

# 部分图像增强算法代码
"""
Project ：Image-Enhancement 
File    ：base_enhance.py
Author  ：MangoloD
Date    ：2022/3/9 13:48 
"""
# 图像增强算法，图像锐化算法
# 1）基于直方图均衡化 2）基于拉普拉斯算子 3）基于对数变换 4）基于伽马变换 5)CLAHE 6)retinex-SSR 7)retinex-MSR
# 其中，基于拉普拉斯算子的图像增强为利用空域卷积运算实现滤波
# 基于同一图像对比增强效果
# 直方图均衡化:对比度较低的图像适合使用直方图均衡化方法来增强图像细节
# 拉普拉斯算子可以增强局部的图像对比度
# log对数变换对于整体对比度偏低并且灰度值偏低的图像增强效果较好
# 伽马变换对于图像对比度偏低，并且整体亮度值偏高（对于相机过曝）情况下的图像增强效果明显

import cv2
import numpy as np
from PIL import Image
import matplotlib.pyplot as plt


# 直方图均衡增强


def hist(image):
    r, g, b = cv2.split(image)
    r1 = cv2.equalizeHist(r)
    g1 = cv2.equalizeHist(g)
    b1 = cv2.equalizeHist(b)
    image_equal_clo = cv2.merge([r1, g1, b1])
    return image_equal_clo


# 拉普拉斯算子
def laplacian(image):
    kernel = np.array([[0, -1, 0], [-1, 5, -1], [0, -1, 0]])
    image_lap = cv2.filter2D(image, cv2.CV_8UC3, kernel)
    return image_lap


# 对数变换
def log(image):
    image_log = np.uint8(np.log(np.array(image) + 1))
    cv2.normalize(image_log, image_log, 0, 255, cv2.NORM_MINMAX)
    # 转换成8bit图像显示
    cv2.convertScaleAbs(image_log, image_log)
    return image_log


# 伽马变换
def gamma(image):
    fgamma = 2
    image_gamma = np.uint8(np.power((np.array(image) / 255.0), fgamma) * 255.0)
    cv2.normalize(image_gamma, image_gamma, 0, 255, cv2.NORM_MINMAX)
    cv2.convertScaleAbs(image_gamma, image_gamma)
    return image_gamma


# 限制对比度自适应直方图均衡化CLAHE
def clahe(image):
    b, g, r = cv2.split(image)
    clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
    b = clahe.apply(b)
    g = clahe.apply(g)
    r = clahe.apply(r)
    image_clahe = cv2.merge([b, g, r])
    return image_clahe


def replaceZeroes(data):
    min_nonzero = min(data[np.nonzero(data)])
    data[data == 0] = min_nonzero
    return data


# retinex SSR
def SSR(src_img, size):
    L_blur = cv2.GaussianBlur(src_img, (size, size), 0)
    img = replaceZeroes(src_img)
    L_blur = replaceZeroes(L_blur)

    dst_Img = cv2.log(img / 255.0)
    dst_Lblur = cv2.log(L_blur / 255.0)
    dst_IxL = cv2.multiply(dst_Img, dst_Lblur)
    log_R = cv2.subtract(dst_Img, dst_IxL)

    dst_R = cv2.normalize(log_R, None, 0, 255, cv2.NORM_MINMAX)
    log_uint8 = cv2.convertScaleAbs(dst_R)
    return log_uint8


def SSR_image(image):
    size = 3
    b_gray, g_gray, r_gray = cv2.split(image)
    b_gray = SSR(b_gray, size)
    g_gray = SSR(g_gray, size)
    r_gray = SSR(r_gray, size)
    result = cv2.merge([b_gray, g_gray, r_gray])
    return result


# retinex MMR
def MSR(img, scales):
    weight = 1 / 3.0
    scales_size = len(scales)
    h, w = img.shape[:2]
    log_R = np.zeros((h, w), dtype=np.float32)

    for i in range(scales_size):
        img = replaceZeroes(img)
        L_blur = cv2.GaussianBlur(img, (scales[i], scales[i]), 0)
        L_blur = replaceZeroes(L_blur)
        dst_Img = cv2.log(img / 255.0)
        dst_Lblur = cv2.log(L_blur / 255.0)
        dst_Ixl = cv2.multiply(dst_Img, dst_Lblur)
        log_R += weight * cv2.subtract(dst_Img, dst_Ixl)

    dst_R = cv2.normalize(log_R, None, 0, 255, cv2.NORM_MINMAX)
    log_uint8 = cv2.convertScaleAbs(dst_R)
    return log_uint8


def MSR_image(image):
    scales = [15, 101, 301]  # [3,5,9]
    b_gray, g_gray, r_gray = cv2.split(image)
    b_gray = MSR(b_gray, scales)
    g_gray = MSR(g_gray, scales)
    r_gray = MSR(r_gray, scales)
    result = cv2.merge([b_gray, g_gray, r_gray])
    return result


def image_enhance(image, is_gamma=False):
    if is_gamma:
        image = image / 255.0  # 注意255.0得采用浮点数
        image = np.power(image, 0.5) * 255.0
        image = image.astype(np.uint8)

    # numpy实现
    out_min = 0
    out_max = 255

    in_min = np.min(image)
    in_max = np.max(image)

    a = float(out_max - out_min) / (in_max - in_min)
    b = out_min - a * in_min
    img_norm = image * a + b
    img_norm = img_norm.astype(np.uint8)
    return img_norm


def paint(image):
    plt.subplot(4, 2, 1)
    plt.imshow(image)
    plt.axis('off')
    plt.title('Offical')

    # 直方图均衡增强
    image_equal_clo = hist(image)

    plt.subplot(4, 2, 2)
    plt.imshow(image_equal_clo)
    plt.axis('off')
    plt.title('equal_enhance')

    # 拉普拉斯算法增强
    image_lap = laplacian(image)

    plt.subplot(4, 2, 3)
    plt.imshow(image_lap)
    plt.axis('off')
    plt.title('laplacian_enhance')

    # LoG对象算法增强
    image_log = log(image)

    plt.subplot(4, 2, 4)
    plt.imshow(image_log)
    plt.axis('off')
    plt.title('log_enhance')

    # 伽马变换
    image_gamma = gamma(image)

    plt.subplot(4, 2, 5)
    plt.imshow(image_gamma)
    plt.axis('off')
    plt.title('gamma_enhance')

    # CLAHE
    image_clahe = clahe(image)

    plt.subplot(4, 2, 6)
    plt.imshow(image_clahe)
    plt.axis('off')
    plt.title('CLAHE')

    # retinex_ssr
    image_ssr = SSR_image(image)

    plt.subplot(4, 2, 7)
    plt.imshow(image_ssr)
    plt.axis('off')
    plt.title('SSR')

    # retinex_msr
    image_msr = MSR_image(image)

    plt.subplot(4, 2, 8)
    plt.imshow(image_msr)
    plt.axis('off')
    plt.title('MSR')

    plt.show()


if __name__ == "__main__":
    image = cv2.imread("images/BGRT2.jpg")
    image_ = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

    flag = 0
    if flag == 0:
        paint(image_)
    elif flag == 1:
        image_msr = MSR_image(image)
        plt.subplot(1, 2, 1)
        plt.imshow(image_)
        plt.axis('off')
        plt.title('Offical')

        plt.subplot(1, 2, 2)
        plt.imshow(cv2.cvtColor(image_msr, cv2.COLOR_BGR2RGB))
        plt.axis('off')
        plt.title('MSR')

        plt.show()
    else:
        img = image_enhance(image_, is_gamma=False)
        plt.subplot(1, 2, 1)
        plt.imshow(image_)
        plt.axis('off')
        plt.title('Offical')

        plt.subplot(1, 2, 2)
        plt.imshow(img)
        plt.axis('off')
        plt.title('common')

        plt.show()