[Image Processing] Image Enhancement Using Python

1. Description

        The depth and complexity of image enhancement techniques are often overlooked in a series of capture and sharing. From Fourier transforms to white balance and histogram processing, a variety of methods can transform ordinary photos into striking images. This blog post aims to unpack these techniques.

        I took a photo during the festival, during the nighttime festivities. Unfortunately, there is a destructive beam in the center of the image. We are optimistic about removing this light by applying a range of enhancement techniques to achieve a more balanced and appealing composition.

2. The magic of Fourier transform

        If images could speak a language, it would be mathematics, and the Fourier transform would be grammar. This mathematical tool converts an image from the spatial domain to the frequency domain. It decomposes an image into sine and cosine components, which represent how often the brightness of pixels in the image changes.

        Why is this important? Well, many image enhancement techniques, such as filtering and compression, perform better in the frequency domain. These techniques can remove noise, enhance edges, and even compress images for more efficient storage and transmission. Once the desired enhancements have been made, an inverse Fourier transform can be applied to transform the image back into the spatial domain, making it visible again.

        Unfortunately, the Fourier transform technique did not work for my particular image, although it did prove beneficial for a different image of the lunar orbit.

import numpy as np
import matplotlib.pyplot as plt
from skimage.io import imread, imshow
from skimage.color import rgb2gray

# Load and transform image
orbiter = rgb2gray(imread('lunar_orbiter.jpg'))
orbiter_fft = np.fft.fftshift(np.fft.fft2(orbiter))

orbiter_fft2 = orbiter_fft.copy()

# mask vertical component (scan lines)
orbiter_fft2[:280,orbiter_fft.shape[1]//2] = 1 
orbiter_fft2[-280:,orbiter_fft.shape[1]//2] = 1 

# Create 2 x 2 grid for plots
fig, axs = plt.subplots(2, 2, figsize=(12, 12))

# Original Image
axs[0, 0].imshow(orbiter, cmap='gray')
axs[0, 0].set_title('Original Image')
axs[0, 0].axis('off')

# Transformed Image (FFT)
axs[0, 1].imshow(np.log(abs(orbiter_fft)), cmap='gray')
axs[0, 1].set_title('Transformed Image (FFT)')
axs[0, 1].axis('off')

# FFT with Masked Vertical Component
axs[1, 0].imshow(np.log(abs(orbiter_fft2)), cmap='gray')
axs[1, 0].set_title('FFT with Masked Vertical Component')
axs[1, 0].axis('off')

# Inverse FFT of Masked Image
axs[1, 1].imshow(abs(np.fft.ifft2(orbiter_fft2)), cmap='gray')
axs[1, 1].set_title('Inverse FFT of Masked Image')
axs[1, 1].axis('off')

plt.tight_layout()
plt.show()

3. White Balance: Enhance True Color

        White balance is an influential technique that can dramatically change the aesthetic of an image. The essence of the process is to adjust the colors in the photo so that white items in reality are also depicted as white in the picture. This is a crucial step because the color of light sources can vary widely, affecting how colors are perceived in a photo.

        White balance brings greater realism to photos by ensuring that the colors in an image are accurate and consistent, regardless of the lighting conditions in which the photo was taken. Most modern cameras are equipped with an automatic white balance setting, but a greater understanding of the process allows photographers to make adjustments manually for great results. This can significantly improve the aesthetic quality of a photo, aligning it more closely with the scene as seen by the naked eye.

        However, not all images benefit from the use of white balance. In some cases, like the particular image I have, these techniques may not result in a noticeable enhancement. To illustrate the power of white balance, I'll share another example that dramatically improves image quality, demonstrating its potential effectiveness.

import matplotlib.pyplot as plt
import numpy as np
from skimage import util

def plot_channel_histogram(ax, image, percentile):
    for channel, color in enumerate('rgb'):
        channel_values = image[:, :, channel]
        ax.step(np.arange(256),
                np.bincount(channel_values.flatten(),
                 minlength=256) / channel_values.size,
                c=color, label=color)
        ax.axvline(np.percentile(channel_values, percentile),
                ls='--', c=color)

    ax.set_xlim(0, 255)
    ax.set_title(f'Color Channel with {percentile}th Percentile')
    ax.set_xlabel('Channel Value')
    ax.set_ylabel('Fraction of Pixels')
    ax.legend()

def apply_white_patch(image, percentile):
    # Convert and normalize the image
    wp = util.img_as_ubyte((image * 1.0 / np.percentile(image,
              percentile, axis=(0, 1))).clip(0, 1))
    
    # Create subplots
    fig, axs = plt.subplots(1, 3, figsize=(15, 5))
    
    axs[0].imshow(image)
    axs[0].set_title('Original Image')
    
    axs[1].imshow(wp)
    axs[1].set_title(f'Image with {percentile}th Percentile Normalization')
    
    plot_channel_histogram(axs[2], image, percentile)
    
    plt.show()

def main():
    percentiles = [99, 90]
    for percentile in percentiles:
        apply_white_patch(image, percentile)

4. Histogram operation: the art of balancing contrast and brightness

        A histogram serves as a graphical illustration of the distribution of tones in an image. They provide insights into the number of pixels at each brightness level in a picture, helping photographers understand the exposure and contrast of a photo.

        By utilizing histogram processing techniques, photographers can greatly enhance images that initially exhibit poor contrast. These methods allowed them to fine-tune brightness and contrast, improving the overall quality of the photo.

        However, it is important to keep in mind that the effectiveness of histogram operations may vary from image to image. In my particular case, even though I employed histogram manipulation, this technique did not produce the desired enhancement for my images. Nonetheless, to demonstrate the potential of histogram manipulation, I will give an example where this approach is very successful in improving image quality.

from skimage import color, exposure
import numpy as np
import matplotlib.pyplot as plt


# Original image
image = dark_image.copy()

# Convert the image to grayscale
gray_image = color.rgb2gray(image)

# Compute the histogram and cumulative distribution function
# (CDF) of the gray image
hist, bin_centers = exposure.histogram(gray_image)
cdf = np.cumsum(hist) / np.sum(hist)

# Split the color image into separate RGB channels
red_channel = image[:, :, 0]
green_channel = image[:, :, 1]
blue_channel = image[:, :, 2]

# Compute color multipliers from CDF for each channel
multipliers = 0.5 + 0.5 * np.concatenate([cdf]*3)
red_mult = multipliers[0]
green_mult = multipliers[1]
blue_mult = multipliers[2]

# Apply global histogram equalization to each channel
red_eq = exposure.equalize_hist(red_channel)
green_eq = exposure.equalize_hist(green_channel)
blue_eq = exposure.equalize_hist(blue_channel)

# Adjust channels by corresponding multipliers
red_final = red_eq * red_mult
green_final = green_eq * green_mult
blue_final = blue_eq * blue_mult

# Recombine the channels into a color image
adjusted_image = np.stack((red_final, green_final, blue_final), axis=-1)

# Create a 4x2 subplot for original and adjusted channels and images
fig, axs = plt.subplots(4, 2, figsize=(10, 20))

channels = [
    ('Red', red_channel, red_eq), 
    ('Green', green_channel, green_eq),
    ('Blue', blue_channel, blue_eq)
]

for i, (color_name, original, adjusted) in enumerate(channels):
    axs[i, 0].set_title(f'Original {color_name} Channel')
    axs[i, 0].imshow(original, cmap=f'{color_name}s')
    axs[i, 1].set_title(f'Adjusted {color_name} Channel')
    axs[i, 1].imshow(adjusted, cmap=f'{color_name}s')

# Plot the original and final combined images
axs[3, 0].set_title('Original Image')
axs[3, 0].imshow(image)
axs[3, 1].set_title('Adjusted Image')
axs[3, 1].imshow(adjusted_image)

# Show the subplots
plt.show()

V. Conclusion

        Image enhancement techniques, including Fourier transforms, white balance, and histogram manipulation, are powerful tools that can take photography to new heights. Each of these techniques has its own unique benefits and applications, making image enhancement a fascinating but complex process. For example, a photo I took over the Christmas holidays had damaging light that none of these techniques alone could remove. This highlights the complexity of image enhancement and the need for custom approaches.

        Experiments with these techniques revealed their respective potential. While the Fourier transform and white balance didn't provide the expected results for my Christmas images, they played an important role in enhancing other photos. Also, the histogram manipulation didn't significantly improve my image, but worked perfectly for another image. This experience underscores the importance of understanding the unique capabilities of these methods and knowing when to apply each.