(8) Computer Vision-1 Image Augmentation

1. Image augmentation

Large-scale data sets are the prerequisite for the successful application of deep neural networks.

• Image augmentation technology: A series of random changes are made to training images to generate similar but different training samples.

There are two effects as follows:
(1) Expand the size of the training data set.
(2) Randomly changing the training samples can reduce the model's dependence on certain attributes, thereby improving the generalization ability of the model.
For example,
crop the image in different ways to make the object of interest appear in different positions, thereby reducing the model's dependence on the position of the object;
adjust factors such as brightness and color to reduce the model's sensitivity to color.

1.1 Function definition

import numpy as np
import tensorflow as tf
print(tf.__version__)

%matplotlib inline

2.3.0

Single image reading

import matplotlib.pyplot as plt
img = plt.imread("img/cat1.png")
plt.imshow(img)

Output:

<matplotlib.image.AxesImage at 0x6406f5ba8>

img.shape

Output:

(252, 312, 4)

Define drawing functionshow_images

def show_images(imgs, num_rows, num_cols, scale=2):
    figsize = (num_cols*scale, num_rows*scale)
    _, axes = plt.subplots(num_rows, num_cols, figsize=figsize)
    
    for i in range(num_rows):
        for j in range(num_cols):
            axes[i][j].imshow(imgs[i*num_cols+j])

            # 关闭子图中的轴
            axes[i][j].axes.get_xaxis().set_visible(False)
            axes[i][j].axes.get_yaxis().set_visible(False)
            
    return axes

Define auxiliary functionsapply

Most image augmentation methods have a certain degree of randomness.

To facilitate the observation of the effect of image augmentation, define an auxiliary function apply, imgrun the image augmentation method multiple times on the input image augand display all the results:

def apply(img, aug, num_rows=2, num_cols=4, scale=5):
    Y = [aug(img) for _ in range(num_rows*num_cols)]
    show_images(Y, num_rows, num_cols, scale)

1.2 Flip

1.2.1 Flip left and right

As the earliest and most widely used image augmentation method, flipping the image left and right usually does not change the category of the object.

tf.image.random_flip_left_rightFlip left and right images by realizing half the probability:

"""
image = np.array([[[1],[2]],[[3],[4]]])
tf.image.random_flip_left_right(image, seed=5).numpy().tolist()
输出：
	[[[1], [2]], [[3], [4]]]
"""

apply(img, tf.image.random_flip_left_right)

Output:

1.2.2 Flip up and down

Flip up and down is not as common as flipping left and right.

tf.image.random_flip_up_downFlip the image up and down by achieving half the probability:

apply(img, tf.image.random_flip_up_down)

Output:

1.3 Cropping

For the sample image above, the cat is in the middle of the image, but this is not the case in general.

Previous pooling layer a, the layer can be explained pooled to reduce the sensitivity of the convolution target layer position.
In addition, it is also possible to randomly crop the image to make objects appear in different positions of the image in different proportions, thereby also reducing the sensitivity of the model to the target position.

Suppose each time a piece of area is randomly cut out to the original area $10\%\sim .\; 100\%$ of the area, and the ratio of the width and height of the region from randomly$0.5\sim 2.$ Then scale the width and height of the area to 200 pixels. The code is implemented as follows:

aug = tf.image.random_crop
num_rows=2
num_cols=4
scale=5
crop_size=200

Y = [aug(img, (crop_size, crop_size, 3)) for _ in range(num_rows*num_cols)]
show_images(Y, num_rows, num_cols, scale)

Output:

Among them, if there is no special description, aa in this section$a$ andbbThe random number between$b$ refers to the interval$[a,b]$ Continuous value obtained by random uniform sampling.

1.4 Change color

Changing the color is also an augmentation method, which can change the color of an image from four aspects, including: brightness, contrast, saturation, and hue.

1.4.1 Change brightness

Randomly change the brightness of the image to 50% 50% of the original image brightness$50\%$$\sim .\; 1.\; 50\%$ , the following sample code:

aug = tf.image.random_brightness
num_rows=2
num_cols=4
scale=5
max_delta=0.5

Y = [aug(img, max_delta) for _ in range(num_rows*num_cols)]
show_images(Y, num_rows, num_cols, scale)

Output:

1.4.2 Change hue

To randomly change the color tone of the image, the code example is as follows:

img_new = img[:,:,:3]

aug = tf.image.random_hue
num_rows=2
num_cols=4
scale=5
max_delta=0.5

Y = [aug(img_new, max_delta) for _ in range(num_rows*num_cols)]
show_images(Y, num_rows, num_cols, scale)

Output:

reference

"Hands-on Deep Learning" (TF2.0 version)