every blog every motto: You can do more than you think.
0. Preface
Record related problems encountered in data enhancement using keras
1. Text
1.1 Basic introduction
We can use keras.preprocessing.image.ImageDataGenerator to enhance data "online" or "offline"
1.1.1 Generate Object
As shown in the following code:
we can generate an iterable object and specify the specific method of data enhancement (such as: rotation, flip, etc.)
from keras.preprocessing.image import ImageDataGenerator
datagen = ImageDataGenerator(
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
parameter:
- rotation_range is a number of degrees from 0 to 180, which is used to specify the angle of randomly selected pictures.
- Width_shift and height_shift are used to specify the degree of random movement in the horizontal and vertical directions. This is the ratio between the two 0~1.
- The rescale value will be multiplied to the entire image before performing other processing. Our image is an integer of 0 to 255 in the RGB channels . Such an operation may make the value of the image too high or too low, so we set this value to 0 1 The number between.
- shear_range is the degree of shear transformation, refer to shear transformation. zoom_range is used for random zooming.
- horizontal_flip randomly flips the image horizontally. This parameter is suitable for when the horizontal flip does not affect the semantics of the image.
- fill_mode is used to specify how to fill new pixels when pixel filling is required, such as rotation, horizontal and vertical displacement.
The following are two methods of the generator, which can be accessed by next, or traversed by a for loop. Next, take next as an example.
1.1.2 flow
Transform the array after reading the picture, that is, the parameter is an array.
Because it is a generator, we can use next to get the next value of the generator.
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
import tensorflow as tf
from keras.preprocessing.image import ImageDataGenerator
import cv2 as cv
import numpy as np
# 读取图片
img = cv.imread('./input/u=1819216937,2118754409&fm=26&gp=0.jpg')
img = np.expand_dims(img, axis=0)
# 实例化对象
agu = ImageDataGenerator(rotation_range=40) # 随机旋转0-40度之间
# print(type(agu))
# 变换并保存
next(agu.flow(img, save_to_dir='./output', save_format='jpg'))
Explanation: What flow accepts is a 4-dimensional array, which contains btach dimensions, such as: (batch, Height, Width, channel). One dimension is added to the above code. The same below.
1.1.3 flow_from_directory
"Transform the folder where the picture is located", that is, the parameter is a folder.
Note: The parameter is a folder, which must contain subfolders! ! !
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
import tensorflow as tf
from keras.preprocessing.image import ImageDataGenerator
import cv2 as cv
import numpy as np
# 输入文件夹
path = r'./input'
# 实例化对象
agu = ImageDataGenerator(rotation_range=40) # 随机旋转0-40度之间
# print(type(agu))
# 变换并保存
next(agu.flow_from_directory(path,batch_size=1, save_to_dir='./output', save_format='jpg'))
1.2 The problem
The basic code is the same as 1.1.2, except that the instantiated object is different, so only the instantiated object part is posted
1.2.1 Question 1
The error is as follows:
UserWarning: This ImageDataGenerator specifies `zca_whitening`, which overrides setting of `featurewise_center`.
warnings.warn('This ImageDataGenerator specifies '
Part of the code:
# 实例化对象
agu = ImageDataGenerator(rotation_range=40,zca_whitening=True)
Only the zca_whitening parameter was added, and the above warning appeared.
Explanation: When zca_whitening=True, feature_center will be set to True, even though we have not set it, all warnings will appear.
Source code:
solution, just add feature_center=True.
# 实例化对象
agu = ImageDataGenerator(rotation_range=40,zca_whitening=True,featurewise_center=True)
1.2.2 Question 2
Related problems encountered in image whitening:
UserWarning: This ImageDataGenerator specifies `featurewise_center`, but it hasn't been fit on any training data. Fit it first by calling `.fit(numpy_data)`.
warnings.warn('This ImageDataGenerator specifies '
Instantiate object code
# 实例化对象
agu = ImageDataGenerator(rotation_range=40,zca_whitening=True,featurewise_center=True)
Reason: We used feature_center=True in the parameter, but did not use the fit method.
Official explanation:
Solution: call the fit method code before using the flow or flow_from_directory method
:
Also: when the fit method is used, statistical information (mean variance) will be calculated, This process is very slow. If the picture is large, the following error will occur
1.2.3 Question 3
The error is as follows:
MemoryError: Unable to allocate 857. GiB for an array with shape (479700, 479700) and data type float32
Reason: To calculate relevant statistical information, SVD will be calculated. The value in it is relatively large, so the above error occurred.
One attempt:
replace the fit method with the following and calculate it manually:
agu.mean = np.mean(img,axis=0)
Complete code:
but the result is not necessarily correct:
this is just an attempt, the result is not necessarily correct
1.2.4 Question 4
Only the rotation parameter is added, and the color of the saved picture changes.
Code:
Reason: We use cv to read the picture, and the read is BRG. The writing is exactly the opposite of the original RGB, so an error occurs.
Method 1: Adjust the BGR sequence
Result:
Method two: Conversion
method three: PIL reading
1.2.5 Question 5
Note: To be precise, this is not a problem, but curious, as follows: When
setting multiple parameters, only use one at a time, or use them at the same time.
Test, use the two parameters of rotation and vertical flip at the same time.
Code:
Tested twice, and the results are as follows:
Conclusion: We found that the two parameters work randomly, sometimes only rotating, not flipping; sometimes both work, etc.
Has conducted a test with the following results:
we found, respectively towards the left and right have been rotated, which also overturned we said before, the note inside or comments: "The rotation towards 0-40 degrees (set Value) to rotate", but it should be between -40-40 degrees . In the source code, we found that this is indeed the case.
The comment in the above code is wrong, it is not rotated between 0-40 degrees, I won't go back and modify it, please pay attention! ! !
1.3 Two ways of use
1.3.1 Ordinary use
This is an official example, through the loop, continuously obtain data
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
y_train = np_utils.to_categorical(y_train, num_classes)
y_test = np_utils.to_categorical(y_test, num_classes)
datagen = ImageDataGenerator(
featurewise_center=True,
featurewise_std_normalization=True,
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
horizontal_flip=True)
# compute quantities required for featurewise normalization
# (std, mean, and principal components if ZCA whitening is applied)
datagen.fit(x_train)
# fits the model on batches with real-time data augmentation:
model.fit_generator(datagen.flow(x_train, y_train, batch_size=32),
steps_per_epoch=len(x_train), epochs=epochs)
# here's a more "manual" example
for e in range(epochs):
print 'Epoch', e
batches = 0
for x_batch, y_batch in datagen.flow(x_train, y_train, batch_size=32):
loss = model.train(x_batch, y_batch)
batches += 1
if batches >= len(x_train) / 32:
# we need to break the loop by hand because
# the generator loops indefinitely
break
1.3.2 Combine with keras.utils.Sequence
The use of keras.utils.Sequence refers to the previous article, which is attached.
Note:
It is also very simple to use,
- During initialization, a data enhancement object is generated
- __getitem __ Return next, just keep calling and generating.
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
import tensorflow as tf
from keras.preprocessing.image import ImageDataGenerator, load_img
import cv2 as cv
import numpy as np
class Date(tf.keras.utils.Sequence):
def __init__(self):
self.input = r'./data'
self.file = os.listdir(self.input)
print(self.file)
self.batch = 2
self.agu = ImageDataGenerator(
# 增强的项目。翻转，缩放，平移，旋转，颜色等。
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2, # 比例平移
zoom_range=0.2,
fill_mode='nearest',
horizontal_flip=True,
vertical_flip=True,
zca_whitening=True,
featurewise_center=True,
rescale=1. / 255
)
def __len__(self):
"""
此方法要实现，否则会报错
正常程序中返回1个epoch迭代的次数
:return:
"""
return len(self.file)
def generate_batch(self):
x_batch = np.zeros((3, 256, 256, 3))
for i, file in enumerate(self.file):
path = os.path.join(self.input, file)
# print(path)
img = cv.imread(path)
x_batch[i,] = img
x_batch = np.array(x_batch)
return x_batch
def __getitem__(self, index):
"""生成一个batch的数据"""
x_batch = self.generate_batch()
# self.agu.fit(x_batch)
return next(self.agu.flow(x_batch,batch_size=1,save_to_dir='./output',save_format='jpg'))
# return next((self.agu.flow_from_directory(self.input,save_to_dir='./output',save_format='jpg')))
# return x_batch
# 实例化数据
date = Date()
for batch_number, x in enumerate(date):
print('正在进行第{} batch'.format(batch_number))
references
[1] https://blog.csdn.net/weixin_39190382/article/details/109195031
[2] https://blog.csdn.net/xijuezhu8128/article/details/79895856
[3] https://blog.csdn.net/wang263334857/article/details/88749379
[4] https://blog.csdn.net/qq_36537768/article/details/103066394
[5] https://www.cnblogs.com/hutao722/p/10075150.html
[6] https://blog.csdn.net/jacke121/article/details/79245732
[7] https://blog.csdn.net/qq_27825451/article/details/90172030
[8] https://blog.csdn.net/dugudaibo/article/details/87719078
[9] https://tensorflow.google.cn/versions/r2.0/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator
[10] https://keras-cn.readthedocs.io/en/latest/preprocessing/image/#imagedatagenerator