【Record】Summary and implementation of data processing methods
background
As a key step in pre-processing, data enhancement plays an important role in the entire computer vision;
Data augmentation is often the key to determining the quality of datasets, and is mainly used for data augmentation. In deep learning-based tasks, the diversity and quantity of data can often determine the upper limit of the model;
This record is mainly the source code implementation of some methods in data enhancement;
Common Data Augmentation Methods
First of all, if you use the Pytorch framework, its internal torchvision has already packaged many methods of data enhancement;
from torchvision import transforms
data_aug = transforms.Compose[
transforms.Resize(size=240),
transforms.RandomHorizontalFlip(0.5),
transforms.ToTensor()
]
Next, implement some main methods yourself;
Common data enhancement methods are: Compose, RandomHflip, RandomVflip, Reszie, RandomCrop, Normalize, Rotate, RandomRotate
1、Compose
Function: sort and integrate multiple methods, and call them sequentially;
# 排序(compose)
class Compose(object):
def __init__(self, transforms):
self.transforms = transforms
def __call__(self, img):
for t in self.transforms:
img = t(img) # 通过循环不断调用列表中的方法
return img
2、RandomHflip
Function: random horizontal flip;
# 随机水平翻转(random h flip)
class RandomHflip(object):
def __call__(self, image):
if random.randint(2):
return cv2.flip(image, 1)
else:
return image
Through the random number 0 or 1, the image may be reversed or not reversed;
3、RandomVflip
Function: random vertical flip
class RandomVflip(object):
def __call__(self, image):
if random.randint(2):
return cv2.flip(image, 0)
else:
return image
4、RandomCrop
Function: random cropping;
# 缩放(scale)
def scale_down(src_size, size):
w, h = size
sw, sh = src_size
if sh < h:
w, h = float(w * sh) / h, sh
if sw < w:
w, h = sw, float(h * sw) / w
return int(w), int(h)
# 固定裁剪(fixed crop)
def fixed_crop(src, x0, y0, w, h, size=None):
out = src[y0:y0 + h, x0:x0 + w]
if size is not None and (w, h) != size:
out = cv2.resize(out, (size[0], size[1]), interpolation=cv2.INTER_CUBIC)
return out
# 随机裁剪(random crop)
class RandomCrop(object):
def __init__(self, size):
self.size = size
def __call__(self, image):
h, w, _ = image.shape
new_w, new_h = scale_down((w, h), self.size)
if w == new_w:
x0 = 0
else:
x0 = random.randint(0, w - new_w)
if h == new_h:
y0 = 0
else:
y0 = random.randint(0, h - new_h)
out = fixed_crop(image, x0, y0, new_w, new_h, self.size)
return out
5、Normalize
Function: Regularize the image data, that is, subtract the mean and divide the variance;
# 正则化(normalize)
class Normalize(object):
def __init__(self,mean, std):
'''
:param mean: RGB order
:param std: RGB order
'''
self.mean = np.array(mean).reshape(3,1,1)
self.std = np.array(std).reshape(3,1,1)
def __call__(self, image):
'''
:param image: (H,W,3) RGB
:return:
'''
return (image.transpose((2, 0, 1)) / 255. - self.mean) / self.std
6、Rotate
Function: rotate the image;
# 旋转(rotate)
def rotate_nobound(image, angle, center=None, scale=1.):
(h, w) = image.shape[:2]
# if the center is None, initialize it as the center of the image
if center is None:
center = (w // 2, h // 2) # perform the rotation
M = cv2.getRotationMatrix2D(center, angle, scale) # 这里是实现得到旋转矩阵
rotated = cv2.warpAffine(image, M, (w, h)) # 通过矩阵进行仿射变换
return rotated
7、RandomRotate
Function: random rotation, widely used in image enhancement;
# 随机旋转(random rotate)
class FixRandomRotate(object):
# 这里的随机旋转是指在0、90、180、270四个角度下的
def __init__(self, angles=[0,90,180,270], bound=False):
self.angles = angles
self.bound = bound
def __call__(self,img):
do_rotate = random.randint(0, 4)
angle=self.angles[do_rotate]
if self.bound:
img = rotate_bound(img, angle)
else:
img = rotate_nobound(img, angle)
return img
8、Resize
Role: to achieve scaling;
# 大小重置(resize)
class Resize(object):
def __init__(self, size, inter=cv2.INTER_CUBIC):
self.size = size
self.inter = inter
def __call__(self, image):
return cv2.resize(image, (self.size[0], self.size[0]), interpolation=self.inter)
Other Data Augmentation Methods
Some other data enhancement methods are mostly special clipping;
1. Center cropping
# 中心裁剪(center crop)
def center_crop(src, size):
h, w = src.shape[0:2]
new_w, new_h = scale_down((w, h), size)
x0 = int((w - new_w) / 2)
y0 = int((h - new_h) / 2)
out = fixed_crop(src, x0, y0, new_w, new_h, size)
return out
2. Random brightness enhancement
# 随机亮度增强(random brightness)
class RandomBrightness(object):
def __init__(self, delta=10):
assert delta >= 0
assert delta <= 255
self.delta = delta
def __call__(self, image):
if random.randint(2):
delta = random.uniform(-self.delta, self.delta)
image = (image + delta).clip(0.0, 255.0)
# print('RandomBrightness,delta ',delta)
return image
3. Random contrast enhancement
# 随机对比度增强(random contrast)
class RandomContrast(object):
def __init__(self, lower=0.9, upper=1.05):
self.lower = lower
self.upper = upper
assert self.upper >= self.lower, "contrast upper must be >= lower."
assert self.lower >= 0, "contrast lower must be non-negative."
# expects float image
def __call__(self, image):
if random.randint(2):
alpha = random.uniform(self.lower, self.upper)
# print('contrast:', alpha)
image = (image * alpha).clip(0.0,255.0)
return image
4. Random saturation enhancement
# 随机饱和度增强(random saturation)
class RandomSaturation(object):
def __init__(self, lower=0.8, upper=1.2):
self.lower = lower
self.upper = upper
assert self.upper >= self.lower, "contrast upper must be >= lower."
assert self.lower >= 0, "contrast lower must be non-negative."
def __call__(self, image):
if random.randint(2):
alpha = random.uniform(self.lower, self.upper)
image[:, :, 1] *= alpha
# print('RandomSaturation,alpha',alpha)
return image
4. Boundary Expansion
# 边界扩充(expand border)
class ExpandBorder(object):
def __init__(self, mode='constant', value=255, size=(336,336), resize=False):
self.mode = mode
self.value = value
self.resize = resize
self.size = size
def __call__(self, image):
h, w, _ = image.shape
if h > w:
pad1 = (h-w)//2
pad2 = h - w - pad1
if self.mode == 'constant':
image = np.pad(image, ((0, 0), (pad1, pad2), (0, 0)),
self.mode, constant_values=self.value)
else:
image = np.pad(image,((0,0), (pad1, pad2),(0,0)), self.mode)
elif h < w:
pad1 = (w-h)//2
pad2 = w-h - pad1
if self.mode == 'constant':
image = np.pad(image, ((pad1, pad2),(0, 0), (0, 0)),
self.mode,constant_values=self.value)
else:
image = np.pad(image, ((pad1, pad2), (0, 0), (0, 0)),self.mode)
if self.resize:
image = cv2.resize(image, (self.size[0], self.size[0]),interpolation=cv2.INTER_LINEAR)
return image
Of course, there are many other ways of data enhancement, so I won’t continue to explain them here;
expand
In addition to using the data enhancement package that comes with Pytorch, you can also use the imgaug package (a package based on data processing, including a large number of data processing methods, and the code is completely open source)
Code address: https://github.com/aleju/imgaug
Documentation: https://imgaug.readthedocs.io/en/latest/index.html
It is strongly recommended that you take a look at this documentation. Many data processing methods in it can be quickly applied to actual projects, and can also deepen your understanding of image processing;