图像增强 imgaug

imgaug确实是图像处理、增强的又一大利器，使用它确实能省不少事。

1.使用imgaug显示图片及bounding box

如要在目标检测任务中，可通过imgaug显示图片及其bbox。
以安全帽的数据集中的一张图片为例。

其对应的annotation xml文件内容为：

<annotation>
	<folder>hat01</folder>
	<filename>000000.jpg</filename>
	<path>D:\dataset\hat01\000000.jpg</path>
	<source>
		<database>Unknown</database>
	</source>
	<size>
		<width>947</width>
		<height>1421</height>
		<depth>3</depth>
	</size>
	<segmented>0</segmented>
	<object>
		<name>hat</name>
		<pose>Unspecified</pose>
		<truncated>0</truncated>
		<difficult>0</difficult>
		<bndbox>
			<xmin>60</xmin>
			<ymin>66</ymin>
			<xmax>910</xmax>
			<ymax>1108</ymax>
		</bndbox>
	</object>
</annotation>

imgaug代码如下：

from PIL import Image
import numpy as np
import imgaug as ia
from imgaug.augmentables.bbs import BoundingBox, BoundingBoxesOnImage
img_path=r"D:\1.jpg"

img = np.array(Image.open(img_path).convert('RGB'), dtype=np.uint8)
ia.imshow(img)

boxes=[[0,60,66,910,1108]]
bboxes = BoundingBoxesOnImage(
            [BoundingBox(*box[1:], label=box[0]) for box in boxes],
            shape=img.shape)

ia.imshow(bboxes.draw_on_image(img, size=2))

结果为：

使用时要注意BoundingBox中的关键字（默认为左上角、右下角的绝对坐标）：

imgaug.augmentables.bbs.BoundingBox(x1, y1, x2, y2, label=None)

2.来一波图像增广操作

from PIL import Image
import numpy as np
import imgaug as ia
from imgaug.augmentables.bbs import BoundingBox, BoundingBoxesOnImage
import imgaug.augmenters as iaa
img_path=r"D:\1.jpg"

img = np.array(Image.open(img_path).convert('RGB'), dtype=np.uint8)
# ia.imshow(img)

boxes=[[0,60,66,910,1108]]
bboxes = BoundingBoxesOnImage(
            [BoundingBox(*box[1:], label=box[0]) for box in boxes],
            shape=img.shape)

seq = iaa.Sequential([
            iaa.Dropout([0.0, 0.01]),
            iaa.Sharpen((0.0, 0.2)),
            # rotate by -45 to 45 degrees (affects segmaps)
            iaa.Affine(rotate=(-20, 20), translate_percent=(-0.2, 0.2)),
            iaa.AddToBrightness((-30, 30)),
            iaa.AddToHue((-20, 20)),
            iaa.Fliplr(0.5),
        ], random_order=True)

image_aug, bbs_aug = seq(image=img, bounding_boxes=bboxes)

bbs_aug=bbs_aug.remove_out_of_image().clip_out_of_image()

ia.imshow(bbs_aug.draw_on_image(image_aug, size=2))

由于这个sequential的各个变换都涉及随机值，因此每次输出的效果都是不一样的。

抽取几个结果显示如下：

好像变换后有的bbox贴合的也不是很好。

3.使用类及超类实现transform

此处参考了 eriklindernoren/PyTorch-YOLOv3的代码。这在项目中使用起来很方便，应该来说非常实用。

换张图片说明下效果：

from PIL import Image
import numpy as np
import imgaug as ia
from imgaug.augmentables.bbs import BoundingBox, BoundingBoxesOnImage
import imgaug.augmenters as iaa

class ImgAug(object):
    def __init__(self, augmentations=[]):
        self.augmentations = augmentations

    def __call__(self, data):
        # Unpack data
        # print(self.augmentations)
        img, boxes = data

        # Convert xywh to xyxy
        boxes = np.array(boxes)
        # boxes[:, 1:] = xywh2xyxy_np(boxes[:, 1:])

        # Convert bounding boxes to imgaug
        bounding_boxes = BoundingBoxesOnImage(
            [BoundingBox(*box[1:], label=box[0]) for box in boxes],
            shape=img.shape)

        # Apply augmentations
        img, bounding_boxes = self.augmentations(
            image=img,
            bounding_boxes=bounding_boxes)

        # Clip out of image boxes
        bounding_boxes = bounding_boxes.clip_out_of_image()

        # Convert bounding boxes back to numpy
        boxes = np.zeros((len(bounding_boxes), 5))
        for box_idx, box in enumerate(bounding_boxes):
            # Extract coordinates for unpadded + unscaled image
            x1 = box.x1
            y1 = box.y1
            x2 = box.x2
            y2 = box.y2

            # Returns (x, y, w, h)
            boxes[box_idx, 0] = box.label
            boxes[box_idx, 1] = x1
            boxes[box_idx, 2] = y1
            boxes[box_idx, 3] = x2
            boxes[box_idx, 4] = y2

        return img, boxes


class DefaultAug(ImgAug):
    def __init__(self, ):
        self.augmentations = iaa.Sequential([
            iaa.Dropout([0.0, 0.01]),
            iaa.Sharpen((0.0, 0.2)),
            # rotate by -45 to 45 degrees (affects segmaps)
            iaa.Affine(rotate=(-20, 20), translate_percent=(-0.2, 0.2)),
            iaa.AddToBrightness((-30, 30)),
            iaa.AddToHue((-20, 20)),
            iaa.Fliplr(0.5),
        ], random_order=True)


img_path = r"D:\1.jpg"

img = np.array(Image.open(img_path).convert('RGB'), dtype=np.uint8)

boxes = [[0, 30, 106, 109, 198], [0, 97, 69, 187, 172], [0, 193, 33, 296, 145]]

bboxes = BoundingBoxesOnImage(
    [BoundingBox(*box[1:], label=box[0]) for box in boxes],
    shape=img.shape)

ia.imshow(bboxes.draw_on_image(img, size=2))

data = ((img, boxes))
transform = DefaultAug()
img, boxes = transform(data)

bboxes = BoundingBoxesOnImage(
    [BoundingBox(*box[1:], label=box[0]) for box in boxes],
    shape=img.shape)

ia.imshow(bboxes.draw_on_image(img, size=2))

使用时注意看下box的坐标格式：xywh/xyxy或者是放缩过的xywh。

我这里使用的是它的绝对坐标。

参考文献

[1] https://nbviewer.jupyter.org/github/aleju/imgaug-doc/blob/master/notebooks/B02%20-%20Augment%20Bounding%20Boxes.ipynb
[2] https://github.com/eriklindernoren/PyTorch-YOLOv3/blob/1675d1898c2ec6e0bca6057b482667a2d884ceb8/pytorchyolo/utils/transforms.py#L56