Face Mask Adding Based on FaceX-Zoo Realization of Face Mask Adding

Face series article directory

---

foreword

At present, both face and mask are mapped on the position of the face, which is very unreal. Some time ago, JD.com open sourced a PRnet-based 3D mask fitting human face. Adding a mask is very realistic, but it is only a single demo of adding a mask without a complete process. So I combined face detection and key points alignment. PRnet adds masks to realize a complete process of adding masks. It only needs one command, which is simple and convenient to call, and realizes adding masks to faces.
https://github.com/JDAI-CV/FaceX-Zoo
My complete code: https://github.com/zengwb-lx/Face_Mask_Add

1. The effect of adding a mask

As you can see from the picture below, the effect is still very realistic.

2. Implementation principle

From the figure above, we can see the a->b->c process:
1. Find the mask style that needs to be added, segment the mask, and get the mask texture
2. Generate the UC texture map through 3D reconstruction
as shown in the figure below. There are 8 mask styles. If you need other You can make it yourself:

From the above figure, we can see the d->e->f->g process:
1. Input the original image and 106 key points of the face, and 3D reconstruction to generate a UV texture map of the face.
2. Combining masks to fuse into a new UV texture map
3. Generate a new masked image through cv rendering

2. Code implementation

Mainly look at main.py, I will detect the face and align the key points. PRnet plus masks are written into three classes, which are easy and convenient to embed into your own project.
The code is as follows (example):

def main():
    show_result = False
    # image_path = './Data/test-data/test1.jpg'
    image_path = './1.png'
    image = cv2.imread(image_path, cv2.IMREAD_COLOR)
    face_detector = FaceDet()  # 人脸检测类
    face_align = FaceAlign()   # 人脸对齐类

    bboxs = face_detector(image)  # 获取人脸bbox
    # print(bboxs)
    image_show = image.copy()
    face_lms = []
    for box in bboxs:   
        # print(box)
        landmarks = face_align(image, box)   #  #获取人脸landmarks
        # print(landmarks, landmarks.shape)
        lms = np.reshape(landmarks.astype(np.int32), (-1))
        # print(lms, lms.shape)
        face_lms.append(lms)
        cv2.rectangle(image_show, (box[0], box[1]), (box[2], box[3]), (0, 0, 255), 2)
        for (x, y) in landmarks.astype(np.int32):
            cv2.circle(image_show, (x, y), 2, (255, 0, 0), -1)
        if show_result:
            cv2.imshow('lms', image_show)
            cv2.waitKey(0)

    # face masker
    is_aug = True
    mask_template_name = '0.png'
    mask_template_name2 = '1.png'
    masked_face_path = 'test1_mask1.jpg'
    face_masker = FaceMasker(is_aug)
    # ======masked one face========
    # 入参原始image，人脸关键点，返回加口罩后的image
    new_image = face_masker.add_mask_one(image, face_lms[0], mask_template_name, mask_template_name)
    # imsave(mask_template_name, new_image)
    plt.imshow(new_image)
    plt.show()