--
3. FIG face alignment. Left: the face is detected and the flag convex hull. In: Delaunay triangulation of the convex hull points. Right: triangles face aligned by the affine distortions.
1 face alignment
1.1 the face detecting Landmarks
Two very different facial geometry, so we need to source some face twisted so as to cover the target face, but we also want to make sure we do not distort and does not recognize.
First, the use of two images on dlib detecting facial marker . However, with " facial deformity" different , we do not use all the points should not face alignment.
We just need people shown in Fig point on the border to the outer face.
1.2 Find the convex hull of
computer vision and mathematical terms, a set of boundary points or shape called a "package." No concave boundaries called "convex hull" .
In FIG. 3, left panel shows the use of the detected face dlib landmark red, convex hull is shown in blue.
A set of points in the convex hull may be used OpenCV concealHull function calculated .
python：
# points is numpy array of points obtained # using dlib. hullIndex = cv2.convexHull(points, returnPoints = False) # hullIndex is a vector of indices of points # that form the convex hull.
c++:
vector<int> hullIndex; // points is of type vector<Point2f> obtained // using dlib. convexHull(points, hullIndex, false, false); // hullIndex is a vector of indices of points // that form the convex hull.
1.3 Delaunay triangulation
aligned next step is a point on the convex hull Delaunay triangulation . Shows triangulation intermediate image of FIG. 3.
This allows us to face can be divided into smaller portions. My previous article detailed explanation of the Delaunay triangulation , see here
1.4 twist triangle affine
For face flush the final step, to account for the corresponding triangular face between the source and the target face, and face the source triangle affine distortions to the target face.
About my face deformed more information, please refer to my article .
However, as you can see in the right of FIG. 3, Align the top face and a slap in the face to another face does not look natural .
Since the light and the color difference between the two images, the joint can be seen. The next step shows how a seamless combination of the two images.
Seamless Cloning 2
Good technical idea is like magic. Magic combines outstanding physics, psychology, and good old hand skills to achieve incredible results.
Only the distorted image looks very bad. It with Seamless Cloning used in combination , the result is amazing! I wrote an article explaining the details here .
This is a feature of OpenCV 3, allows you to seamlessly part of the source image (the mask identifies) was cloned into the target image.
python:
output = cv2.seamlessClone(src, dst, mask, center, cv2.NORMAL_CLONE)
c++:
seamlessClone(src, dst, mask, center, output, NORMAL_CLONE);
The above usage of src image as shown in FIG. (Right). The DST image is an image on which we want to mix the source image (ie Trump image).
By using fillConvexPoly calculates a convex hull filled with white mask plate , comprising the center and the center of the bounding box of the mask.
to sum up:
Landmark detection face (Facial Landmark) point
Find convex hull (Convex Hull) plane
Delaunay triangulation (Delaunay Triangulation)
Affine twisted triangular (triangular face affine distortions source to target face) (Affine warp triangles)
Seamless cloning (Seamless Cloning)
--