Figure 1: Example of seamless cloning: an airplane is cloned into the evening sky.
https://www.learnopencv.com/seamless-cloning-using-opencv-python-cpp/
One of the exciting new features introduced in OpenCV 3 is called seamless cloning. With this new feature, you can copy objects from one image and paste them into another image, making the composition look seamless and natural. The image above was created using sky and airplane scenes. If I simply overlay the airplane image on the sky image, the result looks ridiculous (see Figure 2).
Figure 2: The sky overlaps the plane.
Of course, no one now has such a composition in their heads. You will obviously cover up the image carefully, maybe after spending a long time in Photoshop, the resulting image is shown in Figure 3.
Figure 3: The plane obscures the carefully masked sky image.
If you are an artist, you need to spend half a day, and then carefully adjust the lighting on the plane to suit the lighting of the sky image and create a beautiful composition.
However, there are two problems. First of all, you don't have half a day. Secondly, you may not be an artist!
Wouldn’t it be cool if you could make a very rough mask around the plane and create a beautiful composition as shown in Figure 1? What if only 10 lines of code are used? Now, this is not only cool, it will also be a serious badass!
Before showing you the code, let me spend an hour explaining the exciting theory behind seamless cloning. Oh wait, I have a better idea. Let's dive into the code first.
Seamless cloning example
Quickly understand usage
Python
0 |
output = cv2.seamlessClone(src, dst, mask, center, flags) |
C ++
0 |
seamlessClone(Mat src, Mat dst, Mat mask, Point center, Mat output, int flags) |
| src |
The source image that will be copied to the destination image. In our example it is an airplane. |
| dst |
The target image to which the source image will be cloned. In our example, it is the sky image. |
| mask |
A rough mask around the object to be cloned. This should be the size of the source image. If you are lazy, set it to an all-white image! |
| central |
The position of the center of the source image in the target image. |
| Sign |
The two flags currently working are NORMAL_CLONE and MIXED_CLONE. I provide an example to illustrate the difference. |
| Output |
Output/result image. |
Now, let's take a look at the code I used to generate the image above.
Python example
6 |
src = cv2.imread("images/airplane.jpg") |
7 |
dst = cv2.imread("images/sky.jpg") |
10 |
# Create a rough mask around the airplane. |
11 |
src_mask = np.zeros(src.shape, src.dtype) |
12 |
poly = np.array([ [4,80], [30,54], [151,63], [254,37], [298,90], [272,134], [43,122] ], np.int32) |
13 |
cv2.fillPoly(src_mask, [poly], (255, 255, 255)) |
15 |
# This is where the CENTER of the airplane will be placed |
19 |
output = cv2.seamlessClone(src, dst, src_mask, center, cv2.NORMAL_CLONE) |
22 |
cv2.imwrite("images/opencv-seamless-cloning-example.jpg", output); |
C ++示例
3 |
// Read images : src image will be cloned into dst |
4 |
Mat src = imread("images/airplane.jpg"); |
5 |
Mat dst = imread("images/sky.jpg"); |
8 |
// Create a rough mask around the airplane. |
9 |
Mat src_mask = Mat::zeros(src.rows, src.cols, src.depth()); |
11 |
// Define the mask as a closed polygon |
13 |
poly[0][0] = Point(4, 80); |
14 |
poly[0][1] = Point(30, 54); |
15 |
poly[0][2] = Point(151,63); |
16 |
poly[0][3] = Point(254,37); |
17 |
poly[0][4] = Point(298,90); |
18 |
poly[0][5] = Point(272,134); |
19 |
poly[0][6] = Point(43,122); |
21 |
const Point* polygons[1] = { poly[0] }; |
22 |
int num_points[] = { 7 }; |
24 |
// Create mask by filling the polygon |
26 |
fillPoly(src_mask, polygons, num_points, 1, Scalar(255,255,255)); |
28 |
// The location of the center of the src in the dst |
29 |
Point center(800,100); |
31 |
// Seamlessly clone src into dst and put the results in output |
33 |
seamlessClone(src, dst, src_mask, center, output, NORMAL_CLONE); |
36 |
imwrite("images/opencv-seamless-cloning-example.jpg", output); |
在上面的示例中，我使用的克隆类型（标志）为NORMAL_CLONE。还有另一种MIXED_CLONE，与NORMAL_CLONE稍有不同。让我们看看这两种类型在细节上有何不同。
普通克隆（NORMAL_CLONE）与混合克隆（MIXED_CLONE）
我有一个5岁的儿子，如果我对他好，他会给我“我爱你”门票。曾经有一段时间，孩子们渴望得到父母的批准，但如今，父母们必须竭尽全力获得“我爱你”的门票。无论如何，回到克隆。我将在Computer Vision服务中使用这些“我爱你的票”之一（请参见图4）。
图4：“我爱你”门票。
让我们尝试将该图像克隆到图5所示的木质纹理上。我们会很懒，使用全白色的源蒙版，然后将源图像克隆在木质纹理图像的中心。
图5：木材纹理
Python范例
4 |
# Read images : src image will be cloned into dst |
5 |
im = cv2.imread("images/wood-texture.jpg") |
6 |
obj= cv2.imread("images/iloveyouticket.jpg") |
8 |
# Create an all white mask |
9 |
mask = 255 * np.ones(obj.shape, obj.dtype) |
11 |
# The location of the center of the src in the dst |
12 |
width, height, channels = im.shape |
13 |
center = (height/2, width/2) |
15 |
# Seamlessly clone src into dst and put the results in output |
16 |
normal_clone = cv2.seamlessClone(obj, im, mask, center, cv2.NORMAL_CLONE) |
17 |
mixed_clone = cv2.seamlessClone(obj, im, mask, center, cv2.MIXED_CLONE) |
20 |
cv2.imwrite("images/opencv-normal-clone-example.jpg", normal_clone) |
21 |
cv2.imwrite("images/opencv-mixed-clone-example.jpg", mixed_clone) |
C ++示例
2 |
Mat src = imread("images/iloveyouticket.jpg"); |
3 |
Mat dst = imread("images/wood-texture.jpg"); |
5 |
// Create an all white mask |
6 |
Mat src_mask = 255 * Mat::ones(src.rows, src.cols, src.depth()); |
8 |
// The location of the center of the src in the dst |
9 |
Point center(dst.cols/2,dst.rows/2); |
11 |
// Seamlessly clone src into dst and put the results in output |
15 |
seamlessClone(src, dst, src_mask, center, normal_clone, NORMAL_CLONE); |
16 |
seamlessClone(src, dst, src_mask, center, mixed_clone, MIXED_CLONE); |
19 |
imwrite("images/opencv-normal-clone-example.jpg", normal_clone); |
20 |
imwrite("images/opencv-mixed-clone-example.jpg", mixed_clone); |
正常克隆结果
如果我们通过使用NORMAL_CLONE标志使用“正常克隆”，我们将得到如图6所示的结果。现在我们没有使用良好的蒙版，您会发现单词“ I”和“ Love”之间以及“ you”之间的过度平滑”和“ Paa”。当然，我们很懒。我们本可以创建一个粗糙的蒙版并改善结果。但是，如果您懒惰和聪明，则可以使用混合克隆。
图6：OpenCV普通克隆示例
混合克隆结果
在“正常克隆”中，源图像的纹理（渐变）保留在克隆的区域中。在混合克隆中，克隆区域的纹理（渐变）由源图像和目标图像的组合确定。混合克隆不会产生平滑区域，因为它会选择源图像和目标图像之间的主要纹理（渐变）。混合克隆的结果如图7所示。请注意，“ I”和“ Love”之间以及“ you”和“ Paa”之间的纹理不再平滑。懒人加油！
图7：OpenCV混合克隆示例
无缝克隆视频结果
我拍摄了飞机和天空的图像，并更改了飞机的位置以创建此动画。MIXED_CLONE提供了更好的结果，并且您几乎没有注意到任何工件。将300×194图像（飞机）克隆到1000×560图像（天空）需要大约0.4秒。
下载无缝克隆代码和示例图像
Scroll down to the download section (bottom of this article) to immediately access the C++ and Python code and images in this article.
Poisson image editing
A seamless clone of OpenCV titled the realization of an influential SIGGRAPH 2003 paper "Poisson Image Editing" by Patrick Perez, Michelle Gangnet and Andrew Blake.
Now we know that if the intensity (RGB value) of the source image (airplane) is blended with the target image (sky) using a carefully created mask, we will get the result shown in Figure 3. Using image gradients instead of image intensity can produce more realistic results. After seamless cloning, the intensity of the resulting image in the mask area is different from the intensity of the source area in the mask area. In contrast, the gradient of the resulting image in the masked area is approximately the same as the gradient of the source area in the masked area. In addition, the intensity of the result image at the boundary of the occlusion area is the same as the intensity of the destination image (sky).
The author shows that this is done by solving the Poisson equation, so the title of the paper-Poisson image editing can be solved. The theory and implementation details of the paper are actually cool, but they are beyond the scope of this article. However, if you read this article and have questions, please feel free to ask questions in the comments section.
