"Digital Image Processing"
Class: Digital Media Technology Class 1 of 2020
Name: Happy Little Blue
Student ID: XXXXXXXXXX
School of Information, XXXX University
Table of contents
1. Detect the largest connected domain
2. Detection based on skin color
3. Use the tool face recognition that comes with matlab
1.2 Algorithm implementation (code)
1.3 Experimental results and comparative analysis
2.2 Algorithm implementation (code block)
2.3 Experimental results and analysis
4.2 Algorithm implementation (code)
5.2 Algorithm implementation (code)
6.2 Algorithm implementation (code)
6.3 Experimental results and analysis
Realization of face positioning, maximum connected domain, skin color detection, VJ algorithm
I chose the simple Meitu Xiuxiu, and used Matlab's appdesigner to realize the functional interactive interface. The functional modules specifically include face processing functions such as positioning the face, smoothing the skin, removing acne, enlarging eyes, and reducing the nose.
Mainly referring to Meitu Xiuxiu and the usual learning related algorithms, I implemented a basic version of Meitu Xiuxiu, but what I usually learn is not enough to support such a project, so I consulted a lot of functions according to the functions I want to achieve. The data and related codes were implemented, the effect test was carried out, and the code was modified.
I checked a lot of posts on the Internet, and there are mainly 3 algorithms that do not involve deep learning
1. Detect the largest connected domain
Principle: By morphologically processing the image, multiple connected areas are formed, and the largest connected area is the area where the face is located by default.
defect:
1) Sometimes for static pictures, after morphological processing, the largest connected domain is not the part of the face, so this algorithm has relatively large limitations.
2) Only one face in the image can be detected
2. Detection based on skin color
Principle: There are many methods of skin color detection, but whether based on different color spaces or different skin color models, the fundamental starting point is the aggregation of skin color distribution, that is, the components of skin color generally gather within a certain range. Different color spaces have different distribution ranges of skin tones.
1) RGB color space
According to statistics, under uniform light, the distribution range of skin color is: R>95 and G>40 and B>20 and max{R,G,B}-min{R,G,B}>15, and |RG| >15 and R>G and R>B
2) YCbCr color space
According to statistics, the distribution range of skin color in YCbCr color space is 77<=Cb<=127,133<=Cr<=173. The following is the color space used for face detection based on skin color, and the skin color model uses a threshold model. , without choosing a Gaussian model
3) HSV color space
The skin color detection model established in HSV space is required to meet the requirements of 0 degree<H<=25 degree or 335 degree<=H<=360 degree and 0.2<=S=0.6 and 0.4<=V
defect:
- Targeting is inaccurate if the background of the image or the clothing of the person is similar to skin tone
- Some portrait pictures with added filters can't even detect skin color pixels.
3. Use the tool face recognition that comes with matlab
The Cascaded Object Detector uses the Viola-Jones algorithm to detect a person's face, nose, eyes, mouth or upper body. You can also use Image Labeler to train custom classifiers for use with this System object. This tool is relatively accurate in face positioning, and can easily locate the mouth, nose, and eyes. Multiple faces can be located in one picture.
The algorithm used by this tool is relatively complicated. According to the blog posts I read on the Internet, I have a general understanding. The summary is as follows:
The algorithm used by this detector is the VJ algorithm, which is named after the two big names who proposed this algorithm, Viola and Jones. The essence is that, on the basis of the AdaBoost algorithm, the face detection is carried out by using the Haar-like wavelet feature (referred to as haar feature) and the integral map method. First look at the AdaBoost algorithm, which essentially combines multiple weak classifiers into a strong classifier.
- Initialize the sample, assuming that there are N samples, then the weight of each sample is 1/N at the beginning.
- Train the classifier. If the sample is classified as a positive sample during the classification process, that is, the sample classification is accurate, then in the next classification iteration, the weight of the positive sample will be reduced. Conversely, if it is a negative sample, then Increase the weight and iterate N times to obtain a classifier that minimizes the value of the error function of sample classification, thus obtaining a weak classifier, and combining multiple weak classifiers to obtain a strong classifier.
So where do AdaBoost's training samples come from? Then we have to talk about Haar-like wavelet features. Although everyone looks different, human faces have some fixed features, such as darker eyes and nose. The face is relatively dark, but the middle part is brighter than the cheeks, and the mouth part is also darker. According to these features, VJ uses four rectangular features, and subtracts the gray value of the white part from the gray value of the black part to generate a human face. rectangular features. A 24*24 image has 160,000 feature rectangles. If you traverse and sum all the pixels in these feature rectangles once, the amount of calculation is very large, and the integral map appears at this time.
Integral graph, characterized by the integral of the pixel at position (i, j) is the sum of all pixels in the upper left, if you know the integral of some pixels, and find the sum of pixels in a certain area, you only need to perform a few simple additions Minus, no need to traverse each time. Therefore, in a picture, each pixel is represented as an integral, and the integral value is stored in a matrix with the same size as the original image. When calculating the sum of pixels in a specified area, you only need to find the integral at the corresponding position to add and subtract. , the pixel sum of this area can be obtained, and the amount of calculation is greatly reduced.
Defect: Sometimes parts that are not mouths are positioned, or eyes are positioned as mouths.
In general, in order of effect: VJ algorithm > maximum connected domain > skin color detection.
1.2 Algorithm implementation (code)
The first is the face detection code that detects the most connected domain:
Detecting the most connected region
rgb = Img;
I = rgb2gray(rgb);%灰度化
[n1,n2] = size(I);%获取图像矩阵大小
%灰度图
% figure,imshow(I),title('灰度图')
tic
avel=fspecial('average',3);
I=filter2(avel,I)/255;
% figure,imshow(I),title('均值滤波')
BW = imbinarize(I);
% figure,imshow(BW),title('二值化')
B = ones(21);%结构元素
BW = -imerode(BW,B) + BW;
% BW=imdilate(BW,B)-imerode(BW,B);
% figure,imshow(BW),title('形态学边界提取')
BW = bwmorph(BW,'thicken');
% figure,imshow(BW),title('加粗边界')
BW = ~(bwareaopen(~(BW), 50));%使用bwareaopen 函数删除包含的像素数少于50的对象
% figure,imshow(BW),title('把空洞填了')
%进行形态学运算
B = strel('line',50,90);
BW = imdilate(BW,B);
BW = imerode(BW,B);
%figure,imshow(BW),title('闭运算操作之后')
B = strel('line',10,0);
BW = imerode(BW,B);
figure,imshow(BW),title('闭操作之后再腐蚀')
BW = gpuArray(BW);%%将数据载入gpu可以加速，电脑不一定支持
%最小化背景
%细分
div = 10;
r = floor(n1/div);%分成10块 行
c = floor(n2/div);%分成10块 列
x1 = 1;x2 = r;%对应行初始化
s = r*c;%块面积
%判断人脸是否处于图片四周，如果不是就全部弄黑
for i=1:div
y1 = 1;y2 = c;%对应列初始化
for j=1:div
loc = find(BW(x1:x2,y1:y2)==0);%统计这一块黑色像素的位置
num = length(loc);
rate = num*100/s;%统计黑色像素占比
if (y2<=0.2*div*c||y2>=0.8*div*c)||(x1<=r||x2>=r*div)
if rate <=100
BW(x1:x2,y1:y2) = 0;
end
imshow(BW)
else
if rate <=25
BW(x1:x2,y1:y2) = 1;
end
%imshow(BW)
end%下一列
y1 = y1 + c;
y2 = y2 + c;
end%下一行
x1 = x1 + r;
x2 = x2 + r;
end
figure
subplot(1,2,1)
imshow(BW)
title('最终处理')
L = bwlabel(BW,8);%利用belabel函数对8连通域区间进行标号
BB = regionprops(L,'BoundingBox');%得到矩形框，框出每一个连通域
BB = cell2mat(struct2cell(BB));
[s1,s2] = size(BB);
BB = reshape(BB,4,s1*s2/4)';
pickshape = BB(:,3)./BB(:,4);%
shapeind = BB(0.3<pickshape&pickshape<3,:);%筛选掉尺寸比例不合格
[~,arealind] = max(shapeind(:,3).*shapeind(:,4));
subplot(1,2,2)
imshow(rgb)
hold on
rectangle('Position',[shapeind(arealind,1),shapeind(arealind,2),shapeind(arealind,3),shapeind(arealind,3)],...
'EdgeColor','g','Linewidth',2)
title('人脸检测')The second is face detection based on YCbCr color space
code block:
%肤色检测模型
image = imread('person10.jpg');
%三通道分离
image_red = image(:,:,1);
image_green = image(:,:,2);
image_blue = image(:,:,3);
[ROW,COL] = size(image_red);
figure
imshow(image);
title('原图');
YCbCr = rgb2ycbcr(image);%将 RGB 图像的红色、绿色和蓝色值转换为 YCbCr 图像的亮度 (Y) 和色度（Cb 和 Cr）值。
Y = YCbCr(:,:,1);
Cb = YCbCr(:,:,2);
Cr = YCbCr(:,:,3);
pic_gray = zeros(ROW,COL);%创建和原图大小相同的全零矩阵
for i = 1:ROW
for j = 1:COL
if(Cb(i,j) > 77 && Cb(i,j) < 127 && Cr(i,j) > 133 && Cr(i,j) < 173)
pic_gray(i,j) = 255;
else
pic_gray(i,j) = 0;
end
end
end
figure
imshow(pic_gray);
title('肤色检测图');
se = strel('square',20);
erode = imerode(pic_gray,se);
figure
imshow(erode);
title('形态学滤波');
x_min = 1024;
x_max = 0;
y_min = 768;
y_max = 0;
for i = 1:ROW
for j = 1:COL
if(erode(i,j) > 0 && x_min > j)
x_min = j;
end
if(erode(i,j) > 0 && x_max < j)
x_max = j;
end
end
end
for i = 1:ROW
for j = 1:COL
if(erode(i,j) > 0 && y_min > i)
y_min = i;
end
if(erode(i,j) > 0 && y_max < i)
y_max = i;
end
end
end
for i = 1:ROW
for j = 1:COL
if(j == x_min && i >= y_min && i <= y_max)
image_test(i,j,1) = uint8(255);
image_test(i,j,2) = uint8(255);
image_test(i,j,3) = uint8(255);
elseif(j == x_max && i >= y_min && i <= y_max)
image_test(i,j,1) = uint8(255);
image_test(i,j,2) = uint8(255);
image_test(i,j,3) = uint8(255);
elseif(i == y_max && j >= x_min && j <= x_max)
image_test(i,j,1) = uint8(255);
image_test(i,j,2) = uint8(255);
image_test(i,j,3) = uint8(255);
elseif(i == y_min && j >= x_min && j <= x_max)
image_test(i,j,1) = uint8(255);
image_test(i,j,2) = uint8(255);
image_test(i,j,3) = uint8(255);
else
image_test(i,j,1) = image(i,j,1);
image_test(i,j,2) = image(i,j,2);
image_test(i,j,3) = image(i,j,3);
end
end
end
%subplot(2,2,4);
figure
imshow(image_test);
title('头像检测');The third is to use matlab tools to locate the face
code block:
detector = vision.CascadeObjectDetector;%获取检测器
%Read the input image
I = imread('person.jpg');%读入图像
face_dtect = step(detector,I);%人脸检测，返回人脸区域的位置向量
figure
imshow(I)%显示图像
hold on
for i = 1:size(face_dtect,1)%检测器可能检测到多个
rectangle('Position',face_dtect(i,:),'LineWidth',1,'LineStyle','-','EdgeColor','m');%用矩形标注出人脸区域
end
title('人脸检测');1.3 Experimental results and comparative analysis
Skin color detection:
Result analysis:
The effect is better for images whose background and people's clothes are not close to the skin color threshold, but most of the test image positioning is not accurate, especially if the face area is small, or the people's clothes are close to the background and skin color threshold, the effect Just bad.
Maximum connected domain:
Result analysis: The largest connected domain defaults to the largest connected domain of the entire image is the face, so the processing effect is very good for pictures with a large proportion of faces in the image (the bigger the better, the face should be as complete as possible), but like the first In the two pictures, the whole little girl is standing far away, and she is standing sideways. There is a large break in the face part, which cannot be connected, so the detection result is completely wrong. But in general, the probability of success of the test effect is much better than that of skin color detection.
VJ algorithm:
Result analysis:
This algorithm is relatively accurate in face positioning, and can even locate facial features and upper body. The accuracy of the image of the front face is higher, and the face part can be relatively small, but it must be complete. The positioning of the last two pictures above failed. The penultimate picture located two faces. The clothes of the characters are complex in structure. The detector also regarded it as a face. The last picture did not detect the face directly. The error was compared. Big, and when I was testing the facial features, I found that when the detector detects the mouth, it often marks the eyes, but the detection of the nose is more accurate. But the overall effect is much better than the previous two algorithms.
Microdermabrasion: In essence, it is the smoothing of the image. According to the current algorithm for smoothing images learned, there are Gaussian filtering, median filtering, and mean filtering in the spatial domain, and low-pass filtering in the frequency domain. When I was testing, I found that this filter will blur the facial features and boundaries, and the effect is not ideal. Looking through the textbook, I found that the bilateral filter was used in the textbook to smooth the face. My head was dizzy after reading the formula in the book, but after listening to Liu Longhao's explanation, I found this algorithm is not difficult.
This algorithm is an improvement on Gaussian filtering. Gaussian filtering only considers the influence of position on pixel values, that is, the closer the position is, the greater the impact on pixel values, so important information such as boundaries will be blurred. In bilateral filtering, position and pixel value are considered equally important. In flat areas, the position of pixels dominates the effect of bilateral filtering. In the edge area, since the pixel values on both sides of the edge are very different, the weight of the pixels on the other side of the edge is reduced, which hardly affects the pixels on this side of the edge, which achieves the effect of edge preservation.
2.2 Algorithm implementation (code block)
%用双边滤波实现人脸美化的效果
I = imread('person2.jpg');%读取图像
figure(1)
imshow(I),title('原图');%显示原图
bfilt_rgb(I,15,6,0.1);%调用双边滤波函数，进行滤波处理
%函数实现
function g = bfilt_rgb(f,r,a,b)
% f彩色图；r滤波半径；a全局方差；b局部方差
[x,y] = meshgrid(-r:r);%创建二维网格坐标
w1 = exp(-(x.^2+y.^2)/(2*a^2));%计算滤波内的空间权值
f = im2double(f);%将uint8转换成double，方便后续处理
h = waitbar(0,'Applying bilateral filter...');%设置进度条
set(h,'Name','Bilateral Filter Progress');
detector = vision.CascadeObjectDetector;%目标探测器
facePart = step(detector,f);
%获取脸部坐标
X_min = facePart(1,1);%人脸区域的左下x坐标
Y_min = facePart(1,2);%y坐标
Width = facePart(1,3);%区域宽度
Height = facePart(1,4);%区域高度
%三通道分离
fr = f(:,:,1);
fg = f(:,:,2);
fb = f(:,:,3);
[m,n] = size(fr);%图像矩阵大小，方便后面像素的遍历
fr_temp = padarray(fr,[r r],'symmetric');%镜像填充矩阵，方便处理边界像素
fg_temp = padarray(fg,[r r],'symmetric');%镜像填充矩阵，方便处理边界像素
fb_temp = padarray(fb,[r r],'symmetric');%镜像填充矩阵，方便处理边界像素
[gr,gg,gb] = deal(zeros(size(fr)));%创建三个和原图像大小相同的全零矩阵
for i = r+1+Y_min:Height+Y_min+r%对人脸部分进行双边滤波
for j = r+1+X_min:X_min+Width+r
temp1 = fr_temp(i-r:i+r,j-r:j+r);%三通道的滤波区域31X31的正方形
temp2 = fg_temp(i-r:i+r,j-r:j+r);
temp3 = fb_temp(i-r:i+r,j-r:j+r);
dr = temp1 - fr_temp(i,j);
dg = temp2 - fg_temp(i,j);
db = temp3 - fb_temp(i,j);
w2 = exp(-(dr.^2+dg.^2+db.^2)/(2*b^2));%求滤波内每个像素对目标像素的影响权值（基于像素值）
w = w1.*w2;%将两种权值相加，求得滤波范围内每个像素的最终权值
gr(i-r,j-r) = sum(sum(temp1.*w))/sum(w(:));%将权值与对应像素值相乘相加，再除以权值和，即求得该像素点经双边滤波后的像素值
gg(i-r,j-r) = sum(sum(temp2.*w))/sum(w(:));
gb(i-r,j-r) = sum(sum(temp3.*w))/sum(w(:));
end
waitbar((i-r)/n);%进度条
end
%将人脸部分赋黑，方便后面图像的叠加
for m=X_min+2:X_min+Width
for n=Y_min+1:Height+Y_min
f(n,m,1)=0;
f(n,m,2)=0;
f(n,m,3)=0;
end
end
g = cat(3,gr,gg,gb);%串联数组，合并三通道
result=imadd(g,f);
figure(2)
imshow(result),title('双边滤波');%显示处理图
end2.3 Experimental results and analysis:
Effect analysis: Bilateral filtering not only smoothes the skin, but also preserves the facial features. The effect is really good, but the processing speed is slower.
The first is to use the face detector of the matlab toolbox to detect the position vectors of the two eyes, and then zoom in on the two eyes separately: first determine the center coordinates of the enlarged area according to the position vector of each eye, and then set the enlarged area The length of the side, traverse the area that needs to be enlarged, and use the nearest neighbor interpolation method to obtain the enlarged pixel value. After the traversal is completed, the enlargement effect is realized.
%放大眼睛
I = imread('person13.jpg');%读入图像
I = im2double(I);%数据类型转化，以便后续处理
%V-J算法检测眼睛
detector = vision.CascadeObjectDetector;%调用检测器
detector.ClassificationModel='LeftEyeCART';%调用眼睛检测器
bboxes = detector(I);%检测眼睛，返回眼睛区域的位置向量
%放大眼睛
strength=20;%放大强度
pointx=floor(bboxes(1,2)+(bboxes(1,3)/2));%根据检测器大致确定眼睛中心点
pointy=floor(bboxes(1,1)+(bboxes(1,4)/2))+10;
r=50;%放大区域边长
left=pointy-r;
right=pointy+r;
top=pointx-r;
bottom=pointx+r;
space=r*r;%放大的正方形区域
I1=I;%将原图缓存，存储处理后的图像
for x=top:bottom%遍历放大区域的像素
offsetx=x-pointx;
for y=left:right
offsety=y-pointy;
xy=offsetx*offsetx+offsety*offsety;
if xy<=space
scale=1-xy/space;
scale=1-strength/100*scale;
%减少计算量，采用最邻近插值，利用四舍五入函数实现
posy=round(offsety*scale+pointy);%放大后的Y坐标值
posx=round(offsetx*scale+pointx);%放大后的X坐标值
%分别对三通道进行处理
I1(x,y,1)=I(posx,posy,1);
I1(x,y,2)=I(posx,posy,2);
I1(x,y,3)=I(posx,posy,3);
end
end
end
pointx=floor(bboxes(2,2)+(bboxes(2,3)/2));%对另一只眼睛进行同样的处理
pointy=floor(bboxes(2,1)+(bboxes(2,4)/2));
r=50;
left=pointy-r;
right=pointy+r;
top=pointx-r;
bottom=pointx+r;
space=r*r;
I2=I1;
for x=top:bottom
offsetx=x-pointx;
for y=left:right
offsety=y-pointy;
xy=offsetx*offsetx+offsety*offsety;
if xy<=space
scale=1-xy/space;
scale=1-strength/100*scale;
posy=round(offsety*scale+pointy);
posx=round(offsetx*scale+pointx);
I2(x,y,1)=I1(posx,posy,1);
I2(x,y,2)=I1(posx,posy,2);
I2(x,y,3)=I1(posx,posy,3);
end
end
end
figure
subplot(121),imshow(I2),title('放大眼部');
subplot(122),imshow(I),title('原图');
Effect analysis: Because this function is mainly based on the position recognition of human eyes, the accuracy of human eye position positioning directly affects the realization effect of the eye enlargement function. For some portraits, the eyes cannot even be detected, so it is fundamental This function cannot be used, and some portraits will recognize the mouth part as the eye part, so the mouth part will be enlarged, but as long as the eyes can be accurately positioned, the processing effect will be very good.
This algorithm is still based on face detection VJ and area zooming algorithm, and the steps are similar to eye zooming. First use matlab to locate the position of the nose, then determine the center coordinates of the nose according to the position vector returned by the tool, and finally use the area enlargement algorithm, but change the intensity to a negative number to achieve the effect of area reduction, that is, nose reduction surgery.
4.2 Algorithm implementation (code)
%瘦鼻
I = imread('person25.jpg');
detector = vision.CascadeObjectDetector;%检测器
detector.ClassificationModel='Nose';
detector.MergeThreshold=10;%增加合并阈值
bboxes=detector(I);
Nose=insertObjectAnnotation(I,'rectangle',bboxes,'Nose');%用矩形框标出鼻子定位的矩形区域
figure(9);imshow(Nose);title('Nose'); %显示图像
%区域图像缩小
strength=-20;%缩小强度
pointx=floor(bboxes(1,2)+(bboxes(1,3)/2));%根据检测器大致确定鼻子中心点
pointy=floor(bboxes(1,1)+(bboxes(1,4)/2));
r=50;%放大区域边长
left=pointy-r;
right=pointy+r;
top=pointx-r;
bottom=pointx+r;
space=r*r;%缩小的正方形区域
I1=I;%将原图缓存，存储处理后的图像
for x=top:bottom%遍历缩小区域的像素
offsetx=x-pointx;
for y=left:right
offsety=y-pointy;
xy=offsetx*offsetx+offsety*offsety;
if xy<=space
scale=1-xy/space;
scale=1-strength/100*scale;
%减少计算量，采用最邻近插值，利用四舍五入函数实现
posy=round(offsety*scale+pointy);%放大后的Y坐标值
posx=round(offsetx*scale+pointx);%放大后的X坐标值
%分别对三通道进行处理
I1(x,y,1)=I(posx,posy,1);
I1(x,y,2)=I(posx,posy,2);
I1(x,y,3)=I(posx,posy,3);
end
end
end
figure
imshow(I1);
Effect analysis: This function is almost the same as enlarging the eyes, so as long as the nose is positioned accurately, the effect of reducing the nose wing is very good, but for portraits whose nose cannot be positioned, it cannot be reduced, and I found a magic It turns out that the nose becomes smaller, which will have the effect of visually enlarging the eyes.
This algorithm is mainly based on pixel translation. I refer to a post on the liquefaction algorithm on the Internet, which gives the inverse transformation formula and the steps to implement the algorithm, but does not give the implementation code. Therefore, I did not understand this algorithm deeply at the beginning. At the time, it was impossible to write the code for implementation, but when I looked at the algorithm again later, I found that this algorithm is actually pixel translation, because when the translated pixel reaches the target position, the coordinates calculated according to the transformation formula are Double-precision, and the coordinates of the image matrix are positive integers, so when translating, no matter what interpolation algorithm is used, the problem of pixel overlap or missing pixel values may occur, so the inverse transformation is used to reverse the original position with the target position The position of the pixel, because the inverse is also double-precision, so the bilinear interpolation algorithm is used for the adjacent pixels of the original pixel value, and the calculated pixel value is the pixel value at the destination position, thus achieving a relatively smooth translation of the pixel. After the algorithm is implemented, I use the interactive selection of a circular area, and then select a translation center, the purpose is to smoothly translate the pixels on the edge of the face, so as to achieve the effect of thinning the face visually, but in fact, in operation, Once the distance between the selected area and the center of the target circle is too far, distortion will occur, resulting in severe deformation of the face. I have tried many face images, and only one of them has a good effect, but it also requires many times of inconspicuous translation of pixels. , will see a more obvious face-lifting effect. I've been thinking about how to perfect it, but I don't seem to have enough time, so I gave up.
5.2 Algorithm implementation (code)
for cou =1:300
if cou==1
I = imread('C:\Users\leishen\Desktop\数字图像处理\期末大作业\测试效果\person7.jpg');%读入图片
else
I = imread('C:\Users\leishen\Desktop\数字图像处理\期末大作业\测试效果\person25.jpg');%读入修改之后的图片，以便继续进行瘦脸处理
end
figure
imshow(I);
h = drawcircle('Color','k','FaceAlpha',0.4);%交互式选取圆形区域
M = ginput(1); %交互式选取坐标，即变化后的圆心的位置
[m,n,~] = size(I);%图像矩阵的大小
center = [floor(h.Center(2)),floor(h.Center(1))];%变形前的圆心
radius = h.Radius;%变形的圆半径
RGB_buff = I;
%遍历圆形选区的每一个像素
for i=1:m
for j=1:n
distant = sqrt((i-center(1)).^2+(j-center(2)).^2);%判断该像素点是否在选取的圆形区域内
if distant <= radius
x = [i,j];%变形后的坐标
U = x - ((radius^2 - (x - center).^2) / (radius^2 - (x - center).^2 + (M - center).^2))^2 * (M - center);%逆变换公式
uu = floor(U);%舍弃小数部分，保留整数
ab = U-uu;%这个数据用于后面双线性插值处理
a = ab(1);
b = ab(2);
m1 = uu(1);
n1 = uu(2);
for k=1:3
I(i,j,k)=(1-a)*(1-b)*RGB_buff(m1,n1,k)+a*(1-b)*RGB_buff(m1+1,n1,k)+(1-a)*b*RGB_buff(m1,n1,k)+a*b*RGB_buff(m1+1,n1+1,k);%双线性插值
end
end
end
end
figure
imshow(I);
answer = questdlg('是否要继续调整', ...
'选择', ...
'YES','NO','NO');
switch answer
case 'YES'
imwrite(I,'C:\Users\leishen\Desktop\数字图像处理\期末大作业\测试效果\person25.jpg');
case 'NO'
msgbox("退出调整");
break;
end
end
It can be seen that the left side of the face has achieved a large pixel translation effect, so it has the effect of thinning the face visually, but if you look closely, you will find that the pixel translation has caused local distortion and blurring of the image.
This implementation principle is relatively simple. First, interactively select the area you want to remove acne, and then extract the first pixel in the upper left corner of this area, and use the pixel value in the range from -5 to +5 to randomly fill the selection. Area. Then, in order to weaken the filled boundary, Gaussian filtering is used to process the face with Gaussian filtering.
6.2 Algorithm implementation (code)
g=imread('person1.jpg');
% n 为操作次数，默认为1，g为原图
n=3;
g2 = g;
%得到原图像的大小
[ M,N,~ ] = size( g );
while( n ~=0 )
n = n - 1;
%进行交互选择处理区域
mask = roipoly( g2 );%roipoly函数--指定多边形感兴趣区域，将蒙版作为二进制图像返回，所以下面的乘法就可以得到原图像被提取的区域，因为0的部分乘以任何像素值都为0,1乘以原像
%素还是等于原像素
x1 = immultiply( mask,g2( :,:,1 ) );
x2 = immultiply( mask,g2( :,:,2 ) );
x3 = immultiply( mask,g2( :,:,3 ) );
x = cat( 3,x1,x2,x3 );%将x1和x2和x3沿第三维度串联起来，这里相当于三通道合并
% figure
% imshow(x);%原来x是交互时提取的区域
%预分配内存，f1,f2,f3存储三个通道的运算结果
f1 = zeros( M,N );
f2 = zeros( M,N );
f3 = zeros( M,N );
%跳出双重循环设置flag
flag = 0;
%找到第一个像素值不为0的点，得到该点像素值，作为采样后填充的像素
for i = 1:M
for j = 1:N
if( x1( i,j ) ~= 0 )
r = x( i,j,: );
flag = 1;%找到之后，跳出循环
break
end
if( flag == 1 )
break
end
end
end
%随机产生填充图像，这就是填充痘印部分的图像
y = zeros(3,3,3);
y( :,:,1 ) = randi([r(1)-5,r(1)+5],[3,3]);%取像素值-5到像素值+5的范围的随机数组成一个3X3的矩阵
y( :,:,2 ) = randi([r(2)-5,r(2)+5],[3,3]);
y( :,:,3 ) = randi([r(3)-5,r(3)+5],[3 3]);
%类型转换
y = double(y);
%对于三个通道分别进行处理，用采样得到的像素点取代原来的像素点
for i = 2:3:M-1
for j = 2:3:N-1
f1( i-1:i+1,j-1:j+1 ) = mask( i-1:i+1,j-1:j+1 ).* y( :,:,1 );
f2( i-1:i+1,j-1:j+1 ) = mask( i-1:i+1,j-1:j+1 ).* y( :,:,2 );
f3( i-1:i+1,j-1:j+1 ) = mask( i-1:i+1,j-1:j+1 ).* y( :,:,3 );
end
end
%将三个通道连接在一起
f = cat( 3,f1,f2,f3 );
%类型转换
f = uint8( f );
%得到处理后图像，居然可以直接加减，那还学什么运算函数
a = g2 - x;
f = f + a;
g2 = f;
end
%图像滤波处理，将图像进行高斯滤波处理，平滑
f = double(f);
b = double( imguidedfilter( uint8( f ) ) ) - f + 128 ;
t = imfilter( b, fspecial( 'gaussian',[5 5]) );
f = ( f*50 + ( f+2*t-256 )*50 )/100;
f = uint8(f);
%显示图像
subplot( 1,2,1 ),imshow( g ),title('原图');
subplot( 1,2,2 ),imshow( f ),title('填充处理后图像');6.3 Experimental results and analysis
Result analysis: Because the pixels in the acne-removing area are randomly filled according to the range of the first pixel +5-5 in the upper left corner of the area. It can be seen that if the acne grows to the dead corner of the face, the filling effect is very abrupt. Even in a flat area, the face and the filled area are not well connected, and the filled area is rather abrupt. Although Gaussian filtering is performed, the effect It's still average, but for faces without highlights, the effect of removing acne should be good.
In essence, it is a texture extraction process. First, separate the three channels, choose a channel for processing, and then invert this channel, and then use Gaussian filtering to filter the negative film to remove some noise. Finally, the positive film and the Negative films are superimposed, and after final normalization, a more ideal sketch effect is obtained.
7.2 Algorithm implementation (code block)
I = imread('person11.jpg');
[height,width,~]=size(I);%获取图像矩阵的大小
N=zeros(height,width);%创建两个全零矩阵
G=zeros(height,width);
%分离三通道
rc = I(:,:,1);
gc = I(:,:,2);
bc = I(:,:,3);
%选择一个通道进行处理
channel = gc;
out=zeros(height,width);
spec=zeros(height,width,3);
%颜色取反
for i=1:height
for j=1:width
N(i,j)=uint8(255-channel(i,j));
end
end
%高斯模糊
gausize = 10;%滤波器大小，越大越模糊
gausigma = 12; %越大越模糊
GH = fspecial('gaussian', gausize, gausigma);%构建高斯滤波模板
G = imfilter(N, GH);%滤波处理
for i=1:height
for j=1:width
b=double(G(i,j));
a=double(channel(i,j));
temp=a+a*b/(256-b);%将滤波后的负片和正片相叠加
out(i,j)=uint8(min(temp,255));
end
end
%模糊程度越高，得到的素描结果越清晰，框架纹理颜色越深
figure
subplot(121),imshow(out/255);%归一化处理并显示图像
subplot(122),imshow(I);
Effect analysis: The clearer the picture, the better the effect, because this processing is mainly to extract the texture, and then achieve a sketch-like effect.
Define a rectangular window, use this window to cross each pixel of the image (three channels are processed separately), calculate the gray value of each pixel in the rectangle, and take the largest pixel value in this area as the current pixel value, This achieves a simple oil painting effect.
8.2 Algorithm implementation (code block)
A=imread('person4.jpg');
% 油画效果
%创建12X12的窗口矩阵，窗口越大，丢失的信息就越多，效果就越明显
m=12;
n=12;
Image=uint8(zeros([size(A,1)-m,size(A,2)-n,3]));
figure
imshow(Image);
%计算每个RGB值的直方图
for v=1:3
for i=1:size(A,1)-m
for j=1:size(A,2)-n
mymask=A(i:i+m-1,j:j+n-1,v);
h=zeros(1,256);
for x=1:(m*n)
h(mymask(x)+1)=h(mymask(x)+1)+1;
end
[maxvalue,pos]=max(h);%记录最大值和最大值对应的位置
Image(i,j,v)=pos-1;%将最大值赋值给当前的像素
end
end
end
figure
subplot(121),imshow(Image);
subplot(122),imshow(A);
Effect analysis: The larger the window, the more obvious the effect, but because the maximum value is taken, the sudden change in some places is very abrupt, and the effect is not very ideal, but the oil painting effect is basically realized.
First create a processing window, you can customize the size, then use this window to slide each pixel of the image, randomly select a pixel of this window, assign it to the currently sliding pixel, until all pixels are traversed, the frosted glass effect is completed .
9.2 Algorithm implementation (code block)
A=imread('buiding1.jpg');%读入图像
%定义一个6X7的矩形窗口
m=6;
n=7;
Image=uint8(zeros([size(A,1)-m,size(A,2)-n,3]));%存储窗口处理后的图像
for i=1:size(A,1)-m
for j=1:size(A,2)-n
mymask=A(i:i+m-1,j:j+n-1,:);%将图像对应像素值赋值给窗口
x2=ceil(rand(1)*m);%从窗口像素随机挑选一个像素，因为坐标必须为正整数，所以用ceil四舍五入为整数
y2=ceil(rand(1)*n);
Image(i,j,:)=mymask(x2,y2,:);%将选出的像素值赋值给当前的像素
end
end
imshow(Image);%显示图像
The realization of this effect is relatively simple, in essence, it is to extract the boundary of the image, and then highlight the boundary. The specific steps are to perform Sobel filtering on the grayscale image, extract the boundary, and then perform grayscale conversion.
10.2 Algorithm implementation (code block)
img=imread('building2.jpg');%读入图像
f0=rgb2gray(img);%改为灰度图像
f1=im2double(f1);
h2=fspecial('sobel'); %进行sobel滤波
g3=filter2(h2,f1,'same');
K=mat2gray(g3); %将矩阵转换成灰度图
figure
imshow(K);
Effect analysis: The relief effect depends on the effect of boundary extraction, so the image with a large change in gray value has a good processing effect, so this effect is not bad for images such as buildings, so I posted a picture of a processed building .
In essence, it is the rotation of the matrix. The rotation axis is the center of the image. Let each coordinate of the image calculate the rotated coordinates through the rotation formula, but the calculated coordinates are floating-point numbers, and the coordinates must be positive integers, so they must be rounded, but After rounding, the pixel values on some coordinates may overlap, and there may be no pixels on some coordinates, so the coordinates after transformation are used to determine the coordinates before transformation, and then the pixels near the coordinates before transformation are used for simple linear Interpolation, calculate the pixel value of the transformed coordinates, and after traversing the image, the rotated image is obtained.
11.2 Algorithm implementation (code block)
%图像旋转
I = imread('person10.jpg');%读入图像
I = im2double(I);
I2=imrotate(I,-22);
figure
subplot(121),imshow(I2),title('顺时针旋转22度');%调用函数实现
%自己写代码实现
%旋转22度
a = 22/180*pi;%角度转换成弧度
Rotate = [cos(a) -sin(a);sin(a) cos(a)];%旋转变换矩阵
[m,n,~] = size(I);%矩阵大小
center = [m,n]/2;%旋转轴中心
% 计算显示完整图像需要的画布大小
hh = floor(m*sin(a)+n*cos(a))+1;
ww = floor(m*cos(a)+n*sin(a))+1;
c2 = [hh; ww] / 2;%旋转之后的轴中心
res = zeros(size(I));%创建存储结果的全零矩阵
for k=1:3%三个通道
for i=1:hh
for j=1:ww
R = Rotate*([i;j]-c2)+center;%变换公式
R1 = floor(R);%向负无穷舍入
ab = R-R1;%线性插值算法
a = ab(1);
b = ab(2);
m1 = R1(1);
n1 = R1(2);
if (R(1) >= 1 && R(1) <= m && R(2) >= 1 && R(2) <= n)
res(i, j, k) =(1-a)*(1-b)*I(m1,n1, k) + a*(1-b)*I(m1+1, n1, k)...
+ (1-a)*b*I(m1,n1+1,k)+ a*b*I(m1+1,n1+1, k);%简单线性插值
end
end
end
end
subplot(122),imshow(res),title('顺时针旋转变化22');%显示图像
For the self-programming function to achieve rotation, there are problems such as stretching of the picture and incomplete display of the picture, and there are still some problems in the implementation of the algorithm.
Use the interactive function to extract the cropped area, and then display the cropped image according to the returned parameters.
12.2 Algorithm implementation (code block)
%交互式裁剪图像
I =imread(‘person.jpg’）；%读取图像
[J,rect] = imcrop(I);调用函数交互式裁剪图像，J返回的是裁剪的区域，rect返回的是指定裁剪区域的四元素位置向量。
I2 = imcrop(I,rect);%根据返回的位置向量裁剪图像并返回
imshow(I2);%显示裁剪的图像
Effect analysis: This is the realization of interactive cutting.
Use the matlab function fspecial to get the motion simulation filter by setting the motion displacement parameter and displacement angle parameter, and then use the filter to filter the image to obtain the motion blurred image.
13.2 Algorithm implementation (code block)
%运动模糊
I = imread('buiding1.jpg');%读入图像
len=20;%设置运动位移为20个像素
theta=50;%设置运动角度为45度
psf = fspecial('motion',len,theta);%建立二维运动仿真滤波器psf
j = imfilter(I,psf,'circular','conv');%用滤波器产生退化图像
figure
subplot(121),imshow(j),title('运动模糊');%显示运动模糊图像
subplot(122),imshow(I),title('原图');%显示原图像
The flipping of an image is essentially the flipping of the element sequence on the matrix. The flip function, if there is only the image I that needs to be flipped, flips the element sequence of each column and row by default. If you add dimension 1 to the back, it flips each column. The element sequence of each row achieves the effect of flipping up and down. If dimension 2 is added behind, the element sequence of each row is flipped, and the effect of flipping left and right is realized.
14.2 Algorithm implementation (code block)
%图像翻转
I = imread('person.jpg');
res = flip(I,2);%左右翻转
res1 = flip(I,1);%上下翻转
figure,
subplot(131),imshow(I),title('原图');
subplot(132),imshow(res),title('左右翻转');
subplot(133),imshow(res1),title('上下翻转');
This function mainly relies on matlab's interactive selection of polygonal regions of interest. After selection, the unselected part of the image is assigned a value of 0. The specific operation is: because the roipoly function returns a binary image, the element value of the selected area is 1. , the value of the unselected part of the element is 0, so the three channels of the image are multiplied by the binary image, and then the cat function is used to realize the synthesis of the three channels, and the deducted image is obtained.
15.2 Algorithm implementation (code block)
%抠除背景
%基于提取感兴趣区域扣除背景
I = app.updateImg;
g2=I;
%进行交互选择处理区域
mask = roipoly( g2 );%roipoly函数--指定多边形感兴趣区域，将蒙版作为二进制图像返回，所以下面的乘法就可以得到原图像被提取的区域，因为0的部分乘以任何像素值都为0,1乘以原像
%素还是等于原像素
app.mask=mask;
x1 = immultiply( mask,g2( :,:,1 ) );
x2 = immultiply( mask,g2( :,:,2 ) );
x3 = immultiply( mask,g2( :,:,3 ) );
x = cat( 3,x1,x2,x3 );%将x1和x2和x3沿第三维度串联起来，这里相当于三通道合并
app.Bg=x;
imshow(x,'parent',app.UIAxes),title('扣取的图像');
Effect analysis: It is this kind of interactive. It is very cumbersome to cut out a more perfect object, and you can only pull a straight line. The edge of the deduction object will be very stiff and there are many redundant parts.
This function is mainly based on the previous function of removing the background, because this function is essentially an image superposition, so if the background image is smaller than the deducted image, an error will be reported, so I made a judgment, if the selected background If the picture is too small, a prompt box will pop up, then you can only select the background picture again. Then use the size of the image to crop the background image to facilitate the addition of the subsequent images, and finally subtract the deducted image from the background image (use multiplication to make the pixel value of the deducted portrait part 0), and then use the superposition of the image to get Composite image of background and foreground.
16.2 Algorithm implementation (code block)
%创建文件选择对话框选择指定图像
[file,path]=uigetfile('*.png');
bg=imread(fullfile(path,file));%读取图像
[M,N,~]=size(bg);
[A,B,~]=size(app.Bg);
%比较背景图和人像图的矩阵大小
if(M<A|N<B)
%弹出警告对话框
msgbox("当前选择背景图过小，请重新选择！","警告",'non-modal');
end
%基于提取感兴趣区域扣除背景
I = app.Bg;
[M,N,~] = size(I);%获取图片的大小
%按照图像裁剪背景图
bg = imcrop(bg,[0,0,N,M]);
mask=app.mask;
% figure
% imshow(bg),title('裁剪的图像');
%用裁剪的图像减去扣取的图像
b1 = immultiply( ~mask,bg( :,:,1 ) );
b2 = immultiply( ~mask,bg( :,:,2 ) );
b3 = immultiply( ~mask,bg( :,:,3 ) );
b =cat(3,b1,b2,b3);
%处理之后的背景图加上扣取的图像
app.updateImg=b+I;
imshow(b+I,'Parent',app.UIAxes);
Effect analysis: Because when dealing with the overlay of background and portrait, I cut the background image according to the size of the portrait image, so the background image cannot always be displayed completely, only the part in the upper left corner can be displayed. Moreover, the background image must be larger than the portrait image, otherwise, when cropping, the array will go out of bounds and an error will be reported.
Realization of face positioning, maximum connected domain, skin color detection, VJ algorithm
Bilateral filtering is used to process the skin smoothing effect, which has a good edge protection effect
Enlarge the eyes, mainly based on eye positioning, and then enlarge the partial image
Reducing the alar of the nose is similar to the principle of enlarging the eyes. Firstly, the nose is positioned, and then the partial image is reduced.
Face-lifting, although the implementation effect is very poor, but the algorithm written according to the formula is still very successful. It may be necessary to perform a continuous liquefaction translation for each pixel in the circular area, but there is no time to implement it.
At the beginning of learning this course, after I got the teaching material, I opened the book, I was dumbfounded, it was all Chinese characters, but I just couldn’t understand it. Every time after class, I wanted to take out the textbook to read, but gave up after reading a few pages. In the later stage of this course, in the part of morphology, I feel that these algorithms can be used to achieve a lot of functions, so when I was working on a big project, I opened the book again. The codes and algorithms in the book seemed not so strange, and some Algorithms can even be written out by themselves.
To realize the simple version of Meitu Xiuxiu, I think that the problem of face positioning must be solved first, so I started to check the relevant algorithms, searched for a few days, and I summarized it. There are four main methods: maximum connected domain It is a human face), skin color detection (there are many color models, and the threshold range of skin color is available in textbooks, I use the YCbCr color model), VJ algorithm (based on haar-like feature extraction), and deep learning (not involved ). Then I tested the first three algorithms with 20 clearer portrait samples to compare their pros and cons. There is no doubt that the VJ algorithm is far better than the other two, so I used the face detector that comes with matlab (based on the VJ algorithm). Then I went to see the VJ algorithm and read a lot of blog posts. After class, I opened the blog post explaining the VJ algorithm again, and this time I understood 80% to 90%.
Then there is facial skin resurfacing. At first, I only thought of various filters in the spatial domain and frequency domain, so I implemented it myself. The effect was very poor, and the facial features were almost smoothed. I don’t know when I opened the textbook by chance, and found that there is a smoothing filter in the textbook - bilateral filtering, which not only considers the position of the pixel, but also considers the value of the pixel. If the position is close but the difference in gray value is large, this algorithm will reduce The weight of this pixel value, so the effect of smooth edge preservation is achieved. So I wrote it with reference to the code in the textbook. Of course, the textbook is the processed grayscale image. I can only separate the three channels of the color image, perform bilateral filtering respectively, and then synthesize it again. Although this algorithm is computationally intensive, The processing speed is very slow, but the effect is better than other filters.
The third is the function of eye enlargement. I had no idea about this at the beginning. Although the face positioning in matlab can locate the eyes, it is sometimes inaccurate. For example, when positioning the mouth, the eyes will be recognized together. I think that the enlargement of the eyes is not the local enlargement of the image in essence? As long as the increased range is determined and then interpolated, it can be realized, so I started to use matlab to locate a single eye. It is funny that I only let it locate the left eye, but it returned the position of the two eyes. The position vector, and then I enlarge the image of the two eyes separately. I use the nearest neighbor interpolation instead of bilinear interpolation, so the amount of calculation is less, and the effect is not bad.
Shrinking the wings of the nose, this function is when I was sleeping, I suddenly wanted to realize such a function. At first, I thought about reducing the nose area proportionally. After realizing it, I found that the effect was very poor, and the nose was crooked, because the algorithm itself is also The pixel-based interval sampling, and then linear interpolation, realizes the reduction of the image. The nose area will be reduced to the upper left corner of the original nose area. Later, I found out that the function of enlarging the eyes is not the same effect? Just change the zoom in to zoom out, so I changed a few lines of code, and the effect came out quickly.
Again, it is the effect of thinning the face. I had no idea at first. The algorithms I searched were all in python language, and I didn't understand the blogger's explanation. This opportunity was another algorithm explanation given by Liu Longhao. Although the liquefaction algorithm is essentially a translation of pixels, I have never found the formula for inverse transformation. Therefore, I have put this algorithm on hold for a long time and searched the entire network every day. Finally, I finally saw an article explaining the liquefaction algorithm. The blogger spoke very clearly. Although the code was not given, but with the inverse transformation formula, I tried to write the code by myself. In fact, it is relatively simple. First, traverse the selected circular area pixels, and then determine the center of the circle after the translation, you can calculate the corresponding position after the transformation according to the inverse transformation formula, and the corresponding pixel value before the transformation, and then perform bilinear interpolation. This place reminds me of the sophomore "Computer Graphics", I didn't know the use of this algorithm before, but I didn't expect it to come in handy this semester. Although the pixels in the circular area are translated as expected, but when the face pixels are moving, there is a big split. So, although I spent a lot of time on this feature, this feature was a failure.
In addition to the realization of each function, we also need to consider the relationship between buttons, how to set variables to make the whole software run smoothly, instead of reporting errors at random. I used several attributes to save the transformed image, so that the processed image can be processed further, but in the filter part, those effects cannot be superimposed, so I set a save button for the user to confirm the desired effect After wanting this effect, save the effect picture, and then perform subsequent processing. These made me realize that the problem that an image processing software needs to consider is not only the realization of each functional module, but also according to the user's habits. Functional modules are linked together for implementation, while functional modules like filters need to be implemented independently.
Reference blog:
(37 messages) Integral graph algorithm and code implementation (notes)_molihong28's blog-CSDN blog
(37 messages) [Matlab] Simple face acne removal_ZJU_fish1996's blog-CSDN blog
(37 messages) Image processing algorithm for thin face and enlarged eyes
(37 messages) Viola-Jones face detection_Communication Program Ape's Blog-CSDN Blog
(37 messages) Getting started with MATLAB App Designer (1) - slandarer's blog - CSDN blog