Train a neural network to make her recognize me
During this time, I am learning the convolutional neural network part of tensorflow. In order to have a deeper understanding of the convolutional neural network, it is a better way to implement a routine by myself, so I chose such an interesting project.
In order for her to recognize me, she needs to give her some pictures of me and let her remember my facial features. In order for her to distinguish me from others, she also needs to give her some pictures of other people as a reference, so you need two Set a data set for her to learn. If you want her to know a few more people, just give her a few more sets of pictures to learn. Now let's start to build this "she" who can know me.
Operating environment
The following is a system environment for the operation of the software.
System: window or linux
Software: python 3.x, tensorflow
python support library:
tensorflow:
pip install tensorflow #cpu version pip install rensorflow-gpu #gpu version, requires the support of cuda and cudnn, if you are unclear, you can choose the cpu version
numpy:
pip install numpy
opencv:
pip install opencv-python
dlib:
pip install dlib
Get my picture collection
Of course, the way to get my own photos is to take pictures. We need to take pictures of ourselves through the program. If you have photos yourself, you can also use those ready-made photos, but the premise is that you have enough photos. The number of photos used this time is 10,000. After the program runs, you have to sit in front of the computer and keep putting your face in various poses. This can improve the success rate of identifying yourself after training. Random changes are added to the program. The contrast and brightness modules are also designed to increase the diversity of photo samples.
The program uses dlib to recognize the face part, and opencv can also be used to recognize the face. In actual use , the recognition effect of dlib is better than that of opencv, but the speed of opencv recognition will be much faster , and 10,000 faces can be obtained. In the case of photos, dlib takes about 1 hour, while opencv takes about 20 minutes. opencv might recognize some strange parts, so after all things considered I used dlib to recognize faces.
get_my_faces.py
import cv2
import dlib
import os
import sys
import random
output_dir = './my_faces'
size = 64
if not os.path.exists(output_dir):
os.makedirs(output_dir)
# 改变图片的亮度与对比度
def relight(img, light=1, bias=0):
w = img.shape[1]
h = img.shape[0]
#image = []
for i in range(0,w):
for j in range(0,h):
for c in range(3):
tmp = int(img[j,i,c]*light + bias)
if tmp > 255:
tmp = 255
elif tmp < 0:
tmp = 0
img[j,i,c] = tmp
return img
#使用dlib自带的frontal_face_detector作为我们的特征提取器
detector = dlib.get_frontal_face_detector()
# 打开摄像头 参数为输入流，可以为摄像头或视频文件
camera = cv2.VideoCapture(0)
index = 1
while True:
if (index <= 10000):
print('Being processed picture %s' % index)
# 从摄像头读取照片
success, img = camera.read()
# 转为灰度图片
gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# 使用detector进行人脸检测
dets = detector(gray_img, 1)
for i, d in enumerate(dets):
x1 = d.top() if d.top() > 0 else 0
y1 = d.bottom() if d.bottom() > 0 else 0
x2 = d.left() if d.left() > 0 else 0
y2 = d.right() if d.right() > 0 else 0
face = img[x1:y1,x2:y2]
# 调整图片的对比度与亮度， 对比度与亮度值都取随机数，这样能增加样本的多样性
face = relight(face, random.uniform(0.5, 1.5), random.randint(-50, 50))
face = cv2.resize(face, (size,size))
cv2.imshow('image', face)
cv2.imwrite(output_dir+'/'+str(index)+'.jpg', face)
index += 1
key = cv2.waitKey(30) & 0xff
if key == 27:
break
else:
print('Finished!')
break
在这里我也给出一个opencv来识别人脸的代码示例：
import cv2
import os
import sys
import random
out_dir = './my_faces'
if not os.path.exists(out_dir):
os.makedirs(out_dir)
# 改变亮度与对比度
def relight(img, alpha=1, bias=0):
w = img.shape[1]
h = img.shape[0]
#image = []
for i in range(0,w):
for j in range(0,h):
for c in range(3):
tmp = int(img[j,i,c]*alpha + bias)
if tmp > 255:
tmp = 255
elif tmp < 0:
tmp = 0
img[j,i,c] = tmp
return img
# 获取分类器
haar = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
# 打开摄像头 参数为输入流，可以为摄像头或视频文件
camera = cv2.VideoCapture(0)
n = 1
while 1:
if (n <= 10000):
print('It`s processing %s image.' % n)
# 读帧
success, img = camera.read()
gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = haar.detectMultiScale(gray_img, 1.3, 5)
for f_x, f_y, f_w, f_h in faces:
face = img[f_y:f_y+f_h, f_x:f_x+f_w]
face = cv2.resize(face, (64,64))
'''
if n % 3 == 1:
face = relight(face, 1, 50)
elif n % 3 == 2:
face = relight(face, 0.5, 0)
'''
face = relight(face, random.uniform(0.5, 1.5), random.randint(-50, 50))
cv2.imshow('img', face)
cv2.imwrite(out_dir+'/'+str(n)+'.jpg', face)
n+=1
key = cv2.waitKey(30) & 0xff
if key == 27:
break
else:
break
获取其他人脸图片集
需要收集一个其他人脸的图片集，只要不是自己的人脸都可以，可以在网上找到，这里我给出一个我用到的图片集：
网站地址:http://vis-www.cs.umass.edu/lfw/
图片集下载:http://vis-www.cs.umass.edu/lfw/lfw.tgz
先将下载的图片集，解压到项目目录下的input_img目录下，也可以自己指定目录(修改代码中的input_dir变量)
接下来使用dlib来批量识别图片中的人脸部分，并保存到指定目录下
set_other_people.py
# -*- codeing: utf-8 -*-
import sys
import os
import cv2
import dlib
input_dir = './input_img'
output_dir = './other_faces'
size = 64
if not os.path.exists(output_dir):
os.makedirs(output_dir)
#使用dlib自带的frontal_face_detector作为我们的特征提取器
detector = dlib.get_frontal_face_detector()
index = 1
for (path, dirnames, filenames) in os.walk(input_dir):
for filename in filenames:
if filename.endswith('.jpg'):
print('Being processed picture %s' % index)
img_path = path+'/'+filename
# 从文件读取图片
img = cv2.imread(img_path)
# 转为灰度图片
gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# 使用detector进行人脸检测 dets为返回的结果
dets = detector(gray_img, 1)
#使用enumerate 函数遍历序列中的元素以及它们的下标
#下标i即为人脸序号
#left：人脸左边距离图片左边界的距离 ；right：人脸右边距离图片左边界的距离
#top：人脸上边距离图片上边界的距离 ；bottom：人脸下边距离图片上边界的距离
for i, d in enumerate(dets):
x1 = d.top() if d.top() > 0 else 0
y1 = d.bottom() if d.bottom() > 0 else 0
x2 = d.left() if d.left() > 0 else 0
y2 = d.right() if d.right() > 0 else 0
# img[y:y+h,x:x+w]
face = img[x1:y1,x2:y2]
# 调整图片的尺寸
face = cv2.resize(face, (size,size))
cv2.imshow('image',face)
# 保存图片
cv2.imwrite(output_dir+'/'+str(index)+'.jpg', face)
index += 1
key = cv2.waitKey(30) & 0xff
if key == 27:
sys.exit(0)
这个项目用到的图片数是10000张左右，如果是自己下载的图片集，控制一下图片的数量避免数量不足，或图片过多带来的内存不够与运行缓慢。
lfw数据集中有13000多张全世界知名人士互联网自然场景不同朝向、表情和光照环境人脸图片，共有5749人，其中有1680人有2张或2张以上人脸图片。
训练模型
有了训练数据之后，通过cnn来训练数据，就可以让她记住我的人脸特征，学习怎么认识我了。
train_faces.py
import tensorflow as tf
import cv2
import numpy as np
import os
import random
import sys
from sklearn.model_selection import train_test_split
my_faces_path = './my_faces'
other_faces_path = './other_faces'
size = 64
imgs = []
labs = []
def getPaddingSize(img):
h, w, _ = img.shape
top, bottom, left, right = (0,0,0,0)
longest = max(h, w)
if w < longest:
tmp = longest - w
# //表示整除符号
left = tmp // 2
right = tmp - left
elif h < longest:
tmp = longest - h
top = tmp // 2
bottom = tmp - top
else:
pass
return top, bottom, left, right
def readData(path , h=size, w=size):
for filename in os.listdir(path):
if filename.endswith('.jpg'):
filename = path + '/' + filename
img = cv2.imread(filename)
top,bottom,left,right = getPaddingSize(img)
# 将图片放大， 扩充图片边缘部分
img = cv2.copyMakeBorder(img, top, bottom, left, right, cv2.BORDER_CONSTANT, value=[0,0,0])
img = cv2.resize(img, (h, w))
imgs.append(img)
labs.append(path)
readData(my_faces_path)
readData(other_faces_path)
# 将图片数据与标签转换成数组
imgs = np.array(imgs)
labs = np.array([[0,1] if lab == my_faces_path else [1,0] for lab in labs])
# 随机划分测试集与训练集
train_x,test_x,train_y,test_y = train_test_split(imgs, labs, test_size=0.05, random_state=random.randint(0,100))
# 参数：图片数据的总数，图片的高、宽、通道
train_x = train_x.reshape(train_x.shape[0], size, size, 3)
test_x = test_x.reshape(test_x.shape[0], size, size, 3)
# 将数据转换成小于1的数
train_x = train_x.astype('float32')/255.0
test_x = test_x.astype('float32')/255.0
print('train size:%s, test size:%s' % (len(train_x), len(test_x)))
# 图片块，每次取100张图片
batch_size = 100
num_batch = len(train_x) // batch_size
x = tf.placeholder(tf.float32, [None, size, size, 3])
y_ = tf.placeholder(tf.float32, [None, 2])
keep_prob_5 = tf.placeholder(tf.float32)
keep_prob_75 = tf.placeholder(tf.float32)
def weightVariable(shape):
init = tf.random_normal(shape, stddev=0.01)
return tf.Variable(init)
def biasVariable(shape):
init = tf.random_normal(shape)
return tf.Variable(init)
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1,1,1,1], padding='SAME')
def maxPool(x):
return tf.nn.max_pool(x, ksize=[1,2,2,1], strides=[1,2,2,1], padding='SAME')
def dropout(x, keep):
return tf.nn.dropout(x, keep)
def cnnLayer():
# 第一层
W1 = weightVariable([3,3,3,32]) # 卷积核大小(3,3)， 输入通道(3)， 输出通道(32)
b1 = biasVariable([32])
# 卷积
conv1 = tf.nn.relu(conv2d(x, W1) + b1)
# 池化
pool1 = maxPool(conv1)
# 减少过拟合，随机让某些权重不更新
drop1 = dropout(pool1, keep_prob_5)
# 第二层
W2 = weightVariable([3,3,32,64])
b2 = biasVariable([64])
conv2 = tf.nn.relu(conv2d(drop1, W2) + b2)
pool2 = maxPool(conv2)
drop2 = dropout(pool2, keep_prob_5)
# 第三层
W3 = weightVariable([3,3,64,64])
b3 = biasVariable([64])
conv3 = tf.nn.relu(conv2d(drop2, W3) + b3)
pool3 = maxPool(conv3)
drop3 = dropout(pool3, keep_prob_5)
# 全连接层
Wf = weightVariable([8*16*32, 512])
bf = biasVariable([512])
drop3_flat = tf.reshape(drop3, [-1, 8*16*32])
dense = tf.nn.relu(tf.matmul(drop3_flat, Wf) + bf)
dropf = dropout(dense, keep_prob_75)
# 输出层
Wout = weightVariable([512,2])
bout = weightVariable([2])
#out = tf.matmul(dropf, Wout) + bout
out = tf.add(tf.matmul(dropf, Wout), bout)
return out
def cnnTrain():
out = cnnLayer()
cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=out, labels=y_))
train_step = tf.train.AdamOptimizer(0.01).minimize(cross_entropy)
# 比较标签是否相等，再求的所有数的平均值，tf.cast(强制转换类型)
accuracy = tf.reduce_mean(tf.cast(tf.equal(tf.argmax(out, 1), tf.argmax(y_, 1)), tf.float32))
# 将loss与accuracy保存以供tensorboard使用
tf.summary.scalar('loss', cross_entropy)
tf.summary.scalar('accuracy', accuracy)
merged_summary_op = tf.summary.merge_all()
# 数据保存器的初始化
saver = tf.train.Saver()
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
summary_writer = tf.summary.FileWriter('./tmp', graph=tf.get_default_graph())
for n in range(10):
# 每次取128(batch_size)张图片
for i in range(num_batch):
batch_x = train_x[i*batch_size : (i+1)*batch_size]
batch_y = train_y[i*batch_size : (i+1)*batch_size]
# 开始训练数据，同时训练三个变量，返回三个数据
_,loss,summary = sess.run([train_step, cross_entropy, merged_summary_op],
feed_dict={x:batch_x,y_:batch_y, keep_prob_5:0.5,keep_prob_75:0.75})
summary_writer.add_summary(summary, n*num_batch+i)
# 打印损失
print(n*num_batch+i, loss)
if (n*num_batch+i) % 100 == 0:
# 获取测试数据的准确率
acc = accuracy.eval({x:test_x, y_:test_y, keep_prob_5:1.0, keep_prob_75:1.0})
print(n*num_batch+i, acc)
# 准确率大于0.98时保存并退出
if acc > 0.98 and n > 2:
saver.save(sess, './train_faces.model', global_step=n*num_batch+i)
sys.exit(0)
print('accuracy less 0.98, exited!')
cnnTrain()
训练之后的数据会保存在当前目录下。
使用模型进行识别
最后就是让她认识我了，很简单，只要运行程序，让摄像头拍到我的脸，她就可以轻松地识别出是不是我了。
is_my_face.py
output = cnnLayer()
predict = tf.argmax(output, 1)
saver = tf.train.Saver()
sess = tf.Session()
saver.restore(sess, tf.train.latest_checkpoint('.'))
def is_my_face(image):
res = sess.run(predict, feed_dict={x: [image/255.0], keep_prob_5:1.0, keep_prob_75: 1.0})
if res[0] == 1:
return True
else:
return False
#使用dlib自带的frontal_face_detector作为我们的特征提取器
detector = dlib.get_frontal_face_detector()
cam = cv2.VideoCapture(0)
while True:
_, img = cam.read()
gray_image = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
dets = detector(gray_image, 1)
if not len(dets):
#print('Can`t get face.')
cv2.imshow('img', img)
key = cv2.waitKey(30) & 0xff
if key == 27:
sys.exit(0)
for i, d in enumerate(dets):
x1 = d.top() if d.top() > 0 else 0
y1 = d.bottom() if d.bottom() > 0 else 0
x2 = d.left() if d.left() > 0 else 0
y2 = d.right() if d.right() > 0 else 0
face = img[x1:y1,x2:y2]
# 调整图片的尺寸
face = cv2.resize(face, (size,size))
print('Is this my face? %s' % is_my_face(face))
cv2.rectangle(img, (x2,x1),(y2,y1), (255,0,0),3)
cv2.imshow('image',img)
key = cv2.waitKey(30) & 0xff
if key == 27:
sys.exit(0)
sess.close() 程序在我电脑上运行时间：
1、get_my_faces.py 2：30
2、set_other_people.py 28分钟
3. train_faces.py takes 5 minutes, and the output "accuracy less 0.98, exited!" ends.