The following is moved from GitHub. Many hyperlinks are relative paths, so they cannot be clicked, which is a normal phenomenon. Click to view the original document . Please indicate the source.
The original author has not responded to my submission for the time being, here is a synchronous submission to CSDN, click to view the project source code
Face recognition with Pytorch
Click here to return to the English document
Translator's note:
This project facenet-pytorch is a very convenient face recognition library, which can be installed directly through pip .
The library contains two important functions
- Face detection: using MTCNN algorithm
- Face recognition: using the FaceNet algorithm
With this library, face detection and face vector mapping operations can be easily implemented.
In order to facilitate Chinese developers to study face recognition related tasks and contribute code, I translated the README file of this project and the
examplesnecessary parts of several sample scripts located in it into Chinese for reference.Kudos to all contributors to this project.
English to Chinese translation: Brother Yuan is very happy
Translator’s Note:
This project facenet-pytorch is a very convenient face recognition library that can be installed directly via pip.
The library contains two important features:
- Face detection: using the MTCNN algorithm
- Face recognition: using the FaceNet algorithm
With this library, one can easily carry out face detection and face vector mapping operations.
In order to facilitate Chinese developers in studying face recognition and contributing code, I have translated the README file of this project and some necessary parts of several example scripts located in the
examplesdirectory into Chinese.Salute to all contributors to this project.
Translated from English to Chinese by Yuan Ge Ting Le .
This is a repository of Inception Resnet (V1) models in pytorch, pretrained on VGGFace2 and CASIA-Webface.
The Pytorch model weights are initialized with parameters ported from David Sandberg's tensorflow Facenet repo .
Also included in this repository is an efficient pytorch implementation of MTCNN for face detection prior to inference. These models are also pretrained. As far as we know, this is the fastest implementation of MTCNN.
Table of contents
- Table of contents
- Quick Start
- pre-trained model
- sample notebook
- run with docker
- Use this repository in your own git projects
- Tensorflow to Pytorch parameter conversion
- References
Quick Start
- Install:
# 使用pip安装：
pip install facenet-pytorch
# 或克隆此存储库，删除“-”以允许 python 导入：
git clone https://github.com/timesler/facenet-pytorch.git facenet_pytorch
# 或使用 docker 容器（参见 https://github.com/timesler/docker-jupyter-dl-gpu）：
docker run -it --rm timesler/jupyter-dl-gpu pip install facenet-pytorch && ipython
- In python, import facenet-pytorch and instantiate the model:
from facenet_pytorch import MTCNN, InceptionResnetV1
# 如果需要，使用 MTCNN 创建人脸检测模型：
mtcnn = MTCNN(image_size=<image_size>, margin=<margin>)
# 创建一个 inception resnet（在 eval 模式下）：
resnet = InceptionResnetV1(pretrained='vggface2').eval()
- Process the image:
from PIL import Image
img = Image.open(<image path>)
# 获取裁剪和预白化的图像张量
img_cropped = mtcnn(img, save_path=<optional save path>)
# 计算嵌入（解压缩以添加批量维度）
img_embedding = resnet(img_cropped.unsqueeze(0))
# 或者，如果用于 VGGFace2 分类
resnet.classify = True
img_probs = resnet(img_cropped.unsqueeze(0))
help(MTCNN)See and for usage and implementation details help(InceptionResnetV1).
pre-trained model
See: models/inception_resnet_v1.py
The following models have been ported to pytorch (links to download pytorch state_dict are included):
| model name | LFW Accuracy ( as listed here) | training dataset |
|---|---|---|
| 20180408-102900 (111MB) | 0.9905 | CASIA-Webface |
| 20180402-114759 (107MB) | 0.9965 | VGGFace2 |
There is no need to manually download pretrained state_dicts; they are automatically downloaded when the model is instantiated and cached in the torch cache for future use. To use the Inception Resnet (V1) model for face recognition/recognition in pytorch, use:
from facenet_pytorch import InceptionResnetV1
# 对于在 VGGFace2 上预训练的模型
model = InceptionResnetV1(pretrained='vggface2').eval()
# 对于在 CASIA-Webface 上预训练的模型
model = InceptionResnetV1(pretrained='casia-webface').eval()
# 对于具有 100 个类的未经训练的模型
model = InceptionResnetV1(num_classes=100).eval()
# 对于未经训练的 1001 类分类器
model = InceptionResnetV1(classify=True, num_classes=1001).eval()
Both pretrained models were trained on 160x160 pixel images, so they work best when applied to images resized to that shape. For best results, the image should also be cropped to the face using MTCNN (see below).
By default, the above model will return a 512-dimensional image embedding. To enable classification, classify=Truepass to the model constructor, or you can subsequently model.classify = Trueset object properties with . For VGGFace2, the pre-trained model will output a logit vector of length 8631, and for CASIA-Webface, it will output a logit vector of length 10575.
sample notebook
Complete detection and identification process
Face recognition can be easily applied to raw images by first using MTCNN to detect faces and then using an Inception Resnet model to compute embeddings or probabilities. The example code in examples/infer_cn.ipynb provides a complete example pipeline utilizing datasets, data loaders, and optional GPU processing.
Face Tracking in Video Streaming
MTCNN can be used to build a face tracking system (using MTCNN.detect()method). A complete face tracking example can be found in examples/face_tracking_cn.ipynb .
Fine-tune a pretrained model with new data
In most cases, the best way to achieve face recognition is to directly use a pre-trained model to determine the identity of a face through a clustering algorithm or a simple distance metric. However, if fine-tuning is required (i.e. if you want to choose identities based on the model's output logits), examples can be found in examples/finetune_cn.ipynb .
MTCNN guide in facenet-pytorch
This guide demonstrates the functionality of the MTCNN module. Topics covered are:
- basic usage
- image normalization
- face margin
- Multiple faces in a single image
- Batch detection
- Bounding boxes and face landmarks
- Save the face dataset
See kaggle notebook .
Performance comparison of face detection packages
This notebook demonstrates the use of three face detection packages:
- facenet-pytorch
- mtcnn
- dlib
Each package was tested for its speed at detecting faces in a set of 300 images (from all frames of a video) with GPU support enabled. Performance based on Kaggle's P100 notebook kernel. The results are summarized below.
| combo | FPS (1080x1920) | FPS (720x1280) | FPS (540x960) |
|---|---|---|---|
| facenet-pytorch | 12.97 | 20.32 | 25.50 |
| facenet-pytorch (non-batch) | 9.75 | 14.81 | 19.68 |
| dlib | 3.80 | 8.39 | 14.53 |
| mtcnn | 3.04 | 5.70 | 8.23 |
See kaggle notebook .
FastMTCNN Algorithm
This algorithm demonstrates how to achieve extremely efficient face detection, especially in videos, by exploiting the similarity between adjacent frames.
See [kaggle notebook](https://www.kaggle.com/timesler/fast-mtcnn-detector-55-fps-at-full-resolution).
run with docker
The package and any example notebooks can be run using docker (or nvidia-docker):
docker run --rm -p 8888:8888
-v ./facenet-pytorch:/home/jovyan timesler/jupyter-dl-gpu \
-v <path to data>:/home/jovyan/data
pip install facenet-pytorch && jupyter lab
Navigate to the example/ directory and run any ipython notebook.
See timesler/jupyter-dl-gpu for details on docker containers .
Use this repository in your own git projects
To use this code in your own git repository, I recommend first adding this repository as a submodule. Note that the dash ("-") in the repository name should be removed when cloning as a submodule, as it breaks python on import:
git submodule add https://github.com/timesler/facenet-pytorch.git facenet_pytorch
Alternatively, the code can be installed as a package using pip:
pip install facenet-pytorch
Tensorflow to Pytorch parameter conversion
See: models/utils/tensorflow2pytorch.py
Note that this functionality is not required to use models from this repository, which only relies on pytorch-saved ones state_dict.
After instantiating the pytorch model, the weights of each layer are loaded from the equivalent layer in the pretrained Tensorflow model from davidsandberg/facenet .
The equivalence of the output of the original Tensorflow model and the pytorch ported model has been tested and is the same:
>>> compare_model_outputs(mdl, sess, torch.randn(5, 160, 160, 3).detach())
Passing test data through TF model （通过TF模型传递测试数据）
tensor([[-0.0142, 0.0615, 0.0057, ..., 0.0497, 0.0375, -0.0838],
[-0.0139, 0.0611, 0.0054, ..., 0.0472, 0.0343, -0.0850],
[-0.0238, 0.0619, 0.0124, ..., 0.0598, 0.0334, -0.0852],
[-0.0089, 0.0548, 0.0032, ..., 0.0506, 0.0337, -0.0881],
[-0.0173, 0.0630, -0.0042, ..., 0.0487, 0.0295, -0.0791]])
Passing test data through PT model （通过PT模型传递测试数据）
tensor([[-0.0142, 0.0615, 0.0057, ..., 0.0497, 0.0375, -0.0838],
[-0.0139, 0.0611, 0.0054, ..., 0.0472, 0.0343, -0.0850],
[-0.0238, 0.0619, 0.0124, ..., 0.0598, 0.0334, -0.0852],
[-0.0089, 0.0548, 0.0032, ..., 0.0506, 0.0337, -0.0881],
[-0.0173, 0.0630, -0.0042, ..., 0.0487, 0.0295, -0.0791]],
grad_fn=<DivBackward0>)
Distance 1.2874517096861382e-06 （距离1.2874517096861382e-06）
In order to re-run the conversion of Tensorflow parameters to pytorch models, make sure to clone this repository with submodules, since the davidsandberg/facenet repository is included as a submodule and part of it is required for the conversion.
References
-
David Sandberg’s facenet repo: https://github.com/davidsandberg/facenet
-
F. Schroff, D. Kalenichenko, J. Philbin. FaceNet: A Unified Embedding for Face Recognition and Clustering, arXiv:1503.03832, 2015. PDF
-
Q. Cao, L. Shen, W. Xie, O. M. Parkhi, A. Zisserman. VGGFace2: A dataset for recognising face across pose and age, International Conference on Automatic Face and Gesture Recognition, 2018. PDF
-
D. Yi, Z. Lei, S. Liao and S. Z. Li. CASIAWebface: Learning Face Representation from Scratch, arXiv:1411.7923, 2014. PDF
-
K. Zhang, Z. Zhang, Z. Li and Y. Qiao. Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks, IEEE Signal Processing Letters, 2016. PDF