1. Build the insightface environment

In order to avoid conflicts with other python versions in the server, conda is used here to create the environment. During the installation of insightface, relevant dependencies will be installed, and you can wait for the system installation to complete.

conda create -n insightface_chw python=3.6
conda activate insightface_chw
pip install -U insightface 
pip install -U insightface #这个命令执行两遍，我只执行一遍发现并没有安装上，只是安装了一些依赖
pip install onnxruntime-gpu
pip install albumentations

2. Download the insightface project

git clone https://github.com/deepinsight/insightface

3. Face Detection

cd ./insightface/python-package

Then run the following script

import cv2
import numpy as np
import insightface
from insightface.app import FaceAnalysis
from insightface.data import get_image as ins_get_image

app = FaceAnalysis(allowed_modules=['detection'],providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
app.prepare(ctx_id=0, det_size=(640, 640))
img = ins_get_image('ldh')  #不用带后缀，图片放到./insightface/python-package/insightface/data/images
faces = app.get(img)
print("faces::::", faces)
rimg = app.draw_on(img, faces)
cv2.imwrite("./ldh_output.jpg", rimg)

The print result of faces is as follows

faces:::: [{'bbox': array([250.4125 , 140.93515, 439.2733 , 385.86316], dtype=float32), 'kps': array([[306.19827, 253.13426],
       [393.27582, 238.8169 ],
       [360.847  , 305.88684],
       [332.69775, 343.96198],
       [394.14444, 331.2613 ]], dtype=float32), 'det_score': 0.85925925}]

The detected image is a rectangular frame plus 5 key points, as shown in the following figure

3.1 报错1 ImportError: cannot import name 'mesh_core_cython'

from .cython import mesh_core_cython
ImportError: cannot import name 'mesh_core_cython'

The above error I thought was to pip install cython, but it still didn't work after the installation, so I searched the Internet.

I used the above command and found that it still didn't work, so I searched on the computer.

find  / -iname "cython"

Found this in the following directory

./insightface/python-package/insightface/thirdparty/face3d/mesh/cython

Then I went to this directory to have a look and found that there was a setup.py, so I executed

python3 setup.py build_ext -i

Error disappears.

4. Extract facial features

4.1 Model download

https://github.com/deepinsight/insightface/tree/master/model_zoo

Go to the above website to download the onnx model of face recognition, here I downloaded it and renamed it to recg.onnx

4.2 Extracting facial features

import cv2
import numpy as np
import insightface
from insightface.app import FaceAnalysis
from insightface.data import get_image as ins_get_image



app = FaceAnalysis(allowed_modules=['detection'],providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
app.prepare(ctx_id=0, det_size=(640, 640))
img = ins_get_image('ldh')
faces = app.get(img)
print("faces::::", faces)
rimg = app.draw_on(img, faces)
cv2.imwrite("./ldh_output.jpg", rimg)



handler = insightface.model_zoo.get_model('recg.onnx', providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
handler.prepare(ctx_id=0)
img = ins_get_image('ldh')
feature = handler.get(img, faces[0])
print("size of feature:", len(feature))
print("feature:", feature)

In fact, when extracting features, what is passed to the model is the coordinate information of 5 key points of the face, we can take a look at the get function

    def get(self, img, face):
        aimg = face_align.norm_crop(img, landmark=face.kps)
        face.embedding = self.get_feat(aimg).flatten()
        return face.embedding

You can see that landmark=face.kps is used here.

The printed facial features are as follows:

size of feature: 512
feature: [-9.25378799e-01  1.12189925e+00 -3.77961069e-01  1.53634763e+00
  8.14318538e-01 -3.07922035e-01 -1.32581890e+00 -1.20144689e+00
  5.94791770e-03  6.71953619e-01  1.42627060e+00  2.47668549e-01
 -5.13059676e-01  4.16025162e-01 -4.16867733e-01  3.29927027e-01
 -1.76791012e-01 -1.33843150e-03  2.62192875e-01 -7.11937249e-01
 -5.82276583e-01  3.84117991e-01 -4.01041389e-01 -2.82063276e-01
  6.92473352e-01 -2.66507506e-01  2.34112352e-01 -2.18238997e+00
  4.89850283e-01  1.45217252e+00 -1.49105823e+00 -9.29834545e-01
 -2.84832150e-01 -1.42106509e+00 -7.75823414e-01 -9.36572075e-01
  1.22991014e+00  2.59431660e-01  6.32830501e-01  7.25324899e-02
 -4.30947453e-01  3.29869479e-01 -1.40622067e+00  1.75537634e+00
 -2.96340764e-01  1.03546596e+00  4.67442483e-01  1.15163600e+00
 -1.17070563e-01  1.84685513e-01  4.09658134e-01  1.16948152e+00
  9.47982669e-01  1.51446068e+00 -1.94126654e+00 -1.99912921e-01
 -6.44741535e-01  3.84710550e-01  8.06774318e-01  3.03845316e-01
  7.04601586e-01  2.98136622e-01  9.10332084e-01  2.43957829e+00
 -5.86433113e-01 -7.64575064e-01 -1.51625574e+00 -4.70794886e-01
  6.51128173e-01 -1.84837079e+00 -1.00911522e+00  2.16325343e-01
  5.47209024e-01 -6.96201026e-01 -9.97821510e-01  1.13421094e+00
 -4.29270975e-02 -1.99787751e-01  1.08260703e+00  9.74589705e-01
  1.10760403e+00  8.01510334e-01  1.13468552e+00  6.87273800e-01
  1.33836412e+00 -2.80151349e-02  1.25184190e+00 -9.93042111e-01
  1.61516750e+00  1.12915814e+00  8.32992673e-01  1.05787609e-02
 -1.20558095e+00 -1.04558206e+00 -1.72734368e+00 -1.60695121e-01
  8.74173582e-01  4.30314064e-01  3.90443951e-01  3.68365377e-01
 -1.47074330e+00  8.36366653e-01  3.20074102e-03  3.37526709e-01
  1.69285023e+00  1.71148300e+00  1.21127784e+00 -2.46544257e-01
  1.83451712e+00  1.20001090e+00 -1.51531231e+00  4.33688462e-01
 -2.19699001e+00  1.87094784e+00 -1.63829291e+00  1.27979255e+00
  6.68724775e-01 -1.17384040e+00 -9.53805745e-01  6.37416422e-01
  1.25283241e+00 -4.72882658e-01  7.00635672e-01  8.35622549e-01
 -2.15689683e+00 -1.03445494e+00 -2.39766431e+00  1.23635948e+00
  1.42301047e+00 -3.00289780e-01  1.41190976e-01 -4.94515747e-01
  4.81027424e-01 -4.16015983e-01  7.81331480e-01 -1.43692136e+00
 -1.42533112e+00 -1.55742049e+00 -1.27529824e+00  3.73526305e-01
  1.50604856e+00 -1.05595541e+00  1.91644537e+00 -9.77402151e-01
  7.61697665e-02  6.85342073e-01  6.28700614e-01  2.59056002e-01
  8.53248060e-01  9.18204188e-01  8.36220145e-01  2.32384753e+00
 -1.15582526e+00  9.21581209e-01  3.82296853e-02 -1.62712407e+00
  1.31698740e+00 -8.42826188e-01  3.79095599e-03 -5.89607954e-01
  2.12002158e+00 -8.21184874e-01 -3.50165218e-01  1.80107331e+00
  3.67856652e-01  2.78761655e-01 -6.75524492e-03  1.61253417e+00
  2.72980243e-01 -1.45269975e-01  1.33122730e+00  1.67430460e+00
  1.65378183e-01  4.20516253e-01  1.98410237e+00 -2.59076536e-01
  1.04198182e+00 -8.46942216e-02  1.76673305e+00 -8.25684845e-01
  5.74173927e-01  4.86998826e-01  7.77795538e-02  2.13724747e-01
  8.04883659e-01  1.32689762e+00  1.00689387e+00  5.04235148e-01
 -2.08123755e+00  1.74718440e-01 -2.78054941e-02 -2.73483157e+00
 -1.31054863e-01 -1.43446589e+00 -7.20487952e-01  1.12745309e+00
 -1.85685202e-01 -2.09606528e-01  4.63031679e-01  3.72038960e-01
  8.02597225e-01  1.32468903e+00  1.46707821e+00 -5.66261053e-01
 -6.65133119e-01  7.99282640e-02  1.86339438e-01 -5.10585666e-01
  7.00619340e-01  1.47186112e+00 -5.60413182e-01 -5.06068051e-01
 -8.67577553e-01 -1.46331966e+00  5.52830219e-01  3.76271665e-01
  7.95127332e-01 -9.48988020e-01  3.67837697e-01 -2.59218597e+00
  2.93103248e-01  1.77113688e+00 -2.08559811e-01 -2.46169224e-01
  1.23183763e+00  1.68317008e+00 -8.76623929e-01 -2.16812968e+00
 -2.86590476e-02  1.95987976e+00  3.57534051e-01  1.99777931e-01
  8.41746032e-01  1.39935777e-01  1.24161732e+00  1.86693296e-01
  6.47045791e-01  3.79344225e-01 -5.91819167e-01  2.10163534e-01
 -7.84599423e-01  1.44664180e+00  2.21032172e-01  2.82676697e-01
  8.90868306e-01 -1.18004346e+00 -9.65086401e-01  2.37004685e+00
  8.79030675e-02  7.38855898e-01 -1.39655483e+00 -1.16303825e+00
 -1.36244881e+00  3.79605591e-01 -4.81240660e-01 -6.03446811e-02
  1.98825032e-01 -1.40159500e+00 -5.66048771e-02 -1.92057371e-01
 -8.93036604e-01 -2.61887521e-01  6.14840627e-01  6.13342285e-01
 -7.84360528e-01  1.60332644e+00 -3.23775421e-05 -1.68999226e-03
  1.33810937e+00  2.59712756e-01  3.22760701e-01 -4.16172713e-01
  7.11134851e-01  3.86906594e-01 -6.64857864e-01  6.66272283e-01
  1.12394595e+00 -1.36855289e-01 -1.07242525e+00  6.40258074e-01
  7.83331275e-01  2.43918672e-01  5.39867282e-01  2.35217690e-01
  1.18415987e+00 -8.14915299e-01 -2.00127697e+00 -5.36369801e-01
 -4.38131094e-01 -7.53244638e-01  2.13781881e+00 -4.34540629e-01
 -5.81116557e-01 -7.82125771e-01 -6.66840792e-01  1.29590559e+00
 -1.11483300e+00 -1.77420366e+00  3.60799551e-01  1.30988276e+00
 -1.01142776e+00 -9.88633335e-01  2.20396972e+00  4.69280809e-01
  4.59513694e-01 -5.11144161e-01 -1.48858309e-01  5.86034775e-01
 -6.28118455e-01  1.56790364e+00  1.01874316e+00  6.80294931e-01
 -2.48470351e-01 -1.66888607e+00  4.79499936e-01 -9.12623763e-01
  9.35893834e-01 -1.77669048e-01 -1.21470296e+00  2.18366943e-02
  5.45701265e-01  5.46345890e-01  7.42697775e-01  5.29096007e-01
 -1.43474445e-01  1.11457312e+00  3.43216985e-01 -1.74324691e+00
 -4.60702538e-01  5.07811189e-01  1.45009649e+00  3.08589965e-01
 -8.39812815e-01 -1.03414707e-01  6.87325299e-01 -9.23633993e-01
  1.02700031e+00 -5.05108953e-01  1.63558245e-01 -1.67413473e-01
  2.83512324e-01  1.63193595e+00  1.28096566e-01  9.11045551e-01
 -2.57245779e-01  2.25007877e-01 -8.15706328e-03 -7.95329630e-01
  1.59858331e-01  2.67190486e-02 -1.19778216e-01 -1.17415237e+00
  1.74254048e+00 -6.71473622e-01 -1.40046167e+00 -9.61800694e-01
 -5.86933792e-01  6.33914471e-01 -1.58795440e+00  6.98880374e-01
 -9.28590521e-02  9.35192764e-01 -2.28557897e+00 -4.42225575e-01
 -4.72230911e-01 -9.34598207e-01 -4.45788801e-01 -4.51316953e-01
  5.86329758e-01  1.59920883e+00  1.89197445e+00 -4.46383357e-02
 -2.56239414e+00  1.62363434e+00  7.62188077e-01 -7.09390998e-01
 -1.68505698e-01  4.47826356e-01 -8.35269153e-01 -1.03524947e+00
 -9.70352292e-01 -2.40768266e+00  4.71594214e-01 -4.55471456e-01
  9.01260823e-02  7.63986826e-01  5.70428908e-01  7.63611615e-01
  6.84195161e-01 -7.09837198e-01 -1.11740328e-01 -1.13425088e+00
 -1.81686699e-01 -2.69015819e-01 -1.33226430e+00 -2.11236715e+00
  4.21511620e-01 -1.10060155e+00  8.07265043e-01  2.46378160e+00
 -4.99998361e-01 -8.70806098e-01  6.66378617e-01  9.94343683e-02
  9.62515712e-01  1.48241317e+00  2.60850012e-01 -2.82376379e-01
 -1.89661279e-01  2.66471446e-01  2.78322458e-01 -3.23252797e-01
  4.71474715e-02 -4.39821482e-02 -4.69594598e-01 -8.14322233e-01
  8.25921059e-01 -5.86541533e-01 -3.85801554e-01 -2.12531281e+00
  1.31300259e+00 -8.73632789e-01  5.16620517e-01  1.24318704e-01
 -2.04154804e-01 -4.70053017e-01 -1.50033563e-01  2.49002595e-02
  6.47480547e-01  5.42859018e-01 -7.84126759e-01  4.55238432e-01
  2.74345040e+00 -1.86407435e+00 -4.04266864e-01  7.85036743e-01
 -1.60567272e+00 -1.51229072e+00 -1.12004364e+00  2.66206479e+00
 -1.69381893e+00  1.96310788e-01  1.88832358e-01  1.10981144e-01
 -1.83739674e+00  1.21344543e+00 -1.30652237e+00  8.02911043e-01
  1.53382719e+00 -6.55009747e-01  2.13313389e+00  7.01124787e-01
  1.98182344e-01  6.58964694e-01 -2.49153420e-01 -1.13354373e+00
 -1.27103531e+00 -1.12395477e+00  1.48175859e+00 -4.80957925e-01
 -4.48873490e-01 -6.31605089e-01 -3.88538480e-01 -4.72628146e-01
  3.22314769e-01  3.16851974e-01 -1.38024521e+00  9.25058663e-01
 -1.72588539e+00  6.76430106e-01  1.49050048e-02  1.11245954e+00
  8.60826522e-02  7.78218150e-01  5.93334317e-01 -5.20033360e-01
 -3.65191139e-02 -7.97780931e-01 -5.21533564e-02  1.38868642e+00
  5.85699737e-01  1.36364901e+00 -6.18628740e-01 -4.41853821e-01
 -4.93071862e-02 -2.03231573e+00 -8.60165358e-01 -5.70300102e-01
 -1.39265645e+00 -4.70582008e-01  1.58857667e+00 -1.52108145e+00
  1.51820570e-01 -4.00371879e-01  7.94574440e-01  4.33286577e-01
  8.61523628e-01  9.14571762e-01 -5.73427439e-01  4.71240193e-01
  1.11811012e-01  4.66803044e-01 -3.12886178e-01  4.88459438e-01
 -2.12693143e+00  1.74044490e+00  4.55120429e-02  1.72699082e+00
 -2.68102176e-02 -1.43795335e+00 -3.09069902e-01  9.54877675e-01]

references:

https://github.com/deepinsight/insightface

https://github.com/deepinsight/insightface/tree/master/python-package

Face detection and face recognition using insightface