This article mainly focuses on the realization of dual recognition and authentication of face and gesture by small programs, and adds some new elements to the traditional face recognition business. The content is long, it is recommended to read it first.
1. Small program
1.1. Registering an Account and Creating a Mini Program
- Visit the WeChat public platform and click on account registration.
- Select the applet and fill in the required information in the form to register.
- In the development management, select the development settings, and copy the AppID and AppSecret for storage.
- Download and install the WeChat web developer tool and create a new project, fill in the AppId copied in the image above.
1.2, page design and style implementation
- The page needs to provide two entrances for uploading images, and an area for displaying the uploaded images, as well as an image upload button and an area for identifying results.
- WXML content
<view class="containerBox">
<view class="leftBtn" bindtap="loadImage">上传人脸库</view>
</view>
<view>
<image src="{
{reproduction}}" class="showImg"></image>
</view>
<view class="containerBox">
<view class="rightBtn" bindtap="loadImagethis">上传实时照</view>
</view>
<view class="msgTitle">温馨提示：请上传手势数字2的照片</view>
<view style="display: flex;" >
<image src="{
{imgSrc}}" class="nowImg"></image>
<view class="contentBox" wx:if="{
{isShowDetail}}">
<view class="msgTitleDetail">
<view class="resultTitle">识别分数：{
{
score}}</view>
<view class="resultTitle2">人脸相似度得分，推荐阈值80分</view>
<view class="resultTitle2" wx:if="{
{gestures==0}}">未检测到手势数字2！请重新上传！</view>
<view class="resultTitle2" wx:else>检测到手势数字2！识别通过！</view>
</view>
</view>
</view>
<view class="centerBtn" bindtap="identify">图像识别</view>
- WXSS content
.containerBox{
width:750rpx; display:flex;height:62rpx;margin-top:20rpx;
}
.leftBtn{
width:181rpx;height:62rpx;color:#4FAFF2;border:1rpx solid #4FAFF2;border-radius:10rpx;text-align: center;line-height:62rpx;font-size:28rpx;margin-left: 284.5rpx;
}
.rightBtn{
width:181rpx;height:62rpx;color:white;border:1rpx solid #4FAFF2;border-radius:10rpx;text-align: center;line-height:62rpx;font-size:28rpx; margin-left: 284.5rpx;background:#4FAFF2;
}
.centerBtn{
width:181rpx;height:62rpx;color:white;border:1rpx solid #29D124;border-radius:10rpx;text-align: center;line-height:62rpx;font-size:28rpx;margin-left: 284rpx;background:#29D124;margin-top:20rpx;
}
.showImg{
width:300rpx;height:300rpx;margin-left:225rpx;margin-top:25rpx;border-radius:50%;
}
.nowImg{
width:300rpx;height:300rpx;margin-left:20rpx; margin-top:10rpx;border-radius:50%;
}
.result{
margin-top:20rpx;
}
.resultTitle{
margin-left:25rpx;margin-top:10rpx;color:#2B79F5;font-size:25rpx;
}
.resultTitle2{
margin-left:25rpx;margin-top:10rpx;color:red;font-size:25rpx;
}
.msgTitle{
color:#0890FF;font-size:30rpx;height:30rpx;line-height:30rpx;font-weight:bold;margin-top:30rpx;text-align: center;
}
.contentBox{
width:430rpx;height:300rpx;margin-top:10rpx;}
.msgTitleDetail{
width:430rpx;height:300rpx;margin-top: 112.5rpx;
}
.productTableTr{
height: 80rpx;line-height: 80rpx;border-bottom: 5rpx solid #F8F8F8;display:flex;
}
.leftTr{
width: 583rpx;height: 80rpx;line-height: 80rpx;
}
.rightTr{
width: 119rpx;height: 80rpx;line-height: 80rpx;color: #FF2525;font-size: 26rpx;
}
.leftTrText{
color: #2B79F5;font-size: 28rpx;margin-left: 15rpx;width: 283rpx;
}
.productDetailTable{
width: 702rpx;margin-left: 24rpx;border:5rpx solid #F8F8F8;border-radius: 6rpx;
}
.copyBtn{
color:white;background:#2B79F5;border-radius:8rpx;width:100rpx;height:50rpx;margin-top:15rpx;
}
- The content of WXML and WXSS is relatively simple. In JS, we need to implement the corresponding function to display the selected picture on the interface.
wx.chooseImage({
count: 0,
sizeType: ['original', 'compressed'], //原图 / 压缩
sourceType: ['album', 'camera'], //相册 / 相机拍照模式
success(res) {
that.setData({
reproduction: res.tempFilePaths[0]
});
}
})
1.3. Interface parameter splicing
- After uploading the face database and uploading the image selection of the real-time photo, we need to convert the selected image to Base64 format.
wx.getFileSystemManager().readFile({
filePath: res.tempFilePaths[0],
encoding: 'base64',
success(data) {
let baseData = data.data; //'data:image/png;base64,' + data.data;
that.setData({
baseData: baseData
});
}
});
- After completing the above steps, you can see this step on the console as a success.
2. API
2.1. Registering an account and creating an application
Note: Face recognition and human body analysis are two separate applications, and you need to create and obtain the corresponding Key and secret respectively!!!
- Visit Baidu Open Platform to select face recognition, human body detection and receive free resources.
- Create an app with all the information you need to fill out the form.
- After the creation is complete, go back to the application list and copy the API Key and Serect Key. Later, we need to obtain the Token through these credentials.
2.2. Create an API project
- Open Visual Studio and select Create New Project on the right.
- Enter ASP.NET Web in the search box, then select the Net Framework project and click Next.
- Fill in the corresponding project name and select the storage path.
- Because we just need to build the API, select the WebApi project template and click Create.
- After the API is successfully created, the following figure is shown.
2.3. Implement the Get Token interface
- How to get Access Token. As you can see in the figure below, to get the Token is to send a request to the authorization service address and bring the required parameters in the URL. Those two parameters have been copied when the application was created, which can come in handy here.
- Go back to VS, select the Models folder of the solution on the right, click Add, and select Class.
- We now need to encapsulate a method for requesting the service authorization address.
/// <summary>
/// 模拟Get请求
/// </summary>
/// <param name="url"></param>
/// <returns></returns>
public static string HttpGet(string url)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.Method = "GET";
request.ContentType = "application/json";
request.Accept = "*/*";
request.Timeout = 15000;
request.AllowAutoRedirect = false;
WebResponse response = null;
string responseStr = null;
try
{
response = request.GetResponse();
if (response != null)
{
StreamReader reader = new StreamReader(response.GetResponseStream(), Encoding.UTF8);
responseStr = reader.ReadToEnd();
reader.Close();
}
}
catch (Exception)
{
throw;
}
finally
{
request = null;
response = null;
}
return responseStr;
}
- Create an entity class to receive according to the return parameters of the interface to obtain the Token. In this, we mainly use access_token and expires_in, one is the token value, and the other is the expiration time.
/// <summary>
/// 解析Token帮助类
/// </summary>
public class TokenClass {
public string refresh_token {
get; set; }
public string session_key {
get; set; }
public string scope {
get; set; }
public string session_secret {
get; set; }
/// <summary>
/// Access Token的有效期(秒为单位，有效期30天)；
/// </summary>
public int expires_in {
get; set; }
/// <summary>
/// 获取的Access Token
/// </summary>
public string access_token {
get; set; }
}
- Concatenate client_id and client_secret into a method, and pass the two fields as parameters.
/// <summary>
/// 获取token
/// </summary>
/// <returns></returns>
public static TokenClass GetToken(string client_id,string client_secret) {
var grant_type = "client_credentials";
//拼接参数到地址
string tokenUrl = "https://aip.baidubce.com/oauth/2.0/token?grant_type=" + grant_type + "&client_id=" + client_id + "&client_secret=" + client_secret;
string resultStr = RequestHelper.HttpGet(tokenUrl);
if (string.IsNullOrWhiteSpace(resultStr))
{
//返回false
}
TokenClass info = Newtonsoft.Json.JsonConvert.DeserializeObject<TokenClass>(resultStr);
return info;
}
- Go to the default Index action method of the controller and call the method to obtain the token to see if it can be obtained.
public ActionResult Index()
{
ViewBag.Title = "Home Page";
TokenClass info = GetToken("你申请的应用Key", "你申请的应用的Sercet");
return View();
}
- Start the project, hit a breakpoint, and you can get it as shown below.
2.4. Interface data processing
- After the token can be obtained normally, let's take a look at the interface request and callback data of face recognition comparison. Don't be frightened by his long introduction, we just need to splice the corresponding parameters on the URL and convert the image to Base and carry it together.
- Go back to the RequestHelper entity class of VS and define a method to obtain the result of face recognition matching.
- Then go to the controller, implement an action method, receive two image parameters in base64 format passed from the applet, and then call the above method.
/// <summary>
/// 人脸相似度对比及手势识别
/// </summary>
/// <param name="baseSence1"></param>
/// <param name="baseSence12"></param>
/// <returns></returns>
public ActionResult GetImageBase(string img1, string img2)
{
var faceResult = FaceData(img1, img2);
return View();
}
- Go back to the applet, implement a function in the button of image recognition, and request the action method created above.
let that = this;
let data = {
img1: that.data.baseData,img2: that.data.baseDatathis};
wx.request({
url: 'https://localhost:44312/Home/GetImageBase',
method: 'POST',
data: {
img1: that.data.baseData,
img2: that.data.baseDatathis
},
success: function (identify) {
that.setData({
isShowDetail: true,
score: identify.data.score,
gestures: identify.data.gestures,
});
}
})
- Set a breakpoint in the FaceData method of RequestHelper and start the project.
- Click the Image Recognition button in the applet and view the content of the breakpoint. As shown in the figure below, it is the data returned by the request face recognition interface.
- Copy the data out, perform JSON escaping, and then define the corresponding entity class according to the data type of the field to convert and receive the content returned by the interface. As shown in the figure below, what we mainly use here is the score value in the result field, which is the face similarity score.
- Three entity classes are defined according to the data structure, namely FaceClass, scoreInfo, and face, where face and scoreInfo have a one-to-many relationship. In this way, the returned data can be used directly.
/// <summary>
/// 人脸识别返回参数帮助类
/// </summary>
public class FaceClass {
public int error_code {
get; set; }
public string error_msg {
get; set; }
public Int64 log_id {
get; set; }
public int timestamp {
get; set; }
public int cached {
get; set; }
public scoreInfo result {
get; set; }
}
public class scoreInfo
{
/// <summary>
/// 分数
/// </summary>
public decimal score {
get; set; }
public List<face> face_list {
get; set; }
}
public class face {
public string face_token {
get; set; }
}
- After the face recognition is done, the next step is the picture gesture recognition. Define another method in RequestHelper, where only one parameter needs to be defined, because the face recognition comparison is to compare two pictures, and the gesture recognition only needs to be obtained from the real picture.
Note: Another application key and sercet need to be used when obtaining image gesture recognition!!!
- Then call it again in the action method of the controller just now, hit a breakpoint to see the execution result. As shown in the figure below, we copy the result of the result and define the entity to receive it. The value of the classname field in the second array is Two, that is, it is detected that there is gesture two in the picture.
/// <summary>
/// 手势识别返回参数帮助类
/// </summary>
public class GesturesClass {
public Int64 log_id {
get; set; }
public int result_num {
get; set; }
public List<gestures> result {
get; set; }
}
public class gestures {
/// <summary>
/// 手势名称 [1]
/// </summary>
public string classname {
get; set; }
public int height {
get; set; }
public int left {
get; set; }
public decimal probability {
get; set; }
public int top {
get; set; }
public int width {
get; set; }
}
- Write the corresponding judgment on the returned data in the controller, and return different data if the picture contains the gesture of number two. Of course, the ability of the interface is not limited to only recognizing the number two. I just pick one for demonstration here, and interested friends can make the configuration later.
/// <summary>
/// 人脸相似度对比及手势识别
/// </summary>
/// <param name="baseSence1"></param>
/// <param name="baseSence12"></param>
/// <returns></returns>
public ActionResult GetImageBase(string img1, string img2)
{
var faceResult = FaceData(img1, img2);
var gesturesResult = GesturesData(img2);
var score = faceResult.result.score;//人脸相似度分数
var gestures = 0;
if (gesturesResult.result.Count() > 1)
{
if (gesturesResult.result[1].classname == "Two")
{
gestures = 1;
}
}
JsonData info = new JsonData();
info.score = score;
info.gestures = gestures;
return Json(info);
}
2.4, page data rendering
- Looking at the data returned by the interface, you can see that the score field and gestures are returned.
- Store the returned fields in the global data, make corresponding logical judgments on the page, and finally run, you can get a face recognition and gesture dual recognition authentication project.
You can design different effect displays according to your own business scenarios. If you don’t understand, you can send a private message or leave a message below, let us learn together and make progress together!