Recently, at a forum exchange meeting, a guest mentioned that the WeChat mini-program mall that he has been operating for many years often receives feedback from users: when he finds good-looking clothes worn by others while shopping, it is difficult to locate specific products through keywords. It would be great to be able to take photos and locate related products. I wonder if such a function can be implemented in the current mini program. As a software developer, I have a similar experience with daily online shopping. If the product search function can be integrated into the mini program, the user experience can be greatly improved. The guest’s question aroused my great interest.
During the research process, it was discovered that the image search product of Tencent Cloud Image Analysis can intelligently identify the subject of the product in the image based on the input image, search for the same or similar product images in the self-built image library, and give a similarity score. If the image entered for retrieval contains clothing products, the category, color and other characteristic attributes of tops, bottoms, skirts, shoes, bags, accessories and other clothing can be intelligently identified to realize image search in e-commerce scenarios.
Next, I will share in detail how I implement product search in the mini program.
1. Preparation work
1.1 Clear goals
In the mini program, you can locate similar product images by inputting product images, similar to the following:
1.2 Understanding image search
Before starting to use it, I still need to have a more detailed understanding of the product I am about to use.
Official website document introduction :
1.3 Activate image analysis service
Next, you can activate the image analysis service on the Tencent Cloud official website according to the guidelines of the official website documents:
After the service is activated, free resource packages will be issued 10,000 times each for creating pictures and retrieving pictures. You can check the usage on the resource package management page:
2. Development process
2.1 Obtain personal key
Visit the management page on the Tencent Cloud official website and create a new personal key:
2.2 Online debugging
The following is a simple way to implement the function of Polaroid through online debugging.
(1) Gallery type selection
First, look at the documentation for image search , and we select the service type of product image search .
(2) Create a gallery
Create a picture library and search for specified product images. Tencent Cloud official website provides an online API Explorer tool to facilitate our visual calls.
(3) Picture storage
Create a picture, store the product picture into the specified gallery, and the return value contains subject location information.
(4) Product search
Retrieve pictures, specify product pictures, search in the gallery, and obtain similar or identical product pictures. The returned results include product IDs similar to the input pictures.
This basically demonstrates the product search process. For use, the input parameters can be fine-tuned to meet more requirements.
2.3 Call using SDK
(1) Document introduction
For formal access, you need to integrate the SDK provided on Tencent Cloud's official website. At the bottom of the document, SDKs in multiple languages are provided, which can be accessed according to the language you are familiar with.
(2) Example
Take the go language as an example to demonstrate how to use it.
Step 1: Install basic packages:
go get -v -u github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common
Install image analysis dependencies:
go get -v -u github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/tiia
Step 2: Implement the call, taking creating a gallery as an example.
credential := common.NewCredential(***, ***)
clientProfile := profile.NewClientProfile()
clientProfile.HttpProfile.Endpoint = "tiia.tencentcloudapi.com"
cli, err := tiia.NewClient(credential, regions.Guangzhou, clientProfile)
if err != nil {
panic(err)
}
//实例化一个请求结构
req := tiia.NewCreateGroupRequest()
//对请求参数进行赋值
req.GroupId = common.StringPtr("groupId")
req.GroupName = common.StringPtr("groupName")
req.Brief = common.StringPtr("brief")
req.MaxCapacity = common.Uint64Ptr(uint64(100))
req.MaxQps = common.Uint64Ptr(uint64(10))
req.GroupType = common.Uint64Ptr(uint64(5))
//调用请求
resp, err := cli.CreateGroup(req)
if err != nil {
return
}3. Mini program realizes product search
The above introduces the basic capabilities of image search. How to apply product search capabilities to achieve the effect of instant shopping. Next, take a small program as an example to demonstrate a simple application:
3.1 Build the base library
According to the above documents, we use sdk on the server side to initialize the image library, and then access it through the applet. We will not go into details here.
1. Use the secretID and secretkey applied for in 2.1 to create the gallery.
2. Store the product pictures in the warehouse.
3.2 Build mini program
index.js
const app = getApp()
Page({
data: {
urls: [],
inputValue: "图片URL",
buttonStatus: false,
category: ['Jacket', 'dress', 'trousers', 'bag', 'shoe', 'accessories'],
motto: 'Hello World',
userInfo: {},
hasUserInfo: false,
canIUse: wx.canIUse('button.open-type.getUserInfo'),
canIUseGetUserProfile: false,
canIUseOpenData: wx.canIUse('open-data.type.userAvatarUrl') && wx.canIUse('open-data.type.userNickName') // 如需尝试获取用户信息可改为false
},
// 事件处理函数
bindViewTap() {
wx.navigateTo({
url: '../logs/logs'
})
},
onLoad() {
if (wx.getUserProfile) {
this.setData({
canIUseGetUserProfile: true
})
}
},
getUserProfile(e) {
// 推荐使用wx.getUserProfile获取用户信息，开发者每次通过该接口获取用户个人信息均需用户确认，开发者妥善保管用户快速填写的头像昵称，避免重复弹窗
wx.getUserProfile({
desc: '展示用户信息', // 声明获取用户个人信息后的用途，后续会展示在弹窗中，请谨慎填写
success: (res) => {
console.log(res)
this.setData({
userInfo: res.userInfo,
hasUserInfo: true
})
}
})
},
getUserInfo(e) {
// 不推荐使用getUserInfo获取用户信息，预计自2021年4月13日起，getUserInfo将不再弹出弹窗，并直接返回匿名的用户个人信息
console.log(e)
this.setData({
userInfo: e.detail.userInfo,
hasUserInfo: true
})
},
isUrl (url) {
var urlRegExp=/^(?:http(s)?:\/\/)?[\w.-]+(?:\.[\w\.-]+)+[\w\-\._~:/?#[\]@!\$&'\*\+,;=.]+$/;
if(urlRegExp.test(url)){
return true;
}else{
return false;
}
},
imageSearch() {
var that = this
wx.request({
url: 'http://****/search',
data: {
"GroupId": "***",
"ImageUrl": this.data.inputValue
},
method: "POST",
header: {
'Content-Type': "application/json"
},
success (res) {
if (res.data == null) {
wx.showToast({
icon: "error",
title: '请求失败',
})
return
}
console.log(res.data)
that.setData({
urls: res.data.Urls,
object: res.data.Object
})
that.drawInputImage()
},
fail(res) {
wx.showToast({
icon: "error",
title: '请求失败',
})
}
})
},
drawInputImage: function() {
var that = this;
wx.downloadFile({
url: that.data.inputValue,
success: function(res) {
var imagePath = res.tempFilePath
// 绘制
wx.getImageInfo({
src: imagePath,
success: function(res) {
wx.createSelectorQuery()
.select('#input_canvas') // 在 WXML 中填入的 id
.fields({ node: true, size: true })
.exec((r) => {
// Canvas 对象
const canvas = r[0].node
// 渲染上下文
const ctx = canvas.getContext('2d')
// Canvas 画布的实际绘制宽高
const width = r[0].width
const height = r[0].height
// 初始化画布大小
const dpr = wx.getWindowInfo().pixelRatio
canvas.width = width * dpr
canvas.height = height * dpr
ctx.scale(dpr, dpr)
// 清空画布
ctx.clearRect(0, 0, width, height)
let radio = height / res.height
console.log("radio:", radio)
const img = canvas.createImage()
var x = width / 2 - (res.width * radio / 2)
img.src = imagePath
img.onload = function() {
ctx.drawImage(img, x, 0, res.width * radio, res.height * radio)
var allBox = that.data.object.AllBox
ctx.fillStyle = 'rgba(247, 15, 2, 0.7)';
for (var i in allBox) {
if (allBox[i].Rect.X == that.data.object.Box.Rect.X &&
allBox[i].Rect.Y == that.data.object.Box.Rect.Y) {
continue
}
ctx.fillRect(x + (allBox[i].Rect.X * radio),
allBox[i].Rect.Y * radio,
allBox[i].Rect.Width * radio,
allBox[i].Rect.Height * radio);
}
// 绘制主体
ctx.fillStyle = 'rgba(159, 255, 125, 0.7)'; //rgba(0, 0, 200, 0.5)
ctx.fillRect(x + (that.data.object.Box.Rect.X * radio),
that.data.object.Box.Rect.Y * radio,
that.data.object.Box.Rect.Width * radio,
that.data.object.Box.Rect.Height * radio);
console.log(that.data.object.AllBox)
// 绘制文字背景
let text = `${that.data.category[that.data.object.CategoryId]} ${that.data.object.Box.Score}`
ctx.fillStyle = '#221329'
ctx.fillRect(x + (that.data.object.Box.Rect.X * radio), that.data.object.Box.Rect.Y * radio, that.data.object.Box.Rect.Width * radio, 15)
// 绘制文字
ctx.fillStyle = '#fcfafc'
console.log(that.data.category[that.data.object.CategoryId])
ctx.fillText(text, x + (that.data.object.Box.Rect.X * radio), that.data.object.Box.Rect.Y * radio + 10)
}
})
}
})
}
})
},
handlerInput(e) {
console.log(e)
this.setData({
inputValue: e.detail.value
})
},
handlerSearch(e) {
console.log(this.data.inputValue)
if (!this.isUrl(this.data.inputValue)) {
console.log("error url: ", this.data.inputValue)
wx.showToast({
icon: "error",
title: '请输入正确url',
})
return
}
this.setData({buttonStatus: true})
this.imageSearch()
this.setData({buttonStatus: false})
},
handlerInputImage(e) {
console.log(e)
}
})
index.wxss:
/**index.wxss**/
.userinfo {
display: flex;
flex-direction: column;
align-items: center;
color: #aaa;
}
.userinfo-avatar {
overflow: hidden;
width: 128rpx;
height: 128rpx;
margin: 20rpx;
border-radius: 50%;
}
.usermotto {
margin-top: 200px;
}
page {
background: rgb(255, 255, 255);
}
.page-section-spacing {
width: 100%;
}
.container {
padding: 0;
}
.button_container {
margin-top: 600rpx;
}
.flex-wrp{
margin-top: 60rpx;
margin: auto;
width: 90%;
overflow: hidden;
}
.form-item {
margin-bottom: 10px;
margin-top: 10px;
width: 90%;
overflow: hidden;
}
.flex-item{
float: left;
width: 300rpx;
height: 400rpx;
font-size: 26rpx;
margin: 10rpx 10rpx 0rpx 20rpx;
overflow: hidden;
}
.flex-item-V{
margin: 0 auto;
width: 300rpx;
height: 200rpx;
}
.input {
width: 75%;
float: left;
height: 36px;
border: 1px solid #ddd;
}
.button {
float: right;
height: 36px;
background: rgb(236, 111, 61);
color: white;
}
.view_line {
width: 96%;
height: 50rpx;
display: flex;
flex-direction: row;
align-items: center;
justify-content: left;
margin: 0rpx 2% 0rpx 2%;
}
.view_line view {
width: 75%;
height: 2rpx;
background: linear-gradient(to right,#706f70, #706f70);
}
.text_line {
font-size: 25rpx;
color: rgb(66, 66, 66);
margin: 0rpx 2% 0rpx 2%;
}
.image {
width: 100%;
}
.image image {
width: 100%;
background-size: auto 100%;
background-repeat: no-repeat;
}index.wxml:
<view class="container">
<div class="form-item">
<input placeholder=" 商品URL" class="input" bindinput="handlerInput" />
<button class="button" loading="{
{buttonStatus}}" bindtap="handlerSearch" size="mini">搜索 </button>
</div>
<view class="view_line">
<text class="text_line">输入图片</text>
<view></view>
</view>
<canvas
type="2d"
id="input_canvas"
style="background:rgb(228, 228, 225); width: 360px; height: 240px;"
></canvas>
<view class="view_line">
<text class="text_line">搜索结果</text>
<view></view>
</view>
<view class="page-section-spacing">
<view class="flex-wrp" style="flex-direction:row;">
<view wx:for="{
{urls}}" class="flex-item">
<view class="image">
<image mode="aspectFit" style="background-color: #eeeeee;" src="{
{item}}"></image>
</view>
</view>
</view>
</view>
</view>Just compile:
3.3 Build and run
Enter the product image, the image search service will perform subject recognition, and then search for similar images. You can see the following effect:
If we don’t do subject recognition, we may not be able to find the results we want. For example, we want these sunglasses, but if we don’t turn on subject recognition, the clothes may be located, resulting in no results:
Look at the picture below to search for the document introduction of subject recognition:
118px uses product image search to turn on subject recognition by default , involving the following two parameters:
By default, subject recognition is turned on, and the service will perform subject location recognition on the clothing in the input image to achieve better search results, as follows:
According to the above description of subject identification, specify the accessories category, the service will locate the sunglasses, and you can search for the expected products:
At this step, the basic function of product search in the mini program has been implemented. You can refer to the scenarios involving product search. Later, I showed a demo of the mini program to a friend. After experiencing it, I was quite satisfied with the effect.
3.4 Check the call volume
In subsequent observations, you can enter the image analysis console on the Tencent Cloud official website to view recent calls.
Learn more about image search capabilities: https://cloud.tencent.com/product/imagesearch