Table of contents
Preface
This project is based on the Keras framework, introduces CNN for model training, uses Dropout gradient descent algorithm, discards some neurons in proportion, and uses IOT and WeChat applet to realize automatic remote monitoring of fruit maturity and real-time monitoring on mobile terminals, providing fruit farmers with Picking guidance is helpful to save labor, improve production efficiency, and enhance economic benefits.
This project is based on the Keras framework and uses convolutional neural network (CNN) for model training. By introducing the Dropout gradient descent algorithm, proportional discarding of neurons is achieved to improve the robustness and generalization performance of the model. At the same time, using Internet of Things (IoT) technology and WeChat applet, the project has realized the function of automated remote monitoring of fruit maturity and real-time monitoring of orchard status on the mobile terminal. This provides fruit farmers with real-time guidance on picking, helps save labor, improves production efficiency, and thereby improves the economic benefits of orchards.
First, the project uses the Keras framework to build a convolutional neural network, using deep learning technology to accurately identify and predict fruit maturity.
Secondly, the Dropout gradient descent algorithm is introduced to prevent the model from overfitting and improve the generalization ability to new data by randomly discarding neurons.
Then, the project integrated Internet of Things technology to remotely monitor the fruits in the orchard through sensors and other equipment. In this way, fruit farmers can remotely understand the maturity status of fruits at different locations.
At the same time, through the WeChat applet, fruit farmers can monitor the status of orchards in real time and understand information such as fruit maturity and picking timing, so as to arrange picking work more scientifically.
Overall, this project not only introduced advanced deep learning technology in model training, but also implemented an intelligent orchard management system through the Internet of Things and WeChat applet, providing fruit farmers with more convenient and efficient agricultural production solutions.
overall design
This part includes the overall system structure diagram and system flow chart.
Overall system structure diagram
The overall structure of the system is shown in the figure.
System flow chart
The model training process is shown in the figure.
The data upload process is shown in the figure.
The applet process is shown in the figure.
Operating environment
This part includes Python environment, TensorFlow environment, JupyterNotebook environment, PyCharm environment, WeChat developer tools and OneNET cloud platform.
Python environment
详见行客.
TensorFlow environment
详见行客.
Jupyter Notebook environment
详见行客.
Pycharm environment
详见行客.
WeChat Developer Tools
详见行客.
OneNET cloud platform
详见行客.
Module implementation
This project includes 5 modules: data preprocessing, model creation and compilation, model training and saving, uploading results, and applet development. The function introduction and related codes of each module are given below.
1. Data preprocessing
Taking red dates as the experimental object, 1,000 pictures were crawled on the Internet as a data set.
详见行客.
2. Create the model and compile it
After the data is loaded into the model, the model structure needs to be defined and the loss function optimized.
详见行客.
3. Model training and saving
After defining the model architecture and compiling, it is trained through the training set so that the model can identify the maturity level of red dates. Here we will use the training set and test set to fit and save the model.
详见行客.
4. Upload results
There are two methods for uploading results: one is to call the computer camera to take pictures, convert the picture information into a binary data stream and upload it to the OneNET cloud platform; the other is to input the digital picture into the Keras model, obtain the output and upload the recognition results to the OneNET cloud platform.
详见行客.
5. Mini program development
WeChat applet is used to view fruit pictures, obtain picking suggestions and query identification results.
1) Query pictures
The image query function uses two nested callbacks: the first layer obtains the image index directory by accessing the image data stream and passes it to the second layer; the second layer uses the image index directory to access the image data stream information and obtains the image binary data stream. In order to enable the picture to be displayed in the interface, convert the binary data to Base64 format, use the that.setData() function to pass the value to the wxm file, and modify the button in the function The keyword on the button is "Click to view harvesting suggestions" to switch the button function.
//回调图片
send: function () {
var that = this
if (that.data.keyword=='单击查看你的果园'){
//多重回调，两次
const requestPicIndex = wx.request({
url: 'https://api.heclouds.com/devices/586488389/datapoints?datastream_id=pic',
header: {
'content-type': 'application/json',
'api-key': '93IlIl2tfXddMN8sgQIInc7qbXs='
},
success: function (res) {
var picIndex=res.data.data.datastreams[0].datapoints[0].value.index
console.log(res.data.data.datastreams[0].datapoints[0].value.index)
//打印图片索引目录；OneNet上图片的索引
//嵌套的第二次回调
const requestTask = wx.request({
url: 'http://api.heclouds.com/bindata/' + picIndex,//图片url
header: {
'content-type': 'application/json',
'api-key': 'RSKlDBtVrZ7qDWvK=b6IAyFi=Ow='
//master-apikey，可操控OneNET上所有东西
},
responseType: 'arraybuffer',//相应类型
success: function (res) {
console.log(res.data) //打印返回中的data,res代表返回数据
var data = res.data
var base64 = wx.arrayBufferToBase64(res.data)
//二进制数据流转化成base64
base64 = base64.replace(/[\r\n]/g, "") //删去换行符
that.setData({
imgUrl: 'data:image/PNG;base64,' + base64,
//能够显示图片base64的形式，传值给wxml
keyword: '单击查看采收建议'//修改button功能为返回采收建议
})
console.log('http://api.heclouds.com/bindata/'+picIndex)//打印url
},
fail: function (res) {
//异常处理
console.log("fail!!!")
},
complete: function (res) {
console.log("end")
}
})
},
//回调失败则打印fail!!!
fail: function (res) {
console.log("fail!!!")
},
//回调完成打印图片url
complete: function (res) {
console.log("end")
}
})
}
}
2) Query recognition results
After obtaining the recognition result, a numerical judgment is made: "0" represents immature and not suitable for harvesting; "1" represents half-ripe, which is the best time to harvest; "2" represents fully mature and needs Harvest as soon as possible. Assign the value to reM through the that.setDataM() function to display harvesting suggestions.
//回调识别结果
else if (that.data.keyword=='单击查看采收建议'){
const requestTask = wx.request({
url: 'https://api.heclouds.com/devices/586488389/datapoints?datastream_id=rslt', //识别结果的url
header: {
'content-type': 'application/json',
'api-key': '93IlIl2tfXddMN8sgQIInc7qbXs='
},
success: function (res) {
var app = getApp()
app.globalData.Zao = res.data.data.datastreams[0]
var a = app.globalData.Zao.datapoints[0].value
console.log(app.globalData.Zao)
//0代表未成熟，不适合采收；1代表半熟，最佳采收时机；2代表完全成熟，尽快采收
if (a == 2) {
console.log(a)
that.setData({
reM: '完全成熟，请尽快采收！'
})
console.log('reM:' + that.data.reM)
}
else if (a == 1) {
console.log(a)
that.setData({
reM: '半熟，现在是最佳的采收时机'
})
console.log('reM:' + that.data.reM)
}
else if (a == 0) {
console.log(a)
that.setData({
reM: '不成熟，还不能采收哦~'
})
console.log('reM:' + that.data.reM)
}
},
//回调失败则打印fail!!!
fail: function (res) {
console.log("fail!!!")
},
//回调完成打印结果
complete: function (res) {
console.log("end")
}
})
}
System test
This section includes training accuracy, testing effects, and external access effects.
1. Training accuracy
The test accuracy rate reaches about 88%, which means that the prediction model training is relatively successful. As the number of training rounds increases, the loss and accuracy of the model on the training data and test data gradually converge and eventually become stable, as shown in the figure.
2. Test effect
Substituting the test set data into the model for testing, and displaying and comparing the classification labels with the original data, it was verified that the model can realize the identification of three types of ripeness levels of red dates. The test results are shown in the figure.
3. External access effects
Open the mini program and the initial interface is as shown in the figure.
Click the "Login" button at the top of the interface to obtain the user's WeChat avatar and nickname, as shown in the figure.
Click the "Click to view your orchard" button in the interface. A picture of the fruit will appear above the button, and the text on the button will change to "Click to view harvesting suggestions", as shown in the picture.
Click the "Click to View Harvesting Suggestions" button, and fruit maturity information and specific harvesting suggestions will appear below the button, as shown in the figure.
The mobile terminal test results are shown in the figure.
Related other blogs
Project source code download
For details, please see my blog resource download page
Download other information
If you want to continue to understand the learning routes and knowledge systems related to artificial intelligence, you are welcome to read my other blog " Heavy | Complete Artificial Intelligence AI Learning - Basics Knowledge learning route, all materials can be downloaded directly from the network disk without following any routines》
This blog refers to Github’s well-known open source platform, AI technology platform and experts in related fields : Datawhale, ApacheCN, AI Youdao and Dr. Huang Haiguang have about 100G of related information. I hope it can help all my friends.