I. Introduction
The strawberry picking robot control system was designed and developed by Cheng Pengsheng, a mechanical engineering major of Jiangxi University of Science and Technology. Aiming at the time-consuming, long-term and high-cost manual picking of elevated strawberries planted in a strawberry planting base, an omnidirectional mobile strawberry picking robot system scheme was proposed.
The hardware circuit of the system mainly includes the main controller with STM32F407ZGT6 as the core, power supply circuit, WIFI module, laser ranging unit, and motor drive module. The software design of the system mainly includes the transplantation and debugging of the RT-thread real-time operating system on the single-chip computer, the visual recognition of the Raspberry Pi 3B+, the communication debugging of the data transmission between the Raspberry Pi and the single-chip computer, the access of the Gizwits Internet of Things and WIFI modules on the single-chip computer, and the realization of the control software of the Raspberry Pi upper computer.
By using the deep learning algorithm to realize the identification and positioning of strawberries, the picking information and the state of the picking robot are transmitted to the cloud using the Internet of Things technology, so as to realize the autonomous picking and cloud monitoring of the strawberry picking robot, and finally complete a set of stable and controlled strawberry picking system.
2. Overall scheme design of strawberry picking robot
This system is designed to build a mobile strawberry picking robot prototype for the picking of elevated strawberries. The whole system consists of an omnidirectional mobile chassis, a robotic arm, a vision system, and a motion controller. The system design diagram is shown in Figure 1.1. 
Figure 1.1 System Design Diagram
The system framework is shown in Figure 1.2, and the structure diagram of the picking robot is shown in Figure 1.3. The omnidirectional mobile chassis of the picking robot, the manipulator and its control, the visual recognition system and the control system of the whole machine are mainly studied.
Figure 1.2 System block diagram of strawberry picking robot
Figure 1.3 Structural diagram of the whole machine picking robot
According to a strawberry planting base adopting a one-line planting method, the simulated construction of a strawberry stand is shown in Figure 1.4. The early debugging uses simulated strawberries, and the later debugging uses real strawberries. In the fruit picking operation, only the independently grown strawberries are picked, and the strawberries that are in contact with each other are not separated and picked. The action flow of the whole picking robot is shown in Figure 1.5. According to the planting characteristics of the planting base, the strawberry planting ranks and rows are standardized, and the experimental prototype only conducts experiments on single-row planting, and the picking of multiple rows is realized in the follow-up industrial prototype according to the navigation method.
Figure 1.4 Strawberry stand and picking robot
Figure 1.5 The action flow of the whole machine picking robot
3. Control system design of strawberry picking robot
1. Hardware introduction
The block diagram of the hardware control system designed for the strawberry picking robot is shown in Figure 2.1, including the core control unit, sensors, drivers, etc. The core control unit includes a single-chip microcomputer and a Raspberry Pi. The Raspberry Pi is responsible for processing visual data and issuing instructions. The single-chip microcomputer receives and executes instructions. In addition, it also processes some simple sensor data. Sensors include laser ranging, gyroscopes, encoders, and thin-film pressure sensors. The driver includes the motor drive and the servo drive of the manipulator, and the WIFI module in the IoT unit is responsible for uploading the picking data and the state of the robot to the cloud.
Figure 2.1 Block diagram of the hardware platform system
(1) Main control chip
Select the main control chip of the controller according to the hardware requirements of the mobile robot. In view of the energy consumption of the experimental platform and the data processing capability of the controller, the STM32F407ZGT6 chip is selected as the controller chip. And the Raspberry Pi 3B+ is used to run the image processing program, and the calculation result is sent to the single-chip microcomputer after the inverse calculation of the mechanical arm. The physical picture of Raspberry Pi 3B+ is shown in Figure 2.2.
Figure 2.2 Raspberry Pi 3B+ physical map
(2) Wireless module
In order to check the working status and picking quantity of the picking robot conveniently, this paper adopts the way of wireless network connection to realize it, and selects the ATK-ESP8266 module as the node for sending and receiving data of the robot. The ATK-ESP8266 module supports TTL serial communication and is compatible with 3.3V and 5V voltages. It is easy to connect to MCUs with different voltage requirements. The actual module is shown in Figure 2.3, and the functions of each pin are shown in Table 1. To use this module, first pull down IO_0 to enter the firmware programming mode. After the programming is completed, the signal of the module can be searched through the mobile phone or computer. Finally, the sending pin and receiving pin of the module can be cross-connected with the sending and receiving pins of the microcontroller to communicate.
Figure 2.3 ATK-ESP8266 module physical map
Table 1 ATK-ESP8266 module pin function
(3) Hardware module design
Using the STM32F407ZGT6 chip as the core chip of the controller, a PCB base plate that fully matches the experimental platform of this paper is designed. The connection pin diagram of the PCB base plate and the core board and the three-dimensional view of the base plate are shown in Figure 2.4. Interfaces of multiple sensors such as encoders, serial ports, gyroscopes, and the I2C bus interface of the steering gear control board are set on the bottom board, as well as pin headers and female headers for external pins. The encoder interface uses the standard interface of XH2.54 to make the overall layout of the experimental platform more beautiful.
Figure 2.4 Main Controller
2. Software introduction
The software design part of the control system is composed of hardware control and data processing. The hardware part uses a real-time operating system, which can process data quickly when data or commands are generated. When multiple tasks appear at the same time, the operation of real-time tasks will be controlled according to the priority of the tasks, which has the characteristics of timely response and high reliability.
For the picking robot system, it not only needs to control the robot body but also has the function of uploading robot information and operation status. The Internet of Things platform has become an important tool. The realization of the Internet of Things generally requires three basic elements: data cloud, terminal equipment for uploading data, and equipment for obtaining data. This article uses the Gizwits IoT platform commonly used for smart hardware to configure, and connects to Gizwits in the form of WIFI wireless connection. The data uploaded to the cloud includes the information of the robot itself and the number of pickings.
- Internet of things platform implementation
Gizwits Cloud Platform is a platform for smart hardware development and cloud computing applications. It provides developers with an intelligent hardware development tool and an open cloud computing platform. The system has complete SDK and API service functions, which greatly reduces the difficulty and cost of hardware development. The strawberry picking robot designed in this paper uses the ESP-8266 wireless network module to connect to the Gizwits cloud server, so that the local information and picking quantity of the picking robot can be reported to the mobile phone or computer client, and the types of reported information can be expanded as needed.
To develop the Internet of Things function of the Gizwits Cloud platform, first create a new product on the Gizwits Cloud website, and then add data points. The data points are the information that can be seen on the mobile terminal. In this paper, five data points are set: picking quantity, camera status, robot power, manipulator status, and robot posture. The information for creating data points is shown in Figure 2.5. Among them, the maximum statistical quantity of picking is set to 65536, and it is in writable mode, that is, the quantity can be manually cleared after a certain quantity is harvested. The data type of power is alarm mode. When the voltage is lower than 11.7v, it will enter the alarm state. The angle range of the robot attitude is from 0 degrees to 360 degrees. Both the camera state and the manipulator state are represented by Boolean values.
Figure 2.5 Information of data points
After completing the setting of the data points, perform online debugging on the webpage, report the simulated device to view the data on the mobile terminal, adjust the data on the simulated device, and the data on the mobile terminal will change accordingly, and the data on the simulated device will be reported in the form of {variable name: variable value}. For example, set the battery status to 1, the camera status to 1, the manipulator status to 1, the robot posture to 0, and the picking quantity to 20 on the simulation device side, as shown in Figure 2.6. It can be seen that the mobile terminal is updated normally and the data points can be used normally.
Figure 2.6 Data reporting
After debugging, use the MCU development tool of Gizwits to generate the stm32 hardware code. Since the generated code is the code of the hal library, the underlying control code of the robot is written using the standard library, so it needs to be transplanted. According to the Gizwits Cloud network protocol, first establish a time base of 1 millisecond, then add the smart cloud data receiving function to the serial port receiving interrupt processing function, and modify it into a real transmission interface function. Finally, create a new thread to report data regularly, and the transplantation of Gizwits Cloud is completed. In addition, to start the transmission of Gizwits Cloud, you need to set the ESP-8266 to Soft-AP mode in the system initialization, so that the device can be found and connected in the Gizwits Cloud APP.
4. Experiment and analysis of strawberry picking robot
The time-consuming of each process is recorded and counted through the five picking processes, as shown in Table 2. The average time of the image recognition process is 0.23s, the average time for the robotic arm to move to the target point and pick and place is 6.8s, and the total time is about 7s. The process of placing the picked strawberries into the fruit basket takes a long time, because the fruit basket is directly behind the manipulator, and the manipulator needs to be turned 180° to place the strawberries.
Table 2 Picking time-consuming segmentation table
The strawberry frame is built with mm2020mm profiles, the length direction of the strawberry frame is 3000mm, and the height direction is 500mm. This experiment only considers the situation of indoor picking. The picking process is shown in Figure 3, a shows that the picking robot moves to the front of the strawberry; b, the manipulator moves to the top of the fruit; c, the end effector jaw closes; d, the fruit stem is cut; e, the manipulator lowers its posture;
Figure 3 Picking process diagram
V. Summary
In this design, a four-degree-of-freedom manipulator using Mecanum's omnidirectional mobile chassis and a strawberry picking robot prototype that includes visual recognition have been completed. The free walking of the mobile platform has been realized, and the advantages of the threshold method and convolutional neural network have been integrated, as well as the harvesting of strawberries by the manipulator. Design of software and hardware for strawberry picking robot. Select the hardware module and main control chip of the strawberry picking robot, and at the same time carry out PCB design and proofing for the power circuit, drive circuit, and encoder connection circuit. The underlying hardware program framework adopts the RT-Thread real-time operating system. The visual recognition program runs on the Raspberry Pi and uses the wireless module to upload the picking information and robot status to the Gizwits Cloud Internet of Things, which can be viewed on the computer and mobile phones at the same time.