2018
Computer Vision and Machine Learning for Viticulture Technology
Abstract: This paper makes two contributions to the advanced study of viticultural techniques. First, we provide a comprehensive overview of computer vision, image processing, and machine learning techniques in viticulture. We summarize recent advances in vision systems and technologies for various representative studies, including harvest yield estimation, vineyard management and monitoring, grapevine disease detection, quality evaluation, and grapevine phenology. We focus on how computer vision and machine learning techniques can be integrated into current vineyard management and winemaking processes to achieve industry relevant results. The second part of this paper presents the new GrapeCS-ML database, which includes images of grape varieties at different stages of development and corresponding ground truth data (such as pH and Brix) obtained from chemical analysis. One of the goals of this database is to motivate computer vision and machine learning researchers to develop practical solutions to be deployed in smart vineyards. We illustrate the effectiveness of this database for color berry detection applications of white and red varieties and give baseline comparisons using various machine learning methods and color spaces. The article concludes by highlighting future challenges that need to be addressed before this technology can be successfully implemented in the viticulture industry.
2020
Grape detection, segmentation, and tracking using deep neural networks and three-dimensional association（Computers and Electronics in Agriculture）
Abstract: Agricultural applications, such as yield prediction, precision agriculture, and automated harvesting, require systems that can infer crop status from low-cost sensing devices. Proximity sensing using inexpensive cameras combined with computer vision has emerged as a promising alternative following the emergence of convolutional neural networks (CNNs) as an alternative to challenging pattern recognition problems in natural images. Programs have been strengthened. Considering fruit planting monitoring and automation, a fundamental problem is the detection, segmentation and counting of individual fruits in an orchard. Here we show that for wine grapes, a crop that varies greatly in shape, color, size, and compactness, clusters of grapes can be successfully detected, segmented, and tracked using state-of-the-art CNNs. On a test set containing 408 grape cluster images of vineyards based on a grid system, we have achieved an F1 score of 0.91 for example by segmenting each cluster image, allowing a more accurate assessment of fruit size and shape. We also show that clusters can be identified and tracked along video sequences recording orchard rows. We also present a public dataset containing appropriately annotated grape clusters in 300 images, and a new annotation method for segmenting complex objects in natural images. The proposed pipeline for annotation, training, evaluation and tracking of agricultural patterns in images can be replicated for different crops and production systems. It can be used in the development of sensing components for several agricultural and environmental applications.
2021
Grape Bunch Detection at Different Growth Stages Using Deep Learning Quantized Models
Abstract: The agricultural sector plays an important role in our society where automation processes are gaining in importance, thereby having a beneficial impact on the productivity and quality of products. Perception and computer vision methods are fundamental to robotics in agriculture. In particular, deep learning can be used for image classification or object detection, giving machines the ability to perform operations in agricultural settings. In this work, deep learning is used to detect grape bunches in a vineyard, taking into account different growth stages: an early stage after flowering and an intermediate stage when grape bunches exhibit intermediate development. Two state-of-the-art single-shot multi-box models are trained, quantized, and deployed on a low-cost and low-power hardware device, a Tensor Processing Unit. The training input is a novel and publicly available dataset proposed in this work. This dataset contains 1929 images with annotations of two different growth stages, captured by different cameras under different lighting conditions. Intersection of confidence value and joint threshold considering variation of two different parameters. The results show that the deployed model can detect grape bunches in images with a medium average accuracy of 66.96%. Since the method uses low-resource, low-cost, low-power hardware devices and requires a simplified model using 8-bit quantization, the obtained performance is satisfactory. Experiments also showed that the model performed better at identifying grape bunches in the medium growth stage, with grape bunches appearing in vineyards after flowering, because the second class represents smaller grape bunches, more similar in color and texture to surrounding foliage, complex detection.
2022
Computer Vision and Machine Learning Based Grape Fruit Cluster Detection and Yield Estimation Robot
Abstract: At present, they are facing the problem of rapidly increasing labor cost and labor shortage. Early various techniques were developed using hyperspectral cameras, 3D images, sticky block-based segmentation, because it was difficult to find and distinguish grape bunches. This study implemented a new computer vision-based method for counting, detection, and segmentation of blue grape bunches using an open source computer vision library (OpenCV) and a random forest machine learning algorithm. The fruit object segmentation here is based on binary thresholding and Otsu's method. For training and testing, pixel intensity-based classification is performed on single images of grape and non-grape fruit. Validation of the technique represented by the Random Forest algorithm achieved good results with an accuracy of 97.5% and an f1-score of 90.7% compared to Support Vector Machines (SVM). Utilizes noise removal, training, segmentation and classification techniques with high accuracy.
Automatic Bunch Detection in White Grape Varieties Using YOLOv3, YOLOv4, and YOLOv5 Deep Learning Algorithms
Abstract: Over the past few years, several convolutional neural networks for object detection have been proposed, characterized by varying accuracy and speed. In viticulture, yield estimation and forecasting are used for effective crop management using sophisticated viticultural techniques. Convolutional neural networks for object detection are another alternative approach to grape yield estimation, which typically relies on manual harvesting of sample plants. In this paper, we evaluate six versions of the You Only Look Once (YOLO) object detection algorithm (YOLOv3, YOLOv3-tiny, YOLOv4, YOLOv4-tiny, YOLOv5x, and YOLOv5s) for real-time bunch detection and counting of grapes. This study A white grape variety was chosen because identifying white berries on leaf backgrounds was trickier than red berries. The YOLO model was trained using a heterogeneous dataset populated with images retrieved from open datasets and tested under multiple lighting conditions, background and growth stages. The results show that the f1-scores of YOLOv5x and YOLOv4 are 0.76 and 0.77, and the detection speeds are 31 and 32 FPS, respectively. The difference is that the f1-scores of YOLO5s and YOLOv4-tincy are 0.76 and 0.69, respectively, and the detection speeds are 31 and 32 FPS. The speeds are 61 and 196 FPS, respectively. The final YOLOv5x model, considering bundle occlusion, is able to estimate the number of bundles per plant with an average error of 13.3% per plant. YOLOv4-tincy achieves the best combination of accuracy and speed, Real-time estimation of grape yield should be considered, while YOLOv3 suffers from false positive-false negative compensation, which reduces RMSE.
LDD: A Dataset for Grape Diseases Object Detection and Instance Segmentation
Abstract: The instance segmentation task is an extension of the well-known object detection task, which is of great help in many fields, such as precision agriculture: being able to automatically identify plant organs and their possible associated diseases can effectively scale up and automate crop monitoring and its disease control. To address issues related to early disease detection and diagnosis in grapevine plants, we build a new dataset with the aim of advancing the state of the art in disease identification through instance segmentation methods. This was achieved by collecting images of disease-affected leaves and grape clusters in their natural setting. The dataset contains photos of 10 object types, including leaves and grapes with and without symptoms of eight common grape diseases, totaling 17,706 labeled instances in 1,092 images. In order to provide a complete view on the characteristics of the dataset, various statistical measures are proposed. Preliminary results on object detection and instance segmentation tasks achieved by the Mask R-CNN [6] and R3-CNN [10] models as baselines show that the procedure can achieve promising results for the goal of automatic symptom recognition of diseases.
2023
Benchmarking edge computing devices for grape bunches and trunks detection using accelerated object detection single shot multibox deep learning models
Purpose: Visual perception enables robots to perceive their environment. Vision data is processed using computer vision algorithms, which are often time-consuming and require powerful equipment to process vision data in real-time, which is not feasible for field robots with limited energy. This work benchmarks the performance of different heterogeneous platforms for real-time object detection. This research benchmarks three architectures: embedded GPU graphics processing unit (such as NVIDIA Jason nano 2 GB and 4GB, NVIDIA Jason TX2), TPU-tensor processor (such as Coral development board TPU), DPU-deep learning processor Units (such as AMD-Xilinx ZCU104 development board, and AMD-Xilinx Kria KV260 starter kit).
Methods: The authors used ResNet-50, a retinal network fine-tuned using the natural VineSet dataset. The trained model is converted and compiled to a target-specific hardware format for efficient execution.
Conclusions and Results: The platform was evaluated according to the performance and efficiency (inference time) of the evaluation metrics. Graphics processing units (GPUs) are the slowest devices, running at 3 FPS to 5 FPS, and field programmable gate arrays (FPGAs) are the fastest devices, running at 14 FPS to 25 FPS. Efficiency of the tensor processing unit (TPU) is irrelevant, similar to NVIDIA Jetson TX2. The TPU and GPU are the most energy efficient, consuming around 5W. On the evaluation metrics, the performance difference across devices is irrelevant, with F1 around 70% and average average precision (mAP) around 60%.
