Deep fruit detection in orchards

By Suchet Bargoti, James Patrick Underwood. IEEE International Conference on Robotics and Automation, 2016

501 citations recorded by the source corpus.

An accurate and reliable image based fruit detection system is critical for supporting higher level agriculture tasks such as yield mapping and robotic harvesting. This paper presents the use of a state-of-the-art object detection framework, Faster R-CNN, in the context of fruit detection in orchards, including mangoes, almonds and apples. Ablation studies are presented to better understand the practical deployment of the detection network, including how much training data is required to capture variability in the dataset. Data augmentation techniques are shown to yield significant performance gains, resulting in a greater than two-fold reduction in the number of training images required. In contrast, transferring knowledge between orchards contributed to negligible performance gain over initialising the Deep Convolutional Neural Network directly from ImageNet features. Finally, to operate over orchard data containing between 100–1000 fruit per image, a tiling approach is introduced for the Faster R-CNN framework. The study has resulted in the best yet detection performance for these orchards relative to previous works, with an F1-score of > 0.9 achieved for apples and mangoes.

Subfields: Computer Vision and Pattern Recognition, Environmental Engineering, Plant Science

View Paper: https://www.semanticscholar.org/paper/45d45687ebca6281d44e477cd012a1ef48ebf286

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Environmental Engineering

In Plant Science

Citations in the corpus

Deep learning in agriculture: A survey (2018, methodology): “, 2016) or to enable easier data annotation by experts and volunteers (Chen et al., 2017; Bargoti and Underwood, 2016).”
Fruit recognition from images using deep learning (2017, methodology): “shape(local_image) 24 local_height = shape[0] 25 local_width = shape[1] 26 local_depth = shape[2] 27 local_image = tf.”
Detection and counting of flowers on apple trees for better chemical thinning decisions (2019, methodology): “Specifically, Bargoti and Underwood (2017) used the faster R-CNN algorithm (Ren et al.”
A detection algorithm for cherry fruits based on the improved YOLO-v4 model (2021, methodology): “The Faster R-CNN can also generate a model for fruit detection, and the average accuracy results show that using this method can make the accuracy of fruit recognition better [16].”
YOLO-Tomato: A Robust Algorithm for Tomato Detection Based on YOLOv3 (2020, methodology): “[20] also proposed a fruit detection model in orchards, based on the Faster R-CNN method.”
Empowering Things With Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things (2020, background): “…detection/trakcing [139], [201], [140], [141], counting [202], [141], [205], semantic segmentation [202], [141], 3D [139], [140], [141], [251] TL [201], [202], DA [205], RL [217], [218] KG reasoning [225] control [217], interaction [139], [140], [225] Smart Cities / Homes / Build-ings object…”
Deep learning for real-time fruit detection and orchard fruit load estimation: benchmarking of ‘MangoYOLO’ (2019, methodology): “Bargoti and Underwood (2017a) reported imaging of mango orchards in day light hours using high intensity strobe lighting and very short exposure times to avoid issues in direct sun lighting, but this required a camera capable of microsecond exposures and use of high intensity Xenon strobe lights, in”
Automation in Agriculture by Machine and Deep Learning Techniques: A Review of Recent Developments (2021, methodology): “…method, the conclusion being that for the fruits detection task of the dataset used in that research, this approach was not very useful in terms of average precision, and the transfer learning strategy was suggested with variation/diversity in the dataset images (Bargoti & Underwood, 2017a, b).”
A review of the use of convolutional neural networks in agriculture (2018, methodology): “Some of these images and data sets originated from well-known and publicly available resources such as PlantVillage, LifeCLEF, MalayaKew and UC Merced, while others were produced by the authors for their research needs (Xinshao and Cheng, 2015; Sladojevic et al., 2016; Bargoti and Underwood, 2017; R”
Grape detection, segmentation and tracking using deep neural networks and three-dimensional association (2019, methodology): “attempts employed CNNs to perform pixel classiﬁcation, followed by additional steps to segment individual fruits [5, 7]. Further, these earlier approaches were replaced by end-to-end object detection [4, 31, 45] based on the popular Faster R-CNN architecture [42]. Sa et al. [45] employed transfer le”
Deep Count: Fruit Counting Based on Deep Simulated Learning (2017, methodology): “…was tested over 100 randomly-chosen images; despite not having real images The algorithm was tested over 100 randomly-chosen images; despite not having real The algorithm was tested over 100 randomly-chosen images; despite not having The algorithm was tested over 100 randomly-chosen images;…”
A Review of Convolutional Neural Network Applied to Fruit Image Processing (2020, methodology): “Specifically in agriculture, CNN-based approaches have been used for fruit classification [24–26] and fruit detection [27,28].”