Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards

By Suchet Bargoti, James Patrick Underwood. J. Field Robotics, 2016

448 citations recorded by the source corpus.

Ground vehicles equipped with monocular vision systems are a valuable source of high‐resolution image data for precision agriculture applications in orchards. This paper presents an image processing framework for fruit detection and counting using orchard image data. A general‐purpose image segmentation approach is used, including two feature learning algorithms; multiscale multilayered perceptrons (MLP) and convolutional neural networks (CNN). These networks were extended by including contextual information about how the image data was captured (metadata), which correlates with some of the appearance variations and/or class distributions observed in the data. The pixel‐wise fruit segmentation output is processed using the watershed segmentation (WS) and circular Hough transform (CHT) algorithms to detect and count individual fruits. Experiments were conducted in a commercial apple orchard near Melbourne, Australia. The results show an improvement in fruit segmentation performance with the inclusion of metadata on the previously benchmarked MLP network. We extend this work with CNNs, bringing agrovision closer to the state‐of‐the‐art in computer vision, where although metadata had negligible influence, the best pixel‐wise F1‐score of 0.791 was achieved. The WS algorithm produced the best apple detection and counting results, with a detection F1‐score of 0.861. As a final step, image fruit counts were accumulated over multiple rows at the orchard and compared against the post‐harvest fruit counts that were obtained from a grading and counting machine. The count estimates using CNN and WS resulted in the best performance for this data set, with a squared correlation coefficient of r2=0.826 .

Subfields: Analytical Chemistry, Plant Science

View Paper: https://www.semanticscholar.org/paper/b95191e4672475b32d0a4c571bd2817659747dca

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Analytical Chemistry

In Plant Science

Citations in the corpus

An autonomous strawberry‐harvesting robot: Design, development, integration, and field evaluation (2019, methodology): “Deep learning is a promising method to deal with the lighting variations and the general idea is to capture and train images under different lighting conditions (Bargoti & Underwood, 2017; Fu et al., 2018).”
Advances in Agriculture Robotics: A State-of-the-Art Review and Challenges Ahead (2021, unknown): “Yield Estimation Shrimp [90] Apple – RGB camera MLP and CNN”
Recognition and Localization Methods for Vision-Based Fruit Picking Robots: A Review (2020, background): “Visual-institutional coordination and error tolerance can be used to enhance robotic anti-collision precision picking guidance, picking sequence planning and the decision-making behavior of a robot (Tanigaki et al., 2008; Zou et al., 2012; Barth et al., 2019; Zhang et al., 2019).”
Grape detection, segmentation and tracking using deep neural networks and three-dimensional association (2019, methodology): “The first attempts employed CNNs to perform pixel classification, followed by additional steps to segment individual fruits [5, 7].”
Research and development in agricultural robotics: a perspective of digital farming. (2018, background): “…[54-57] ), or (ii) navigation algorithms and robust machine vision systems for development of field robots that can be used in yield estimation [42,58,59] , thinning [60] , weeding and targeted spraying [61-64] , seedling and transplanting [65,66] , delicate handling of sensitive flowers…”
Deep learning for real-time fruit detection and orchard fruit load estimation: benchmarking of ‘MangoYOLO’ (2019, methodology): “However, in reports to date (e.g., Bargoti and Underwood 2017a, Sa et al.”
A Review of Convolutional Neural Network Applied to Fruit Image Processing (2020, background): “Bargoti and Underwood [28,95] presented a framework for apple and mangoes detection and counting, using orchards image data.”
Deep fruit detection in orchards (2016, methodology): “This work is a natural progression of the authors’ line of prior work in orchard imaging [6]–[8], [15], which has focused on achieving stateof-the-art results with fruit detection for orchard-scale yield mapping and estimation.”