DeepFruits: A Fruit Detection System Using Deep Neural Networks

By Inkyu Sa, ZongYuan Ge, Feras Dayoub, Ben Upcroft, Tristan Perez, C. McCool. 2016

1,000 citations recorded by the source corpus.

This paper presents a novel approach to fruit detection using deep convolutional neural networks. The aim is to build an accurate, fast and reliable fruit detection system, which is a vital element of an autonomous agricultural robotic platform; it is a key element for fruit yield estimation and automated harvesting. Recent work in deep neural networks has led to the development of a state-of-the-art object detector termed Faster Region-based CNN (Faster R-CNN). We adapt this model, through transfer learning, for the task of fruit detection using imagery obtained from two modalities: colour (RGB) and Near-Infrared (NIR). Early and late fusion methods are explored for combining the multi-modal (RGB and NIR) information. This leads to a novel multi-modal Faster R-CNN model, which achieves state-of-the-art results compared to prior work with the F1 score, which takes into account both precision and recall performances improving from 0.807 to 0.838 for the detection of sweet pepper. In addition to improved accuracy, this approach is also much quicker to deploy for new fruits, as it requires bounding box annotation rather than pixel-level annotation (annotating bounding boxes is approximately an order of magnitude quicker to perform). The model is retrained to perform the detection of seven fruits, with the entire process taking four hours to annotate and train the new model per fruit.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Media Technology, Plant Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Media Technology

In Plant Science

Citations in the corpus

Pix2seq: A Language Modeling Framework for Object Detection (2021, background): “, 2020), to agriculture (Sa et al., 2016), the specialization and complexity make them difficult to integrate into a larger system, or generalize to a much broader array of tasks associated with general intelligence.”
Deep Learning for IoT Big Data and Streaming Analytics: A Survey (2017, methodology): “In [113], Sa et al. used a variation of CNN, called Region-based CNN, for image analysis of fruits.”
An autonomous strawberry‐harvesting robot: Design, development, integration, and field evaluation (2019, methodology): “In addition to strawberries, deep learning, especially the Faster RCNN network, has been widely used for detection of many other fruits, including sweet pepper, mango, apple, almond, and kiwifruit (Fu et al., 2018; Mai, Zhang, & Meng, 2018; Sa et al., 2016; Zhang et al., 2019).”
YOLO-Tomato: A Robust Algorithm for Tomato Detection Based on YOLOv3 (2020, positive): “Sa et al. [18] applied the Faster R-CNN [19] detector to fruit detection.”
A Review of Convolutional Neural Network Applied to Fruit Image Processing (2020, methodology): “Despite that, although CNNs have recently received much more attention than any other ML algorithm, it has not been widely used in fruit research and reports a single article using CNN [36] as relevant for the date.”
Automation in Agriculture by Machine and Deep Learning Techniques: A Review of Recent Developments (2021, background): “Some of the recent studies were conducted for the fruit detection/harvesting task by wellknown DL architectures or by proposing an improved version of a DL model for forthcoming agricultural robotic projects (Sa et al., 2016; Zhang et al., 2019).”
Fruit recognition from images using deep learning (2017, methodology): “Like in paper [17], this project makes use of the”
Convolutional Neural Networks for Image-Based High-Throughput Plant Phenotyping: A Review (2020, neutral): “, Faster RCNN) for fruit detection [84].”
Detection and counting of flowers on apple trees for better chemical thinning decisions (2019, methodology): “Another example can be found in (Sa et al. 2016) work, in which they used the faster R-CNN algorithm to detect sweet peppers and rock-melons in RGB and Near-Infrared”
Internet of Things for the Future of Smart Agriculture: A Comprehensive Survey of Emerging Technologies (2021, background): “In [193], the authors proposed an intelligent IoT-based agriculture decision support system, applied to large biomedical datasets, such as plant disease datasets, as well as real-time applications, which improved the accuracy of classification by 9.”
An In-field Automatic Wheat Disease Diagnosis System (2017, methodology): “(Sa et al., 2016) proposed a multi-modal Faster R-CNN model, namely Deep-Fruits , to play real-time fruit detection.”