Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN

By Yang Yu, Kailiang Zhang, Li Yang, Dongxing Zhang. 2019

564 citations recorded by the source corpus.

Abstract Deep learning has demonstrated excellent capabilities for learning image features and is widely used in image object detection. In order to improve the performance of machine vision in fruit detection for a strawberry harvesting robot, Mask Region Convolutional Neural Network (Mask-RCNN) was introduced. Resnet50 was adopted as backbone network, combined with the Feature Pyramid Network (FPN) architecture for feature extraction. The Region Proposal Network (RPN) was trained end-to-end to create region proposals for each feature map. After generating mask images of ripe fruits from Mask R-CNN, a visual localization method for strawberry picking points was performed. Fruit detection results of 100 test images showed that the average detection precision rate was 95.78%, the recall rate was 95.41% and the mean intersection over union ( MIoU ) rate for instance segmentation was 89.85%. The prediction results of 573 ripe fruit picking points showed that the average error was ±1.2 mm. Compared with four traditional methods, the method proposed demonstrates improved universality and robustness in a non-structural environment, particularly for overlapping and hidden fruits, and those under varying illumination.

Subfields: Computer Vision and Pattern Recognition, Ecology, Industrial and Manufacturing Engineering, Plant Science, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Plant Science

Citations in the corpus

Instance Segmentation for Autonomous Log Grasping in Forestry Operations (2022, methodology): “Mask R-CNN was also evaluated for strawberries detection [22] in a fruit-picking robot.”
Cucumber Fruits Detection in Greenhouses Based on Instance Segmentation (2019, methodology): “[25] used Mask RCNN to detect ripe and unripe strawberries and proposed a visual location method to determine picking points.”
A Review of Convolutional Neural Network Applied to Fruit Image Processing (2020, methodology): “They adopted a similar procedure to that proposed by Yu et al. [86] based on ResNet, where the Mask R-CNN and a feature pyramid network (FPN) are the feature extractor.”