Image-based plant phenotyping with incremental learning and active contours

By Massimo Minervini, Mohammed M. Abdelsamea, Sotirios A. Tsaftaris. 2014

135 citations recorded by the source corpus.

Plant phenotyping investigates how a plant's genome, interacting with the environment, affects the observable traits of a plant (phenome). It is becoming increasingly important in our quest towards efficient and sustainable agriculture. While sequencing the genome is becoming increasingly efficient, acquiring phenotype information has remained largely of low throughput. Current solutions for automated image-based plant phenotyping, rely either on semi-automated or manual analysis of the imaging data, or on expensive and proprietary software which accompanies costly hardware infrastructure. While some attempts have been made to create software applications that enable the analysis of such images in an automated fashion, most solutions are tailored to particular acquisition scenarios and restrictions on experimental design. In this paper we propose and test, a method for the segmentation and the automated analysis of time-lapse plant images from phenotyping experiments in a general laboratory setting, that can adapt to scene variability. The method involves minimal user interaction, necessary to establish the statistical experiments that may follow. At every time instance (i.e., a digital photograph), it segments the plants in images that contain many specimens of the same species. For accurate plant segmentation we propose a vector valued level set formulation that incorporates features of color intensity, local texture, and prior knowledge. Prior knowledge is incorporated using a plant appearance model implemented with Gaussian mixture models, which utilizes incrementally information from previously segmented instances. The proposed approach is tested on Arabidopsis plant images acquired with a static camera capturing many subjects at the same time. Our validation with ground truth segmentations and comparisons with state-of-the-art methods in the literature shows that the proposed method is able to handle images with complicated and changing background in an automated fashion. An accuracy of 96.7% (dice similarity coefficient) was observed, which was higher than other methods used for comparison. While here it was tested on a single plant species, the fact that we do not employ shape driven models and we do not rely on fully supervised classification (trained on a large dataset) increases the ease of deployment of the proposed solution for the study of different plant species in a variety of laboratory settings. Our solution will be accompanied by an easy to use graphical user interface and, to facilitate adoption, we will make the software available to the scientific community.

Subfields: Computer Vision and Pattern Recognition, Ecology, Environmental Engineering, Plant Science, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Ecology

In Environmental Engineering

In Plant Science

In Meta & Continual Learning

Citations in the corpus

Finely-grained annotated datasets for image-based plant phenotyping (2016, methodology): “For Arabidopsis, we used the approach based on active contours described by Minervini et al. (2014), while for tobacco, a simple color-based approach for plant segmentation was used.”
An interactive tool for semi-automated leaf annotation (2015, methodology): “applications or as part of a phenotyping analysis by integrating also phenotype extraction software [18, 24].”
Utilizing machine learning approaches to improve the prediction of leaf counts and individual leaf segmentation of rosette plant images (2015, background): “The leading approach [12] showed inaccuracies in resolving leaf overlaps in the training and test data sets (data sets and approaches for data analysis are described in detail in [15] and [10]).”
Learning to Count Leaves in Rosette Plants (2015, methodology): “Image data: We use three datasets, namely A1, A2, and A3, consisting of images showing top views on individual plants provided by the LCC CVPPP 2015 challenge organizers [25, 32].”
Annotated Image Datasets of Rosette Plants (2014, methodology): “In [15] we propose a solution to analyze images from plant phenotyping experiments, with particular focus on plant segmentation.”
segmentation in plant phenotyping: a collation study. Machine Vision and Applications (2017, methodology): “For Arabidopsis, we used the approach based on active contours described in [36], while for tobacco, a simple color-based approach for plant segmentation was used.”
3-D Histogram-Based Segmentation and Leaf Detection for Rosette Plants (2014, methodology): “To enhance the robustness of segmentation approaches texture features can be utilized [10], additionally active contours are used to improve segmentation [11].”
Leaf segmentation in plant phenotyping: a collation study (2015, methodology): “Possibly higher performance (and lower variability) can be obtained with methods relying on learned classifiers and active contour models [36].”
Deep Plant Phenomics: A Deep Learning Platform for Complex Plant Phenotyping Tasks (2017, methodology): “…bounding box coordinates supplied in Pascal VOC format (Everingham et al., 2010), regression and classiﬁcation labels, CSV ﬁles, directories, as well as loaders for the International Plant Phenotyping Network (IPPN) phenotyping dataset (Minervini et al., 2014) and other plant phenotyping datasets.”
Learning computationally efficient approximations of complex image segmentation metrics (2013, methodology): “We test our methodology on a dataset of plant image segmentations obtained from image based plant phenotyping experiments [10], examples of which are shown in Fig.”
A Review of Imaging Techniques for Plant Phenotyping (2014, methodology): “[98] proposed a segmentation method for plants in image-based phenotyping experiments.”
Lights, camera, action: high-throughput plant phenotyping is ready for a close-up. (2015, background): “org/) that have successfully stirred the community and encouraged computer scientists to work on common plant datasets [38,39].”