Annotated high-throughput microscopy image sets for validation
By Vebjorn Ljosa, Katherine L. Sokolnicki, Anne E Carpenter. 2012
640 citations recorded by the source corpus.
experiments and for providing biological ground truth for evaluating image-analysis algorithms. If an algorithm is sufficiently robust across samples to handle high-throughput experiments, lowthoughput applications also benefit because tolerance to variability in sample preparation and imaging makes the algorithm more likely to generalize to new image sets. Each image set in the BBBC is accompanied by a brief description of its motivating biological application and a set of groundtruth data against which algorithms can be evaluated. The ground truth sets can consist of cell or nucleus counts, foreground and background pixels, outlines of individual objects, or biological labels based on treatment conditions or orthogonal assays (such as a dose-response curve or positiveand negative-control images). We describe canonical ways to measure an algorithm’s performance so that algorithms can be compared against each other fairly, and we provide an optional framework to do so conveniently within CellProfiler. For each image set, we list any published results of which we are aware. The BBBC is freely available from http://www.broadinstitute. org/bbbc/. The collection currently contains 18 image sets, including images of cells (Homo sapiens and Drosophila melanogaster) as well as of whole organisms (Caenorhabditis elegans) assayed in high throughput. We are continuing to extend the collection during the course of our research, and we encourage the submission of additional image sets, ground truth and published results of algorithms.
Subfields: Biophysics, Media Technology, Evaluation & Benchmarks, Medical Imaging
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Biophysics
- Shantanu Singh
- Oren Z. Kraus
- Anne E Carpenter
- Juan C. Caicedo
- Brendan J. Frey
- C. Stringer
- Marius Pachitariu
- Daniel Gerlich
In Media Technology
- Shantanu Singh
- Marius Pachitariu
- C. Stringer
- Juan C. Caicedo
- Oren Z. Kraus
- Brendan J. Frey
- Anne E Carpenter
- Massimo Minervini
In Evaluation & Benchmarks
- C. Stringer
- Marius Pachitariu
- Shantanu Singh
- Oren Z. Kraus
- Brendan J. Frey
- Loic A. Royer
- Joshua D. Batson
- Christoph Sommer
In Medical Imaging
- C. Stringer
- Shantanu Singh
- Marius Pachitariu
- Oren Z. Kraus
- Brendan J. Frey
- Juan C. Caicedo
- Christoph Sommer
- Loic A. Royer
Citations in the corpus
- Finely-grained annotated datasets for image-based plant phenotyping (2016, methodology): “, 2010), or the biologically focused Broad Bioimage Benchmark Collection (Ljosa et al., 2012)).”
- Classifying and segmenting microscopy images with deep multiple instance learning (2015, methodology): “2 Breast cancer screen We used a benchmarking dataset of MFC-7 breast cancer cells available from the Broad Bioimage Benchmark Collection (image set BBBC021v1) (Ljosa et al., 2012). The images contain three channels with fluorescent markers for DNA, actin filaments, and b-tubulin at a resolution of ”
- MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classification (2021, methodology): “For datasets of resolution 28, ResNet-18 wins higher scores than ResNet-50 on most datasets.”
- Cellpose3: one-click image restoration for improved cellular segmentation (2024, positive): “This dataset consists of the 618 images from the ‘cyto’ dataset (540 training images and 68 testing images), described in detail in [20], from various sources [29, 58–61], and an additional 256 new training images.”
- Automating Morphological Profiling with Generic Deep Convolutional Networks (2016, methodology): “This work used the BBBC021 [2] image set, which was the basis for the comparisons presented in Ljosa et al. [9], and is available from the Broad Bioimage Benchmark Collection [10].”
- Semi-convolutional Operators for Instance Segmentation (2018, background): “4), a subset of the Broad Biomedical Benchmark collection [39].”
- CytoGAN: Generative Modeling of Cell Images (2017, methodology): “We employ Generative Adversarial Networks (GANs) [7] to build a generative model of single cell images from the BBBC021 dataset [12].”
- MedViT: A Robust Vision Transformer for Generalized Medical Image Classification (2023, methodology): “TissueMNIST is adapted from the Broad Bioimage Benchmark Collection [70].”
- Machine learning in cell biology – teaching computers to recognize phenotypes (2013, positive): “For objective benchmarking of learning methods in high-content screening, several annotated reference data sets have been published (Ljosa et al., 2012; Rajaram et al., 2012b).”
- CP-CHARM: segmentation-free image classification made accessible (2016, methodology): “Then, we illustrated the usability of our approach on several kinds of biological datasets, namely high-throughput cell-based assay data freely available from the Broad Bioimage Benchmark Collection (BBBC, [18]) and tissue images from the Human Protein Atlas (HPA, [19]).”