Benchmarking atlas-level data integration in single-cell genomics

By Malte D. Luecken, Maren Büttner, Kridsadakorn Chaichoompu, A. Danese, Marta Interlandi, MF Mueller, D. Strobl, Luke Zappia, M. Dugas, M. Colomé-Tatché and 1 more. Nature Methods, 2020

1,189 citations recorded by the source corpus.

Single-cell atlases often include samples that span locations, laboratories and conditions, leading to complex, nested batch effects in data. Thus, joint analysis of atlas datasets requires reliable data integration. To guide integration method choice, we benchmarked 68 method and preprocessing combinations on 85 batches of gene expression, chromatin accessibility and simulation data from 23 publications, altogether representing >1.2 million cells distributed in 13 atlas-level integration tasks. We evaluated methods according to scalability, usability and their ability to remove batch effects while retaining biological variation using 14 evaluation metrics. We show that highly variable gene selection improves the performance of data integration methods, whereas scaling pushes methods to prioritize batch removal over conservation of biological variation. Overall, scANVI, Scanorama, scVI and scGen perform well, particularly on complex integration tasks, while single-cell ATAC-sequencing integration performance is strongly affected by choice of feature space. Our freely available Python module and benchmarking pipeline can identify optimal data integration methods for new data, benchmark new methods and improve method development. This benchmarking study compares 16 methods for integrating complex single-cell RNA and ATAC datasets and provides a guide to method choice.

Subfields: Biophysics, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/d88158745b69f5732397175389101e2d98799c00

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Molecular Biology

Citations in the corpus

Best practices for single-cell analysis across modalities (2023, methodology): “Once TFs of interest have been identified, scATAC-seq data allow for additional validation of the TF impact through footprinting, which indicates whether the TF is binding in the given cell cluster.”
Mapping single-cell data to reference atlases by transfer learning (2021, methodology): “Reference mapping performance was assessed using ten metrics recently established to evaluate data-integration performance 7 in terms of removal of batch effects and preservation of biological variation.”
Large Scale Foundation Model on Single-cell Transcriptomics (2023, neutral): “Note that merely aligning the read depth would not eliminate the entire batch effect since batch effects can involve other variations such as donor gender, experiment treatment, cell cycle and so on 32 .”
Computational principles and challenges in single-cell data integration (2021, background): “31 , where the researchers compared the performance of 38 methods using increasingly complex datasets and a range of metrics that included scalability, usability and ability to remove batch effects while retaining biological variation.”