Effective gene expression prediction from sequence by integrating long-range interactions

By Žiga Avsec, Vikram Agarwal, D. Visentin, J. Ledsam, Agnieszka Grabska-Barwinska, Kyle R. Taylor, Yannis Assael, John M. Jumper, Pushmeet Kohli, David R. Kelley. 2021

1,222 citations recorded by the source corpus.

How noncoding DNA determines gene expression in different cell types is a major unsolved problem, and critical downstream applications in human genetics depend on improved solutions. Here, we report substantially improved gene expression prediction accuracy from DNA sequences through the use of a deep learning architecture, called Enformer, that is able to integrate information from long-range interactions (up to 100 kb away) in the genome. This improvement yielded more accurate variant effect predictions on gene expression for both natural genetic variants and saturation mutagenesis measured by massively parallel reporter assays. Furthermore, Enformer learned to predict enhancer–promoter interactions directly from the DNA sequence competitively with methods that take direct experimental data as input. We expect that these advances will enable more effective fine-mapping of human disease associations and provide a framework to interpret cis-regulatory evolution. By using a new deep learning architecture, Enformer leverages long-range information to improve prediction of gene expression on the basis of DNA sequence.

Subfields: Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

Citations in the corpus

State-specific protein–ligand complex structure prediction with a multiscale deep generative model (2022, neutral): “…modelling include (1) auto-regressive models as widely adopted in transformer 24 networks for sequence data such as natural language and genomics 25,26 based on a sequential process and (2) diffusion-based generative models that leverage a stochastic process to generate data by sampling from a…”
Long Range Language Modeling via Gated State Spaces (2022, background): “…range of tasks, starting in NLP [Vaswani et al., 2017, Devlin et al., 2019, Radford et al., 2019, Liu et al., 2019], then moving to other classical machine learning areas such as computer vision [Dosovitskiy et al., 2021] and now to the physical sciences [Avsec et al., 2021, Jumper et al., 2021].”
DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome (2023, background): “…superiority in a range of downstream applications, including promoter prediction (Le et al., 2022; Zhang et al., 2022), gene expression prediction (Avsec et al., 2021), DNA methylation prediction (Jin et al., 2022), chromatin state analysis (Lee et al., 2022), promoter-enhancer interaction…”
Gene regulatory network inference in the era of single-cell multi-omics (2023, background): “The large scale of the data sets makes it increasingly possible to train deep learning methods to predict gene expression from sequencing data 175,201,202 .”
Ensembl 2024 (2023, methodology): “We have developed an Ensembl VEP extension to integrate pre-calculated scores from DeepMind’s En-former algorithm ( 33 ), which predicts when variants are likely to impact gene regulation.”
Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning (2022, background): “For instances, DNA motifs within exon sequences which contributes towards protein structure, gene expression, and genetic traits [3, 21, 40].”
Simple and Effective Masked Diffusion Language Models (2024, methodology): “The dataset is based on the splits used in Avsec et al. [3]: the training split comprises of 35 billion tokens covering the human genome.”
CADD v1.7: using protein language models, regulatory CNNs and other nucleotide-level scores to improve genome-wide variant predictions (2024, methodology): “From these and other modeling efforts, open-chromatin information like DNase-seq or A T AC-seq seem to provide good approximations for predicting regulatory variant effects, for example the activity measured from massively parallel reporter assays (MPRAs) ( 48–51 ).”
HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution (2023, methodology): “A canonical example is the Enformer, which uses dilation and downsampling to reach context lengths of 100k nucleotides to predict gene expression tracks [1].”
The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics (2024, methodology): “When compared with DNABERT, HyenaDNA-32kb and Enformer, our Multispecies 2.5B model achieved the highest performance across all tasks (Fig.”