Evaluation of methods for modeling transcription-factor sequence specificity

By Matthew T. Weirauch, Atina G. Coté, Raquel Norel, Matti Annala, Yue Zhao, Todd Riley, J. Sáez-Rodríguez, T. Cokelaer, A. Vedenko, Shaheynoor Talukder and 5 more. 2013

375 citations recorded by the source corpus.

Genomic analyses often involve scanning for potential transcription factor (TF) binding sites using models of the sequence specificity of DNA binding proteins. Many approaches have been developed to model and learn a protein's DNA-binding specificity, but these methods have not been systematically compared. Here we applied 26 such approaches to in vitro protein binding microarray data for 66 mouse TFs belonging to various families. For nine TFs, we also scored the resulting motif models on in vivo data, and found that the best in vitro–derived motifs performed similarly to motifs derived from the in vivo data. Our results indicate that simple models based on mononucleotide position weight matrices trained by the best methods perform similarly to more complex models for most TFs examined, but fall short in specific cases (<10% of the TFs examined here). In addition, the best-performing motifs typically have relatively low information content, consistent with widespread degeneracy in eukaryotic TF sequence preferences.

Subfields: Genetics, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Genetics

In Molecular Biology

Citations in the corpus

SeqGL Identifies Context-Dependent Binding Signals in Genome-Wide Regulatory Element Maps (2015, methodology): “We note that gkm-SVM has a procedure for producing PSSMs from the top ranked k-mers in the model and reports three motifs per TF ChIP-seq experiments.”
Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning (2015, methodology): “Methods were evaluated using the Pearson correlation between the predicted and actual probe intensities, and values from the area under the receiver operating characteristic (ROC) curve (AUC) computed by setting high-intensity probes as positives and the remaining probes as negatives 17 .”
Enhanced Regulatory Sequence Prediction Using Gapped k-mer Features (2014, methodology): “However, in a recent comparison of methods for modeling transcription factor sequence specificity, full k-mer methods outperformed the di-nucleotide approaches when applied to PBM data [26].”
Distinct structural bases for sequence-specific DNA binding by mammalian BEN domain proteins (2022, positive): “Protein-binding microarray (PBM) analysis PBMassayswereperformedfollowingtheproceduredescribedpre-viously (Lam et al. 2011; Weirauch et al. 2013).”
Deep learning of the regulatory grammar of yeast 5′ untranslated regions from 500,000 random sequences (2017, neutral): “The small number of nucleotides typically involved in the binding of proteins to DNA and RNA (4 – 8 nt) or in forming secondary structures (Weirauch et al. 2013) suggests that functional biological motifs will occur often and in a wide range of contexts within these random 5 ′ UTRs.”
JASPAR 2016: a major expansion and update of the open-access database of transcription factor binding profiles (2015, background): “Even though such models perform well overall (22), it has been recurrently shown that some TFs significantly benefit from more complex models when predicting TFBSs (2,23,24).”
UniPROBE, update 2015: new tools and content for the online database of protein-binding microarray data on protein–DNA interactions (2014, methodology): “BEEML-PBM was one of the top two algorithms in the DREAM5 challenge (18) and provided PWMs with better performance The BEEML-PBM data display format is essentially the same, but because k-mers and enrichment scores are not utilized in this algorithm, an IUPAC consensus sequence derived from the PWM…”
A comparative analysis of transcription factor binding models learned from PBM, HT-SELEX and ChIP data (2014, result): “poorer results compared with in vitro binding prediction (10,11).”