Learning generative models for protein fold families

By Sivaraman Balakrishnan, Hetunandan Kamisetty, Jaime G. Carbonell, Su-In Lee, Christopher James Langmead. 2011

341 citations recorded by the source corpus.

We introduce a new approach to learning statistical models from multiple sequence alignments (MSA) of proteins. Our method, called GREMLIN (Generative REgularized ModeLs of proteINs), learns an undirected probabilistic graphical model of the amino acid composition within the MSA. The resulting model encodes both the position‐specific conservation statistics and the correlated mutation statistics between sequential and long‐range pairs of residues. Existing techniques for learning graphical models from MSA either make strong, and often inappropriate assumptions about the conditional independencies within the MSA (e.g., Hidden Markov Models), or else use suboptimal algorithms to learn the parameters of the model. In contrast, GREMLIN makes no a priori assumptions about the conditional independencies within the MSA. We formulate and solve a convex optimization problem, thus guaranteeing that we find a globally optimal model at convergence. The resulting model is also generative, allowing for the design of new protein sequences that have the same statistical properties as those in the MSA. We perform a detailed analysis of covariation statistics on the extensively studied WW and PDZ domains and show that our method out‐performs an existing algorithm for learning undirected probabilistic graphical models from MSA. We then apply our approach to 71 additional families from the PFAM database and demonstrate that the resulting models significantly out‐perform Hidden Markov Models in terms of predictive accuracy. Proteins 2011; © 2011 Wiley‐Liss, Inc.

Subfields: Molecular Biology, Generative Models, Computational Biology & Bioinformatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

In Generative Models

In Computational Biology & Bioinformatics

Citations in the corpus

MSA Transformer (2021, methodology): “…have been proposed to efﬁciently ﬁt the MRF, including mean-ﬁeld inference (Morcos et al., 2011), sparse-inverse covariance estimation (Jones et al., 2012), and the current state-of-the-art, pseudolikelihood maximization (Balakrishnan et al., 2011; Ekeberg et al., 2013; Seemayer et al., 2014).”
Robust and accurate prediction of residue–residue interactions across protein interfaces using evolutionary information (2014, methodology): “…matrix ( Marks et al., 2011 ; Morcos et al., 2011 ; Jones et al., 2012 ), more recent studies have shown that a pseudo-likelihood-based approach ( Balakrishnan et al., 2011 ) results in more accurate predictions ( Ekeberg et al., 2013 ; Kamisetty et al., 2013 ) for a range of alignment sizes and…”
Sequence co-evolution gives 3D contacts and structures of protein complexes (2014, methodology): “Computation of evolutionary couplings Inter- and intra-ECs were calculated on the alignment of concatenated sequences using a global probability model of sequence co-evolution, adapted from the method for single proteins29,30,34 using a pseudo-likelihood maximization (PLM)36 rather than mean field a”
Large-scale determination of previously unsolved protein structures using evolutionary information (2015, methodology): “01) was used to learn a global statistical model of the sequences in large families using pseudolikelihood optimization ( Balakrishnan et al., 2011 ).”
Fast and Accurate Multivariate Gaussian Modeling of Protein Families: Predicting Residue Contacts and Protein-Interaction Partners (2014, methodology): “Key to this recent progress are global statistical inference approaches, like direct-coupling analysis (DCA) [8,10] and sparse inverse covariance estimation (PSICOV) [12], and the GREMLIN algorithm based on pseudo-likelihood maximization [11,16].”
Protein contact prediction by integrating joint evolutionary coupling analysis and supervised learning (2013, background): “Contact: j3xu@ttic.edu Supplementary information: Supplementary data are available at Bioinformatics online.”
Transformer protein language models are unsupervised structure learners (2020, methodology): “Various methods have been developed to fit the underlying Markov Random Field, including meanfield DCA (mfDCA) (Morcos et al., 2011), sparse inverse covariance (PSICOV) (Jones et al., 2011) and pseudolikelihood maximization (Balakrishnan et al., 2011; Ekeberg et al., 2013; Seemayer et al., 2014).”
Protein language model embeddings for fast, accurate, alignment-free protein structure prediction (2021, methodology): “The first compute-intensive tasks of state-of-the-art (SOTA) structure prediction methods is the generation of MSAs along with the processing of evolutionary couplings (Balakrishnan et al., 2011; Marks et al., 2011; Seemayer et al., 2014).”
Single Layers of Attention Suffice to Predict Protein Contacts (2020, background): “Potts models are trained to maximize pseu-dolikelihood on alignments of evolutionarily related proteins (Balakrishnan et al., 2011; Ekeberg et al., 2013; Seemayer et al., 2014).”
Unified framework for modeling multivariate distributions in biological sequences (2019, methodology): “…the application is similar, it is not immediately obvious why maximizing the pseudo-likelihood of a regularized MRF results in a more accurate pairwise term (Balakrishnan et al., 2011; Kamisetty et al., 2013), compared to estimating this term by taking the inverse of the shrunken covariance matrix.”
Synthetic protein alignments by CCMgen quantify noise in residue-residue contact prediction (2018, methodology): “…found to work best for residue contact prediction is the pseudo-likelihood approximation, in which we replace the likelihood with the pseudo-likelihood and maximize the regularized log pseudo-likelihood [2, 10, 50], Here, x − i denotes the vector obtained from x by remov-495 ing the i’th component.”
Contact Potential for Structure Prediction of Proteins and Protein Complexes from Potts Model. (2018, background): “…the collective behavior of neurons (41,42), infer gene-interaction networks from experimentally observed transcription proﬁles (43), predict residue-residue contacts from multiple sequence alignments (37,44,45), study protein ﬁtness landscapes (46,47), and infer epistatic effects from ﬁtness (48).”