Deep Supervised and Convolutional Generative Stochastic Network for Protein Secondary Structure Prediction

By Jian Zhou, Olga G. Troyanskaya. 2014

157 citations recorded by the source corpus.

Predicting protein secondary structure is a fundamental problem in protein structure prediction. Here we present a new supervised generative stochastic network (GSN) based method to predict local secondary structure with deep hierarchical representations. GSN is a recently proposed deep learning technique (Bengio & Thibodeau-Laufer, 2013) to globally train deep generative model. We present the supervised extension of GSN, which learns a Markov chain to sample from a conditional distribution, and applied it to protein structure prediction. To scale the model to full-sized, high-dimensional data, like protein sequences with hundreds of amino acids, we introduce a convolutional architecture, which allows efficient learning across multiple layers of hierarchical representations. Our architecture uniquely focuses on predicting structured low-level labels informed with both low and high-level representations learned by the model. In our application this corresponds to labeling the secondary structure state of each amino-acid residue. We trained and tested the model on separate sets of non-homologous proteins sharing less than 30% sequence identity. Our model achieves 66.4% Q8 accuracy on the CB513 dataset, better than the previously reported best performance 64.9% (Wang et al., 2011) for this challenging secondary structure prediction problem.

Subfields: Molecular Biology, Natural Language Processing, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

In Natural Language Processing

In Generative Models

Citations in the corpus

GSNs : Generative Stochastic Networks (2015, methodology): “After early versions of this work were published (Bengio et al., 2014), the GSN framework has been extended and applied to classification problems in several different ways (Goodfellow et al., 2013; Zhou & Troyanskaya, 2014; Zöhrer & Pernkopf, 2014) yielding very interesting results.”
Opportunities and obstacles for deep learning in biology and medicine (2017, background): “In 2014, Zhou and Troyanskaya demonstrated that they could improve Q8 accuracy by using a deep supervised and convolutional generative stochastic network [194].”
A Deep Learning Network Approach to ab initio Protein Secondary Structure Prediction (2014, methodology): “Additionally, a couple of deep learning protein structure predictors have been developed recently, including a multifaceted prediction tool that predicts several protein structural elements in tandem [36] and a predictor that utilizes global protein information [41].”
Attend and Predict: Understanding Gene Regulation by Selective Attention on Chromatin (2017, background): “For instance, multiple recent studies have successfully used deep learning for modeling protein sequences [23, 37] and DNA sequences [1, 20], predicting gene expressions [29], as well as understanding the effects of non-coding variants [38, 27].”
Efficient Second-Order Gradient Boosting for Conditional Random Fields (2015, methodology): “[Zhou and Troyanskaya, 2014] 0.711 SC-GSN with “kick-start” 0.721”
MUFOLD-SS: New Deep Inception-Inside-Inception Networks for Protein Secondary Structure Prediction (2018, methodology): “When generating the sequence profiles, most researchers 4,13 – 15 chose PSI-BLAST, 22 which can reliably generate a good protein sequence profiles.”
Next-Step Conditioned Deep Convolutional Neural Networks Improve Protein Secondary Structure Prediction (2017, methodology): “We focus here on the particular details of our ﬁnal convolutional architecture, which outperforms previously reported attempts at using convolutions for protein secondary structure prediction (Li & Yu, 2016; Zhou & Troyanskaya, 2014; Lin, Lanchantin, & Qi, 2016; Wang et al ., 2016).”
Deep Learning and Its Applications in Biomedicine (2018, methodology): “For example, Lena et al. introduced a deep spatio-temporal learning architecture, achieved an accuracy roughly 10% higher than other methods [181], and Zhou et al. presented a deep supervised and convolutional generative stochastic network, achieving an accuracy of 66.4% [182].”
High Quality Prediction of Protein Q8 Secondary Structure by Diverse Neural Network Architectures (2018, methodology): “ with less than 20% identity of sub-sequences. Figure 1: Q3 (left); Q8 (right) secondary structure spheres for protein 1AKD in CB513 dataset. 2 In 2014 Zhou and Troyanskaya published their GSN method [40] for SSP. To evaluate the performance of their method, they created a new benchmark termed CB613”
Protein Secondary Structure Prediction with Long Short Term Memory Networks (2014, methodology): “The LSTM network has a correct classiﬁcation rate of 0.674, better than current state of the art performance achieved by a generative stochastic network (GSN) [Bengio & Thibodeau-Laufer, 2013; Troyanskaya, 2014] and a conditional neural ﬁeld (CNF) [Lafferty et al., 2001; Peng et al., 2009].”
Protein Secondary Structure Prediction Using Cascaded Convolutional and Recurrent Neural Networks (2016, methodology): “…classiﬁcation problem, which needs to distinguish among the following 8 categories of secondary structure elements: 3 10 − helix (G), α − helix (H), π − helix (I), β − strand (E), β − bridge (B), β − turn (T), bend (S) and loop or irregular (L) [ Zhou and Troyanskaya, 2014; Yaseen and Li, 2014 ] .”