Convolutional LSTM Networks for Subcellular Localization of Proteins

By Søren Kaae Sønderby, Casper Kaae Sønderby, Henrik Nielsen, Ole Winther. 2015

158 citations recorded by the source corpus.

Machine learning is widely used to analyze biological sequence data. Non-sequential models such as SVMs or feed-forward neural networks are often used although they have no natural way of handling sequences of varying length. Recurrent neural networks such as the long short term memory (LSTM) model on the other hand are designed to handle sequences. In this study we demonstrate that LSTM networks predict the subcellular location of proteins given only the protein sequence with high accuracy (0.902) outperforming current state of the art algorithms. We further improve the performance by introducing convolutional filters and experiment with an attention mechanism which lets the LSTM focus on specific parts of the protein. Lastly we introduce new visualizations of both the convolutional filters and the attention mechanisms and show how they can be used to extract biological relevant knowledge from the LSTM networks.

Subfields: Biophysics, Molecular Biology, Model Architectures, Computational Biology & Bioinformatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

1. Casey S. Greene
2. G. Way
3. Brett K. Beaulieu-Jones
4. Anne E Carpenter
5. D. Ellis
6. Yanjun Qi
7. Jinbo Xu
8. Colin Raffel

In Molecular Biology

1. Kevin Kaichuang Yang
2. Zachary Wu
3. Casey S. Greene
4. Frances H. Arnold
5. Yifan Peng
6. Brett K. Beaulieu-Jones
7. Byunghan Lee
8. G. Way

In Model Architectures

1. Zhiyong Lu
2. Jiabao Guo
3. Anne E Carpenter
4. D. Ellis
5. Yanjun Qi
6. Jinbo Xu
7. Gang Liu
8. Colin Raffel

In Computational Biology & Bioinformatics

1. Kevin Kaichuang Yang
2. Zachary Wu
3. Frances H. Arnold
4. Brett K. Beaulieu-Jones
5. Zhiyong Lu
6. Anne E Carpenter
7. D. Ellis
8. Yanjun Qi

Citations in the corpus

• Opportunities and obstacles for deep learning in biology and medicine (2017, background): “Recurrent neural networks show good performance for homology and protein family identification [245,246].”
• A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists (2017, methodology): “Stacked autoencoders, DBNs, and LSTM could predict protein structures directly from their sequences with much better accuracy than conventional methods (Heffernan et al., 2015; Lyons et al., 2014; Sønderby et al., 2015; Spencer et al., 2015).”
• Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems (2015, methodology): “Applying our feedforward model to these tasks is somewhat disingenuous because they are commutative and therefore may be easier to solve with a model which ignores temporal order.”
• Deep learning in bioinformatics (2016, methodology): “Recurrent neural networks Protein structure [88–90] Brain decoding [168] Gene expression regulation [105–107] Anomaly classification [177, 178] Protein classification [109, 110]”
• Machine learning in protein engineering (2018, methodology): “Visualization of one convolutional filter from a model trained to predict the intracellular location of proteins (Sønderby et al., 2015).”
• Neural Machine Translation: A Review (2019, background): “, 2016), bioinformatics (Sønderby et al., 2015), text summarization (Rush et al.”
• Bidirectional LSTM with attention mechanism and convolutional layer for text classification (2019, background): “The literature [32] demonstrated that LSTM networks predict the subcellular location of proteins given only the protein sequence with high accuracy outperforming current state-of-the-art algorithms.”