Continuous Distributed Representation of Biological Sequences for Deep Proteomics and Genomics

By Ehsaneddin Asgari, Mohammad R. K. Mofrad. 2015

389 citations recorded by the source corpus.

We introduce a new representation and feature extraction method for biological sequences. Named bio-vectors (BioVec) to refer to biological sequences in general with protein-vectors (ProtVec) for proteins (amino-acid sequences) and gene-vectors (GeneVec) for gene sequences, this representation can be widely used in applications of deep learning in proteomics and genomics. In the present paper, we focus on protein-vectors that can be utilized in a wide array of bioinformatics investigations such as family classification, protein visualization, structure prediction, disordered protein identification, and protein-protein interaction prediction. In this method, we adopt artificial neural network approaches and represent a protein sequence with a single dense n-dimensional vector. To evaluate this method, we apply it in classification of 324,018 protein sequences obtained from Swiss-Prot belonging to 7,027 protein families, where an average family classification accuracy of 93%±0.06% is obtained, outperforming existing family classification methods. In addition, we use ProtVec representation to predict disordered proteins from structured proteins. Two databases of disordered sequences are used: the DisProt database as well as a database featuring the disordered regions of nucleoporins rich with phenylalanine-glycine repeats (FG-Nups). Using support vector machine classifiers, FG-Nup sequences are distinguished from structured protein sequences found in Protein Data Bank (PDB) with a 99.8% accuracy, and unstructured DisProt sequences are differentiated from structured DisProt sequences with 100.0% accuracy. These results indicate that by only providing sequence data for various proteins into this model, accurate information about protein structure can be determined. Importantly, this model needs to be trained only once and can then be applied to extract a comprehensive set of information regarding proteins of interest. Moreover, this representation can be considered as pre-training for various applications of deep learning in bioinformatics. The related data is available at Life Language Processing Website: http://llp.berkeley.edu and Harvard Dataverse: http://dx.doi.org/10.7910/DVN/JMFHTN.

Subfields: Molecular Biology, Representation & Self-Supervised Learning, Computational Biology & Bioinformatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

In Representation & Self-Supervised Learning

In Computational Biology & Bioinformatics

Citations in the corpus

Mol2vec: Unsupervised Machine Learning Approach with Chemical Intuition (2018, positive): “This concept was already adopted to protein sequences (ProtVec) for the classification of protein families and disordered proteins 15 but was not applied to molecules so far.”
BERTology Meets Biology: Interpreting Attention in Protein Language Models (2020, positive): “Early work in this direction applied the Skip-gram model [50] to construct continuous embeddings from protein sequences for downstream tasks [7].”
Modeling aspects of the language of life through transfer-learning protein sequences (2019, positive): “We compared the performance of the SeqVec embeddings to state-of-the-art methods using evolutionary information, and also to a popular embedding tool for protein sequences originating from the Word2vec approach, namely ProtVec [42].”
Learning protein sequence embeddings using information from structure (2019, neutral): “Current work in protein sequence embeddings has primarily been focused on unsupervised k-mer co-occurence approaches to learn fixed sized vector representations [7, 8] based on similar methods in NLP [9, 10, 11, 12].”
ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Deep Learning and High Performance Computing (2020, positive): “While outperforming uncontextualized (ProtVec [60]) as well as existing, LSTM-based LMs (SeqVec [17]), all LMs trained here still fall short compared to methods using evolutionary information.”
Deep learning for computational biology (2016, positive): “Skip-gram neural networks have been applied to learn low-dimensional representations of protein sequences and improve protein classification (Asgari & Mofrad, 2015).”
Protein Secondary Structure Prediction Using Cascaded Convolutional and Recurrent Neural Networks (2016, positive): “In our experiments, multiscale CNN layers with kernel size 3 , 7 , and 11 are used to extract local contexts given the window size of “a biological word” in [ Asgari and Mofrad, 2015 ] as a reference.”
Machine learning in protein engineering (2018, neutral): “However, more sophisticated continuous vector encodings of sequences can be learned from patterns in unlabeled sequences (Asgari & Mofrad, 2015; Mazzaferro, 2017; Ng, 2017; Kimothi et al., 2016; Yang et al., 2018a; Schwartz et al., 2018).”
Machine-learning-guided directed evolution for protein engineering (2018, positive): “However, more sophisticated continuous vector encodings of sequences can be learned from patterns in unlabeled sequences 52,70–76 or from structural information 77 .”
PhosContext2vec: a distributed representation of residue-level sequence contexts and its application to general and kinase-specific phosphorylation site prediction (2018, positive): “At the final step, the prot2vec add representation was effectively equivalent to ProtVec that was originally introduced by Asgari and Mofrad 13 .”
Modeling the language of life – Deep Learning Protein Sequences (2019, neutral): “Panel C 7 compared per-protein predictions for subcellular localization between top methods (numbers taken from 8 DeepLoc [46] and embeddings based on single sequences (ProtVec [47] and our SeqVec).”
Backdoor attacks against learning systems (2017, neutral): “For example, besides its general-purpose implementations, Word2Vec [28], a PLM to reconstruct linguistic contexts of words, has a number of domain-speciﬁc versions (e.g., BioVec [29]).”