SMILES Transformer: Pre-trained Molecular Fingerprint for Low Data Drug Discovery
By Shion Honda, Shoi Shi, Hiroki R. Ueda. 2019
269 citations recorded by the source corpus.
In drug-discovery-related tasks such as virtual screening, machine learning is emerging as a promising way to predict molecular properties. Conventionally, molecular fingerprints (numerical representations of molecules) are calculated through rule-based algorithms that map molecules to a sparse discrete space. However, these algorithms perform poorly for shallow prediction models or small datasets. To address this issue, we present SMILES Transformer. Inspired by Transformer and pre-trained language models from natural language processing, SMILES Transformer learns molecular fingerprints through unsupervised pre-training of the sequence-to-sequence language model using a huge corpus of SMILES, a text representation system for molecules. We performed benchmarks on 10 datasets against existing fingerprints and graph-based methods and demonstrated the superiority of the proposed algorithms in small-data settings where pre-training facilitated good generalization. Moreover, we define a novel metric to concurrently measure model accuracy and data efficiency.
Subfields: Computational Theory and Mathematics, Materials Chemistry, Molecular Biology, Computational Chemistry & Drug Discovery
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Theory and Mathematics
- Tianfan Fu
- Shengchao Liu
- Jimeng Sun
- Yuanqi Du
- Seyone Chithrananda
- Gabriel Grand
- Bharath Ramsundar
- Chaopeng Shen
In Materials Chemistry
- Shengchao Liu
- Tianfan Fu
- Jimeng Sun
- Yuanqi Du
- Seyone Chithrananda
- Gabriel Grand
- Bharath Ramsundar
- Chaopeng Shen
In Molecular Biology
- Tianfan Fu
- Seyone Chithrananda
- Gabriel Grand
- Shengchao Liu
- Jimeng Sun
- Bharath Ramsundar
- Guillem Cucurull
- Yuanqi Du
In Computational Chemistry & Drug Discovery
- Shengchao Liu
- Jimeng Sun
- Seyone Chithrananda
- Bharath Ramsundar
- Tianfan Fu
- Gabriel Grand
- Yuanqi Du
- Chaopeng Shen
Citations in the corpus
- Galactica: A Large Language Model for Science (2022, background): “Beyond scientific text, Transformers for protein sequences and SMILES have shown potential for learning natural representations (Rives et al., 2021; Honda et al., 2019; Irwin et al., 2021; Nijkamp et al., 2022; Lin et al., 2022b).”
- ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction (2020, background): “Some recent work has pretrained transformers for molecular property prediction and reported promising results [21, 22].”
- MolGenSurvey: A Systematic Survey in Machine Learning Models for Molecule Design (2022, methodology): “ARs have been widely-used in generation tasks, including recurrent neural network (RNN) [80], [81], PixelCNN [82], WaveNet [83], Transformer [37] and BERT [38].”