Molecular Transformer: A Model for Uncertainty-Calibrated Chemical Reaction Prediction

By Philippe Schwaller, Teodoro Laino, Théophile Gaudin, Peter Bolgar, C. Bekas, Alpha Albert Lee. ACS Central Science, 2018

981 citations recorded by the source corpus.

Organic synthesis is one of the key stumbling blocks in medicinal chemistry. A necessary yet unsolved step in planning synthesis is solving the forward problem: Given reactants and reagents, predict the products. Similar to other work, we treat reaction prediction as a machine translation problem between simplified molecular-input line-entry system (SMILES) strings (a text-based representation) of reactants, reagents, and the products. We show that a multihead attention Molecular Transformer model outperforms all algorithms in the literature, achieving a top-1 accuracy above 90% on a common benchmark data set. Molecular Transformer makes predictions by inferring the correlations between the presence and absence of chemical motifs in the reactant, reagent, and product present in the data set. Our model requires no handcrafted rules and accurately predicts subtle chemical transformations. Crucially, our model can accurately estimate its own uncertainty, with an uncertainty score that is 89% accurate in terms of classifying whether a prediction is correct. Furthermore, we show that the model is able to handle inputs without a reactant–reagent split and including stereochemistry, which makes our method universally applicable.

Subfields: Computational Theory and Mathematics, Inorganic Chemistry, Materials Chemistry

View Paper: https://www.semanticscholar.org/paper/de20b6488e148a19ae6c63defbfca8a6373e4110

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Inorganic Chemistry

In Materials Chemistry

Citations in the corpus

Artificial Intelligence (AI) Applications in Drug Discovery and Drug Delivery: Revolutionizing Personalized Medicine (2024, background): “This class of models is flexible, non-rule-based, and scalable [124,125].”
Enabling late-stage drug diversification by high-throughput experimentation with geometric deep learning (2023, background): “[31, 32] Other studies have shown that learning activation energies of transition-state geometries yields accurate predictions for competing reaction outcomes.”
Artificial intelligence in drug discovery: recent advances and future perspectives (2021, positive): “With regard to template-free forward synthesis prediction, natural language processing approaches based on transformer [184] or recurrent neural network architectures [185] are also becoming popular.”
Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development (2021, methodology): “The backend of IBMRXNretrosynthetic analysis is amolecular transformermodel (Schwaller et al. 2019).”
A Survey of Transformers (2021, methodology): “In addition to language related applications, Transformer has also been adopted in CV [13, 33, 94], audio processing [15, 31, 41] and even other disciplines, such as chemistry [113] and life sciences [109].”
Unsupervised Attention-Guided Atom-Mapping (2020, methodology): “Molecular transformer models, for instance, are the state-of-the-art for forward reaction prediction tasks, achieving an accuracy higher than 90% (Vaswani et al., 2017; Schwaller et al., 2019a; Schwaller & Laino, 2019).”
Geometric deep learning on molecular representations (2021, background): “Transformers String notation (encoded as a graph) Prediction of properties in drug discovery [51], synthesis planning [46, 52], prediction of reaction yields [53], generative molecular design [54] and protein structure prediction [6, 7]”
Leveraging large language models for predictive chemistry (2024, positive): “We have to mention that we did not optimize the fine-tuning of the GPT-3 model, that is, we did not try to optimize how a sentence is presented to the model; one can envision that specific tokenization can have better results for chemical sentences 9,16,27,28 .”