Multi-task Neural Networks for QSAR Predictions

By George E. Dahl, Navdeep Jaitly, Ruslan Salakhutdinov. 2014

287 citations recorded by the source corpus.

Although artificial neural networks have occasionally been used for Quantitative Structure-Activity/Property Relationship (QSAR/QSPR) studies in the past, the literature has of late been dominated by other machine learning techniques such as random forests. However, a variety of new neural net techniques along with successful applications in other domains have renewed interest in network approaches. In this work, inspired by the winning team's use of neural networks in a recent QSAR competition, we used an artificial neural network to learn a function that predicts activities of compounds for multiple assays at the same time. We conducted experiments leveraging recent methods for dealing with overfitting in neural networks as well as other tricks from the neural networks literature. We compared our methods to alternative methods reported to perform well on these tasks and found that our neural net methods provided superior performance.

Subfields: Computational Theory and Mathematics, Materials Chemistry, Molecular Biology, Computational Chemistry & Drug Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Materials Chemistry

In Molecular Biology

In Computational Chemistry & Drug Discovery

Citations in the corpus

Convolutional Networks on Graphs for Learning Molecular Fingerprints (2015, methodology): “[3] used circular fingerprints as inputs to an ensemble of neural networks, Gaussian processes, and random forests.”
Gradient-based Hyperparameter Optimization through Reversible Learning (2015, background): “Modern neural net training procedures often employ various heuristics to set learning rate schedules, or set their shape using one or two hyperparameters set by cross-validation (Dahl et al., 2014; Sutskever et al., 2013).”
Molecular graph convolutions: moving beyond fingerprints (2016, methodology): “In particular, multitask neural networks have produced consistent gains relative to baseline models such as random forest and logistic regression [6, 17, 19, 29].”
Opportunities and obstacles for deep learning in biology and medicine (2017, background): “Many studies employed multi-task learning to predict chemical bioactivity [299,302] and drug toxicity [303,421].”
Massively Multitask Networks for Drug Discovery (2015, background): “The winners of this contest later released a technical report that discusses the use of multitask networks for virtual screening (Dahl et al., 2014).”
Using Rule-Based Labels for Weak Supervised Learning: A ChemNet for Transferable Chemical Property Prediction (2017, methodology): “, the challenge of how to convert a sparsely labeled chemical database into a usable form that can be used in a transfer learning approach for existing chemistry DNN models is a non-trivial task.”
Efficient Parametrization of Multi-domain Deep Neural Networks (2018, methodology): “This paradigm is applied to various learning problems from natural language processing [6] and automated drug discovery [7] to speech recognition [12].”
AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery (2015, methodology): “Recently, deep neural networks have been shown to out-perform random forests and SVMs for QSAR and ligand-based virtual screening [9, 10, 11].”
Visualizing and Understanding Atari Agents (2017, background): “, 2010), image captioning (Karpathy & Fei-Fei, 2015), drug discovery (Dahl et al., 2014), and deep reinforcement learning (Mnih et al.”
Deep learning for computational biology (2016, positive): “Multitask architectures allow learning shared features between outputs, thereby improving generalization performance, and markedly reducing the computational cost of model training compared to learning independent models for each trait (Dahl et al , 2014).”
Machine learning approaches to drug response prediction: challenges and recent progress (2020, background): “In fact, despite the recent prevalence of deep neural network (DNN) methods across many areas and industries, including related ﬁ elds, such as computational chemistry 40 – 45 , DNNs have only fairly recently found their way into the drug response prediction.”
Chemception: A Deep Neural Network with Minimal Chemistry Knowledge Matches the Performance of Expert-developed QSAR/QSPR Models (2017, methodology): “These human-expert features are then used by machine learning algorithms for QSAR and QSPR modeling, which in recent years have also included deep neural networks.(9-23) However, we observed that even in the most recent chemistry literature, deep learning in chemistry still falls under this traditio”
Deep learning in biomedicine (2018, methodology): “Early stages of DNNs are often similar to classic low-level models (for example, position-weight matrices for DNA sequences and edge detectors for medical images), but are learned jointly with, and in support of, higher-level outcomes ( Fig.”
How Much Chemistry Does a Deep Neural Network Need to Know to Make Accurate Predictions? (2017, methodology): “Following this trend, several research groups have started using DNN models to predict numerous properties, including activity, [4, 18, 23, 28] toxicity, [20, 31] reactivity, [12, 13], solubility, [17], ADMET properties, [14] and various QM computed energies.”
SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Predicting Chemical Properties (2017, background): “More recently, deep learning (DL) models have also been developed [9, 21, 28, 31].”