Massively Multitask Networks for Drug Discovery

By Bharath Ramsundar, Steven M. Kearnes, Patrick F. Riley, Dale R. Webster, David E. Konerding, Vijay S. Pande. 2015

499 citations recorded by the source corpus.

Massively multitask neural architectures provide a learning framework for drug discovery that synthesizes information from many distinct biological sources. To train these architectures at scale, we gather large amounts of data from public sources to create a dataset of nearly 40 million measurements across more than 200 biological targets. We investigate several aspects of the multitask framework by performing a series of empirical studies and obtain some interesting results: (1) massively multitask networks obtain predictive accuracies significantly better than single-task methods, (2) the predictive power of multitask networks improves as additional tasks and data are added, (3) the total amount of data and the total number of tasks both contribute significantly to multitask improvement, and (4) multitask networks afford limited transferability to tasks not in the training set. Our results underscore the need for greater data sharing and further algorithmic innovation to accelerate the drug discovery process.

Subfields: Biophysics, Computational Theory and Mathematics, Materials Chemistry, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Computational Theory and Mathematics

In Materials Chemistry

In Representation & Self-Supervised Learning

Citations in the corpus

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, methodology): “Multitask learning (Caruana, 1997) has been successfully applied to multiple domains, including NLP, speech processing, drug discovery and many others (Collobert and Weston, 2008; Deng et al., 2013; Ramsundar et al., 2015; Maurer et al., 2016; Ruder, 2017; Kaiser et al., 2017).”
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (2016, neutral): “These clients rely on TensorFlow for research and production, with tasks as diverse as running inference for computer vision models on mobile phones to large-scale training of deep neural networks with hundreds of billions of parameters on hundreds of billions of example records using many hundreds ”
Concrete Problems in AI Safety (2016, neutral): “A I] 2 1 Ju n”
Molecular graph convolutions: moving beyond fingerprints (2016, methodology): “The pyramidal MTNN had two hidden layers (2000 and 100 units, respectively) with rectiﬁed linear activations, and was trained for 20 million steps using the SGD optimizer with batch size 128 and a learning rate of 0.0003, with periodic checkpointing.”
Learning Graph-Level Representation for Drug Discovery (2017, methodology): “2017) propose atomic neural network to predicate the binding free energy of a subset of protein-ligand complexes found in the PDBBind dataset.(Ramsundar et al. 2015) applied Massively multitask neural architectures to synthesize information from many distinct biological sources.”
Modeling Industrial ADMET Data with Multitask Networks (2016, methodology): “Previous attempts to quantify relatedness have focused on shared compounds or correlations between labels [Erhan et al., 2006; Ramsundar et al., 2015].”
Using Rule-Based Labels for Weak Supervised Learning: A ChemNet for Transferable Chemical Property Prediction (2017, methodology): “Recent work have also started using deep neural networks (DNN) [9, 19, 24, 27], that are on average, typically more accurate than traditional ML models [12, 14].”
AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery (2015, methodology): “That work was followed by Untherhiner et al. [11, 12] and Ramsundar et al. [13] that demonstrated the MT-DNN technique scales to large biochemical databases such as PubChem Bioassays [14] and ChEMBL [15].”
Deep Learning for Health Informatics (2017, neutral): “other methods that process large datasets [49].”