A Survey of Deep Learning for Scientific Discovery
By Maithra Raghu, Erica Schmidt. 2020
155 citations recorded by the source corpus.
Over the past few years, we have seen fundamental breakthroughs in core problems in machine learning, largely driven by advances in deep neural networks. At the same time, the amount of data collected in a wide array of scientific domains is dramatically increasing in both size and complexity. Taken together, this suggests many exciting opportunities for deep learning applications in scientific settings. But a significant challenge to this is simply knowing where to start. The sheer breadth and diversity of different deep learning techniques makes it difficult to determine what scientific problems might be most amenable to these methods, or which specific combination of methods might offer the most promising first approach. In this survey, we focus on addressing this central issue, providing an overview of many widely used deep learning models, spanning visual, sequential and graph structured data, associated tasks and different training methods, along with techniques to use deep learning with less data and better interpret these complex models --- two central considerations for many scientific use cases. We also include overviews of the full design process, implementation tips, and links to a plethora of tutorials, research summaries and open-sourced deep learning pipelines and pretrained models, developed by the community. We hope that this survey will help accelerate the use of deep learning across different scientific domains.
Subfields: Artificial Intelligence, Information Systems and Management
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Sergey Levine
- Pushmeet Kohli
- Peter W. Battaglia
- Josh Tenenbaum
- Maithra Raghu
- Charles Blundell
- Demis Hassabis
In Information Systems and Management
Citations in the corpus
- Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics (2020, background): “1 Introduction While the past few years have seen the development of increasingly versatile machine learning systems capable of learning complex tasks [1, 2, 3, 4, 5], catastrophic forgetting remains a core challenge to designing machine learning systems.”
- Artificial intelligence and illusions of understanding in scientific research (2024, unknown): “Google and Microsoft are invested (and investing) in researchers using their AI models 175,176 .”
- Advancing mathematics by guiding human intuition with AI (2021, background): “AI, in particular the field of machine learning 12–14 , offers a collection of techniques that can effectively detect patterns in data and has increasingly demonstrated utility in scientific disciplines 15 .”
- Mapping the Landscape of Artificial Intelligence Applications against COVID-19 (2020, background): “Finally, since this article assumes a certain background knowledge of both Machine Learning and the nature of the SARS-Cov-2 virus, we invite our readers to consult [2] † for further explanation regarding the potential of ML for scientific research, and [3] † for additional information about the vir”