On the Opportunities and Risks of Foundation Models
By Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, R. Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill and 108 more. 2021
6,923 citations recorded by the source corpus.
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.
Subfields: Artificial Intelligence, Large Language Models, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Percy Liang
- Luke S. Zettlemoyer
- Yoshua Bengio
- Sergey Levine
- Quoc V. Le
- Yejin Choi
In Large Language Models
- Luke S. Zettlemoyer
- Yejin Choi
- Jianfeng Gao
- Hannaneh Hajishirzi
- Mohit Bansal
- Hinrich Schütze
- Noah A. Smith
- Quoc V. Le
In Safety, Robustness & Interpretability
- Percy Liang
- Nicholas Carlini
- Matthias Hein
- Florian Tramèr
- Jacob Steinhardt
- Prateek Mittal
- Yejin Choi
- Nicolas Papernot
Citations in the corpus
- Training language models to follow instructions with human feedback (2022, background): “These risks have been extensively documented (Bender et al., 2021; Bommasani et al., 2021; Kenton et al., 2021; Weidinger et al., 2021; Tamkin et al., 2021).”
- Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model (2022, methodology): “The recently released foundation models [8], such as BERT [12], GPT-2 [52], and RoBERTa [37], represent a paradigm shift in which AI systems can be built by pretraining a general class of models at scale and then adapting them for a wide range of downstream tasks through transfer learning.”
- Pathways: Asynchronous Distributed Dataflow for ML (2022, background): “Finally, researchers are beginning to standardize on a set of foundation models (Bommasani and et. al., 2021; Dean, 2021) that are trained on large data at scale and are adaptable to multiple downstream tasks.”
- AI and the Everything in the Whole Wide World Benchmark (2021, background): “Here, the goal is the development of representations — through either unsupervised or supervised means — that can generalize (with minimal fine-tuning) to a wide range of other tasks they were not specifically developed for [e.g. Huh et al., 2016, Du et al., 2020, Bommasani et al., 2021].”
- Finetuned Language Models Are Zero-Shot Learners (2021, background): “, 2020), as well as various NLP applications (Liu et al., 2019a; Corazza et al., 2020; Wang et al., 2021; Reif et al., 2021).”
- Emergent Abilities of Large Language Models (2022, neutral): “As we continue to train ever-larger language models, lowering the scale threshold for emergent abilities will become more important for allowing research on such abilities to available to the community broadly (Bommasani et al., 2021; Ganguli et al., 2022; Liang et al., 2022).”
- OPT: Open Pre-trained Transformer Language Models (2022, background): “There is a growing body of work detailing ethical and social risks from deploying language models with emergent capabilities at scale (Weidinger et al., 2021a; Bommasani et al., 2021; Dinan et al., 2021; Kenton et al., 2021).”
- Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, methodology): “SSL methods are typically used to train foundation models — models that can readily be fine-tuned using a small amount of labeled data on a down-stream task [117].”
- The Flan Collection: Designing Data and Methods for Effective Instruction Tuning (2023, background): “On the contrary, pretrained models that require extensive finetuning become more costly when aggregating over many millions of additional training steps (Wu et al., 2022; Bommasani et al., 2021).”
- Scaling Instruction-Finetuned Language Models (2022, background): “Scaling language models improves performance in many ways but requires substantial computational resources (Kaplan et al., 2020; Brown et al., 2020; Bommasani et al., 2021; Wei et al., 2022a, inter alia ).”
- CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP (2021, background): “The CLIP model is considered as an important foundation model (Bommasani et al., 2021), therefore a plethora of follow-up work has been published (see Appendix Section A.4).”
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances (2022, methodology): “Our method, SayCan, extracts and leverages the knowledge within LLMs in physically-grounded tasks.”
- PromptChainer: Chaining Large Language Model Prompts through Visual Programming (2022, background): “designers, product managers, front-end developers) can now prototype customML functionality with lower effort and less time, as they bypass the otherwise necessary but expensive process of collecting data and training models upfront [2, 8].”
- AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts (2021, background): “Pretrained on billions of inputs from the Internet [29], generative models like GPT-3 can now perform a wide variety of tasks [10], ranging from translation [12], to question answering [47], and even advanced story writing [60].”