WebGPT: Browser-assisted question-answering with human feedback

By Reiichiro Nakano, Jacob Hilton, S. Balaji, Jeff Wu, Ouyang Long, Ouyang Long, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju and 9 more. 2021

1,953 citations recorded by the source corpus.

We fine-tune GPT-3 to answer long-form questions using a text-based web-browsing environment, which allows the model to search and navigate the web. By setting up the task so that it can be performed by humans, we are able to train models on the task using imitation learning, and then optimize answer quality with human feedback. To make human evaluation of factual accuracy easier, models must collect references while browsing in support of their answers. We train and evaluate our models on ELI5, a dataset of questions asked by Reddit users. Our best model is obtained by fine-tuning GPT-3 using behavior cloning, and then performing rejection sampling against a reward model trained to predict human preferences. This model's answers are preferred by humans 56% of the time to those of our human demonstrators, and 69% of the time to the highest-voted answer from Reddit.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Retrieval-Augmented Generation, Large Language Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Luke S. Zettlemoyer
  3. Jianfeng Gao
  4. Percy Liang
  5. Yejin Choi
  6. Graham Neubig
  7. Quoc V. Le
  8. Yoshua Bengio

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Xiaogang Wang
  5. Ming-Hsuan Yang
  6. Jitendra Malik
  7. Andrew Zisserman
  8. Cordelia Schmid

In Retrieval-Augmented Generation

  1. Sebastian Riedel
  2. Danqi Chen
  3. Hannaneh Hajishirzi
  4. Luyu Gao
  5. Wen-tau Yih
  6. Jamie Callan
  7. Jason Aaron Edward Weston
  8. Matei A. Zaharia

In Large Language Models

  1. Luke S. Zettlemoyer
  2. Hinrich Schütze
  3. Yejin Choi
  4. Timo Schick
  5. Jianfeng Gao
  6. Hannaneh Hajishirzi
  7. Samuel R. Bowman
  8. Noah A. Smith

Citations in the corpus