OpenAssistant Conversations - Democratizing Large Language Model Alignment

By Andreas Kopf, Yannic Kilcher, Dimitri von Rutte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, A. Barhoum, Nguyen Minh Duc, Oliver Stanley, Rich'ard Nagyfi and 8 more. 2023

893 citations recorded by the source corpus.

Aligning large language models (LLMs) with human preferences has proven to drastically improve usability and has driven rapid adoption as demonstrated by ChatGPT. Alignment techniques such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) greatly reduce the required skill and domain knowledge to effectively harness the capabilities of LLMs, increasing their accessibility and utility across various domains. However, state-of-the-art alignment techniques like RLHF rely on high-quality human feedback data, which is expensive to create and often remains proprietary. In an effort to democratize research on large-scale alignment, we release OpenAssistant Conversations, a human-generated, human-annotated assistant-style conversation corpus consisting of 161,443 messages in 35 different languages, annotated with 461,292 quality ratings, resulting in over 10,000 complete and fully annotated conversation trees. The corpus is a product of a worldwide crowd-sourcing effort involving over 13,500 volunteers. Models trained on OpenAssistant Conversations show consistent improvements on standard benchmarks over respective base models. We release our code and data under a fully permissive licence.

Subfields: Artificial Intelligence, LLM Agents & Tool Use

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Luke S. Zettlemoyer
  2. Mohit Bansal
  3. Yejin Choi
  4. Noah A. Smith
  5. Percy Liang
  6. Hannaneh Hajishirzi
  7. Jianfeng Gao
  8. Graham Neubig

In LLM Agents & Tool Use

  1. Yejin Choi
  2. Douwe Kiela
  3. Maosong Sun
  4. Edward Grefenstette
  5. Luke S. Zettlemoyer
  6. Hannaneh Hajishirzi
  7. Michael Lewis
  8. Zhiyuan Liu

Citations in the corpus