PEER: A Collaborative Language Model

By Timo Schick, Jane Dwivedi-Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave, Sebastian Riedel. 2022

106 citations recorded by the source corpus.

Textual content is often the output of a collaborative writing process: We start with an initial draft, ask for suggestions, and repeatedly make changes. Agnostic of this process, today's language models are trained to generate only the final result. As a consequence, they lack several abilities crucial for collaborative writing: They are unable to update existing texts, difficult to control and incapable of verbally planning or explaining their actions. To address these shortcomings, we introduce PEER, a collaborative language model that is trained to imitate the entire writing process itself: PEER can write drafts, add suggestions, propose edits and provide explanations for its actions. Crucially, we train multiple instances of PEER able to infill various parts of the writing process, enabling the use of self-training techniques for increasing the quality, amount and diversity of training data. This unlocks PEER's full potential by making it applicable in domains for which no edit histories are available and improving its ability to follow instructions, to write useful comments, and to explain its actions. We show that PEER achieves strong performance across various domains and editing tasks.

Subfields: Artificial Intelligence, Information Systems, Large Language Models, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Large Language Models

In Natural Language Processing

Citations in the corpus

Toolformer: Language Models Can Teach Themselves to Use Tools (2023, background): “, 2021), Wikipedia markup (Schick et al., 2022), or related texts obtained from an information retrieval system (Guu et al.”
Self-Refine: Iterative Refinement with Self-Feedback (2023, methodology): “Learned refiners: PEER (Schick et al., 2022b), Self-critique (Saunders et al.”
Augmented Language Models: a Survey (2023, methodology): “Importantly and apart from PEER, the works mentioned above employ heuristics to call the LM.”
When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labels (2022, background): “Many recent works have studied how to align language models with human feedback (Nakano et al., 2021; Ouyang et al., 2022; Scheurer et al., 2022; Saunders et al., 2022; Schick et al., 2022).”
The rise and potential of large language model based agents: a survey (2025, unknown): “Humans can also choose to participate in feedback at any time, guiding the agent’s learning in the right direction [366].”
Attributed Text Generation via Post-hoc Research and Revision (2022, methodology): “One potential solution is to leverage recent synthetic data generation recipes to train a smaller model (Lee et al., 2021; Schick et al., 2022).”
Generating Sequences by Learning to Self-Correct (2022, background): “Self-correction relates to recent works on editing text, including modeling Wikipedia edits (Reid & Neubig, 2022; Faltings et al., 2021; Schick et al., 2022), which relies on supervised edits, unsupervised methods (Miao et al., 2019; Liu et al., 2020) that perturb sequences with simple operations…”
ChatGPT is not all you need. A State of the Art Review of large Generative AI models (2023, methodology): “PEER : Collaborative language model developed by Meta AI research trained on edit histories to cover the entire writing process [29].”
Contrastive Distillation Is a Sample-Efficient Self-Supervised Loss Policy for Transfer Learning (2022, unknown): “Similarly, hand-designed actor-critic-like transition functions have been applied to text editing and story-writing to iteratively update contexts based on learned transition functions (Yang et al., 2022; Schick et al., 2022).”
MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback (2023, background): “LLMs can be connected with real-world Application Programming Interfaces (APIs), enabling them to actively engage with diverse external tools (Qin et al., 2023b; Parisi et al., 2022; Schick et al., 2023a; Tang et al., 2023; Patil et al., 2023; Song et al., 2023; Hao et al., 2023; Yuan et al., 2023).”
A Survey on Knowledge Distillation of Large Language Models (2024, methodology): “Motivated by the effectiveness of LLMs in generating their own feedback without relying on external models (Schick et al., 2022; Madaan et al., 2023; Saunders et al., 2022), SelFee (Ye et al., 2023) proposes to train a model that has been fine-tuned to continuously revise its own answer until it…”
RARR: Researching and Revising What Language Models Say, Using Language Models (2022, methodology): “One potential solution is to leverage recent synthetic data generation recipes to train a smaller model (Lee et al., 2021; Schick et al., 2022).”
When large language models meet personalization: perspectives of challenges and opportunities (2023, unknown): “PEER[219] LLM LM-Adapted T5 Editions, Citations, Quotes”
Training Language Models with Language Feedback at Scale (2023, unknown): “Liu et al. (2022); Schick et al. (2022) train LMs to refine outputs based on feedback (without finetuning on the refinements), an approach that improves results when incorporated into ILF, as shown in subsequent work to ours (Shi et al., 2022).”
Benchmarking Cognitive Biases in Large Language Models as Evaluators (2023, background): “…such as machine translation (Kocmi and Feder-mann, 2023), summarization (Shen et al., 2023; Gao et al., 2023), code generation (Zhuo, 2023), writing assistance (Schick et al., 2023; Raheja et al., 2023), factual consistency (Cohen et al., 2023; Gekhman et al., 2023; Luo et al., 2023), and more.”
Self-Collaboration Code Generation via ChatGPT (2023, unknown): “A straightforward way to implement collaborative teamwork entails training different models to handle the corresponding subtasks, subsequently conducting joint training to foster mutual understanding of behaviors to assemble them into a team [Schick et al., 2022].”