Decontextualization: Making Sentences Stand-Alone

By Eunsol Choi, Jennimaria Palomaki, Matthew Lamm, Tom Kwiatkowski, Dipanjan Das, Michael Collins. 2021

129 citations recorded by the source corpus.

Abstract Models for question answering, dialogue agents, and summarization often interpret the meaning of a sentence in a rich context and use that meaning in a new context. Taking excerpts of text can be problematic, as key pieces may not be explicit in a local window. We isolate and define the problem of sentence decontextualization: taking a sentence together with its context and rewriting it to be interpretable out of context, while preserving its meaning. We describe an annotation procedure, collect data on the Wikipedia corpus, and use the data to train models to automatically decontextualize sentences. We present preliminary studies that show the value of sentence decontextualization in a user-facing task, and as preprocessing for systems that perform document understanding. We argue that decontextualization is an important subtask in many downstream applications, and that the definitions and resources provided can benefit tasks that operate on sentences that occur in a richer context.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Measuring Attribution in Natural Language Generation Models (2021, background): “A key idea in our model of meaning in context is the notion of explicatures (Carston 1988; Wilson and Sperber 2004; Choi et al. 2021).”
Towards Continual Knowledge Learning of Language Models (2021, methodology): “(2) We make the selected sentence stand-alone from the article through the decontextualization model provided by Choi et al. (2021).”
Evaluating Factuality in Text Simplification (2022, methodology): “, 2020), or performing decontextualization (Choi et al., 2021) before simplifying.”
Multi-Modal Answer Validation for Knowledge-Based VQA (2021, methodology): “Then, we decontextualize (Choi et al. 2021) each sentence in the Wikipedia pool for better knowledge quality.”
RARR: Researching and Revising What Language Models Say, Using Language Models (2022, methodology): “To improve its accuracy, we first decontextualize (Choi et al., 2021) each sentence in y based on the entire context of y before measuring its auto-AIS score.”
Attributed Text Generation via Post-hoc Research and Revision (2022, methodology): “To improve its accuracy, we ﬁrst decontextualize (Choi et al., 2021) each sentence in y based on the entire context of y before measuring its auto-AIS score.”
CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning (2021, methodology): “as well as sentence rewriting in a document (Choi et al., 2021).”
Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model (2022, methodology): “To measure the accuracy of the decontextualizing markup, we apply the prompt-based teacher and the ﬁne-tuned student models to a manually decontex-tualized dataset, in which references are replaced inline rather than annotated with markup (Choi et al., 2021).”
LLMs Get Lost In Multi-Turn Conversation (2025, unknown): “Apart from the first shard, the other shards should be decontextualized [13] and not refer to each other in a way that implies an order.”
ExpertQA: Expert-Curated Questions and Attributed Answers (2023, background): “Therefore, we might need approaches that can make claims standalone, such as Choi et al. (2021), to improve post-hoc citation quality.”
Can NLI Models Verify QA Systems' Predictions? (2021, methodology): “We believe that a combination of recent advances in natural language generation (Demszky et al., 2018; Choi et al., 2021) and strong models for NLI (Liu et al., 2019) equip us to re-evaluate this approach.”
Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrieval (2021, background): “We expect future work to be able to infer the sentence-level labels in a latent manner and we are excited about work like decontextualization [4] that tries to retain important context for standalone sentences.”