COD3S: Diverse Generation with Discrete Semantic Signatures

By Nathaniel Weir, João Sedoc, Benjamin Van Durme. 2020

15 citations recorded by the source corpus.

We present COD3S, a novel method for generating semantically diverse sentences using neural sequence-to-sequence (seq2seq) models. Conditioned on an input, seq2seq models typically produce semantically and syntactically homogeneous sets of sentences and thus perform poorly on one-to-many sequence generation tasks. Our two-stage approach improves output diversity by conditioning generation on locality-sensitive hash (LSH)-based semantic sentence codes whose Hamming distances highly correlate with human judgments of semantic textual similarity. Though it is generally applicable, we apply COD3S to causal generation, the task of predicting a proposition's plausible causes or effects. We demonstrate through automatic and human evaluation that responses produced using our method exhibit improved diversity without degrading task performance.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mohit Bansal
2. Yejin Choi
3. Matt Gardner
4. Luke S. Zettlemoyer
5. Noah A. Smith
6. Benjamin Van Durme
7. Jianfeng Gao
8. Samuel R. Bowman

In Computer Vision and Pattern Recognition

1. Matthew E. Peters
2. Matt Gardner
3. Gargi Ghosh
4. Mona T. Diab
5. Asli Celikyilmaz
6. Tianlu Wang
7. Hao Peng
8. Badr AlKhamissi

In Generative Models

1. Asli Celikyilmaz
2. Hao Peng
3. Alexis Ross
4. Nathaniel Weir
5. Matthew E. Peters
6. Benjamin Van Durme

Citations in the corpus

• Tailor: Generating and Perturbing Text with Semantic Controls (2021, background): “…syntactic structures for paraphrasing (Iyyer et al., 2018; Chen et al., 2019; Bao et al., 2019; Kumar et al., 2020; Sun et al., 2021; Huang and Chang, 2021) or discrete semantic signatures for diverse generation (Weir et al., 2020) are similar to T ailor in their high-dimensional specification.”
• ALERT: Adapt Language Models to Reasoning Tasks (2022, background): “For example, task963_librispeech_asr_next_word_ prediction (Weir et al., 2020) provides a prompt “Joey’s favourite food is ___”, with the ground truth answer “sandwiches”.”
• InFillmore: Frame-Guided Language Generation with Bidirectional Context (2021, background): “This is compared to other methods for diverse sampling, such as random and nucleus sampling (Holtzman et al., 2020), in which there is no notion of higher level semantic reasoning and a tendency to hallucinate content, or COD3S (Weir et al., 2020), which enables only moderate interpretability not ba”