Bootstrapping Dialog Systems with Word Embeddings

By Gabriel Forgues, Joelle Pineau. 2014

Subfields: Representation & Self-Supervised Learning, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Dialogue & Conversational AI

Citations in the corpus

Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization (2018, background): “The Extreme metric [43] obtains sentence representation by taking the largest extreme values among the embedding vectors of all the words it contains, then calculates the cosine similarity of sentence representations.”
A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues (2016, background): “age), Embedding Extrema (Extrema) and Embedding Greedy (Greedy) (Mitchell and Lapata 2008; Forgues et al. 2014; Rus and Lintean 2012).”
A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version (2015, methodology): “in an unsupervised or reinforcement learning framework) using transfer learning (Fabbrizio et al., 2004; Forgues et al., 2014; Serban and Pineau, 2015; Serban et al., 2016; Lowe et al., 2015a; Vandyke et al., 2015; Wen et al., 2016; Gašić et al., 2016; Mo et al., 2016; Genevay and Laroche, 2016; Che”
Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation (2017, background): “Vector extrema (Forgues et al., 2014) computes a sentence-level embedding by taking the most extreme value of the embeddings of the words composing the sentence for each dimension of the embedding: erd = { maxw∈C ewd if ewd > |minw′∈C ew′d| minw∈C ewd otherwise.”
USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generation (2020, methodology): “Vector Extrema (Forgues et al., 2014) follows a similar setup to Embedding Average, where the score is the cosine similarity between sentence embeddings.”
BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation (2021, background): “2015), embedding average cosine similarity ( AVER ; Sharma et al., 2017), Vector extrema cosine similarity ( VECTOR ; Forgues et al., 2014), Greedy Match ( GREEDY ; Rus and Lintean, 2012).”
DialSummEval: Revisiting Summarization Evaluation for Dialogues (2022, methodology): “Vector extrema (Forgues et al., 2014) is also an embedding based metric similar to Embedding av-5 erage.”
Neural Response Generation via GAN with an Approximate Embedding Layer (2017, background): “In addition, one can also achieve an utterance representation by taking the largest extreme values among the embedding vectors of all the words it contains, before computing the cosine similarities between utterance vectors, which yields the Extreme metric (Forgues et al., 2014).”
DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder (2018, background): “Ex-trema : cosine similarity between the largest extreme values among the word embeddings in the two utterances (Forgues et al. , 2014).”
RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems (2017, methodology): “We call this a referenced metric, because it uses the groundtruth as a reference, akin to existing evaluation metrics.”
Perturbation CheckLists for Evaluating NLG Evaluation Metrics (2021, methodology): “, 2015), Embedding Averaging (EAvg) (Landauer and Dumais, 1997), Greedy Matching (GM) (Rus and Lintean, 2012), and Vector Extrema (VE) (Forgues and Pineau, 2014), we use the implementation provided by Sharma et al.”
Exemplar Encoder-Decoder for Neural Conversation Generation (2018, background): “The extrema representation, for instance, is very sensitive to text length and becomes ineffective beyond single length sentences(Forgues et al., 2014).”
Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generation (2022, unknown): “They include: ROUGE-WE (Morris et al., 2004), SkipThoughts, Embed-dingAverage, VectorExtrema (Forgues et al., 2014), GreedyMatching (Sharma et al., 2017), USE 7 (Cer et al., 2018), WMD (Kusner et al., 7 Cosine similarity between the reference and the hypothesis embeddings.”
Beyond User Self-Reported Likert Scale Ratings: A Comparison Model for Automatic Dialog Evaluation (2020, methodology): “…and the limited references (Gupta et al., 2019) is not addressed in methods that are lexical-level based (Papineni et al., 2002; Lin, 2004; Banerjee and Lavie, 2005), embedding based (Rus and Lintean, 2012; Forgues et al., 2014), or learning based (Tao et al., 2018; Lowe et al., 2017).”
A Survey of Evaluation Metrics Used for NLG Systems (2020, methodology): “SPIDer [72] SPICE, CIDEr WER * tokenizer MultiWER tokenizer TER [122] tokenizer ITER [93] tokenizer CDER [64] tokenizer chrF [100] * * – characTER [138] tokenizer EED [123] tokenizer Vector Extrema [42] * * * * * * tokenizer, pretrained embeddings Vector Averaging [63] * * * * * * tokenizer, pretrai”
Active Evaluation: Efficient NLG Evaluation with Few Pairwise Comparisons (2022, background): “, 2016), Vector Extrema (Forgues et al., 2014), Greedy Matching (Rus and Lintean, 2012), Laser (Artetxe and Schwenk, 2019), BertScore (Zhang et al.”
Hierarchical Neural Network Generative Models for Movie Dialogues (2015, background): “This has been beneficial for classification of user intents (Forgues et al., 2014).”
Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning (2018, methodology): “• Extrema embedding : This method computes the word embedding extrema scores (Forgues et al., 2014) that embed the responses by taking the extrema (maximum of the absolute value) of each dimension, and afterward computes the cosine similarity between them.”
Evaluating Style Transfer for Text (2019, background): “Sentence-level embeddings can be comprised of the most extreme values of word embeddings per dimension (vector extrema) (Forgues et al., 2014), or word embedding averages (Sharma et al.”