Open Question Answering with Weakly Supervised Embedding Models

By Antoine Bordes, Jason Aaron Edward Weston, Nicolas Usunier. 2014

354 citations recorded by the source corpus.

Building computers able to answer questions on any subject is a long standing goal of artificial intelligence. Promising progress has recently been achieved by methods that learn to map questions to logical forms or database queries. Such approaches can be effective but at the cost of either large amounts of human-labeled data or by defining lexicons and grammars tailored by practitioners. In this paper, we instead take the radical approach of learning to map questions to vectorial feature representations. By mapping answers into the same space one can query any knowledge base independent of its schema, without requiring any grammar or lexicon. Our method is trained with a new optimization procedure combining stochastic gradient descent followed by a fine-tuning step using the weak supervision provided by blending automatically and collaboratively generated resources. We empirically demonstrate that our model can capture meaningful signals from its noisy supervision leading to major improvements over paralex, the only existing method able to be trained on similar weakly labeled data.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

Memory Networks (2014, methodology): “Following [4,3], training combines pseudo-labeled QA pairs made of a question and an associated triple, and 35M pairs of paraphrased questions from WikiAnswers like “Who wrote the Winnie the Pooh books?” and “Who is poohs creator?””
The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems (2015, methodology): “We build upon the approach in [2], which has also been recently applied to the problem of question answering [31].”
Visual Madlibs: Fill in the blank Image Generation and Question Answering (2015, background): “Recently, embedding and deep learning methods have shown great promise for question-answering [30, 3, 4].”
Question Answering with Subgraph Embeddings (2014, methodology): “Finally, we demonstrate that we greatly improve upon the model of (Bordes et al., 2014b), which actually corresponds to a setting with the Path representation and C1 as candidate set.”
Neural Variational Inference for Text Processing (2015, background): “One of the popular streams is mapping factoid questions with answer triples in the knowledge base (Bordes et al., 2014a;b; Yih et al., 2014).”
Deep Learning for Answer Sentence Selection (2014, methodology): “[3], given the vector representations of a question q and an answer a (both in R), the probability of the answer being correct is p(y = 1|q,a) = σ(q M a+ b), (1)”
Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base (2015, methodology): “Our CNN models can be analogous to the embedding approaches (Bordes et al., 2014a; Yang et al., 2014), but are more sophisticated.”
FVQA: Fact-Based Visual Question Answering (2016, background): “KB-QA approaches can be divided into two categories: semantic parsing methods [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47] that try to translate questions into accurate logical expressions and then map to KB queries, and information retrieval methods [48], [49], [50], [51], [52],”
Question Answering over Freebase with Multi-Column Convolutional Neural Networks (2015, methodology): “The evaluation metrics macro F1 score (Berant et al., 2013) and precision @ 1 (Bordes et al., 2014a) are reported.”
Straight to the Facts: Learning Knowledge Base Retrieval for Factual Visual Question Answering (2018, background): “There has been interest in the natural language processing community in answering questions based on knowledge bases (KBs) using either semantic parsing [37,38,39,40,41,42,43,44,45,46,47] or information retrieval [48,49,50,51,52,53,54] methods.”
Knowledge Graph Completion via Complex Tensor Factorization (2017, methodology): “Motivating applications of knowledge graph completion include question answering (Bordes et al., 2014b) and more generally probabilistic querying of knowledge bases (Huang and Liu, 2009; Krompaß et al., 2014).”
Sequence-based Structured Prediction for Semantic Parsing (2016, positive): “Finally, Bordes et al. (2014b; 2014a) learn a similarity function between a natural language question and the knowledge base formula encoding its answer.”
Visual Madlibs: Fill in the Blank Description Generation and Question Answering (2015, background): “Recently, embedding and deep learning methods have shown great promise [32, 3, 4].”
SSE: Semantically Smooth Embedding for Knowledge Graphs (2017, background): “In this paper, aiming at further discovering the intrinsic geometric structure of the embedding space, we propose Semantically Smooth Embedding (SSE).”
Convolutional Neural Networks vs. Convolution Kernels: Feature Engineering for Answer Sentence Reranking (2016, methodology): “…matrix M R m × m – an intermediate vector representation x s 1 of a sentence s 1 is projected to a ˜x s 1 = x s 1 M , which is then matched with x s 2 (Bordes et al., 2014a), i.e., by computing a dot-product ˜x s 1 x s 2 , thus resulting in a single similarity score x sim ; (iv) a set of…”