Basic Reasoning with Tensor Product Representations

By Paul Smolensky, Moontae Lee, Xiaodong He, Wen-tau Yih, Jianfeng Gao, Li Deng. 2016

32 citations recorded by the source corpus.

In this paper we present the initial development of a general theory for mapping inference in predicate logic to computation over Tensor Product Representations (TPRs; Smolensky (1990), Smolensky & Legendre (2006)). After an initial brief synopsis of TPRs (Section 0), we begin with particular examples of inference with TPRs in the 'bAbI' question-answering task of Weston et al. (2015) (Section 1). We then present a simplification of the general analysis that suffices for the bAbI task (Section 2). Finally, we lay out the general treatment of inference over TPRs (Section 3). We also show the simplification in Section 2 derives the inference methods described in Lee et al. (2016); this shows how the simple methods of Lee et al. (2016) can be formally extended to more general reasoning tasks.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence (2020, background): “A team of people including Smolensky and Schmidhuber have produced better results on a mathematics problem set by combining BERT with a tensor products (Smolensky et al., 2016), a formal system for representing symbolic variables and their bindings (Schlag et al.”
Reasoning in Vector Space: An Exploratory Study of Question Answering (2015, background): “r logical reasoning to computation over TPRs, generalizing our models under the rigorous TPR formalism. Due to the page limit, this theoretical foundation is relegated to the supplementary materials (Smolensky et al. (2016)). The experimental results show that accurate inference based on common-sens”
Mapping natural-language problems to formal-language solutions using structured neural representations (2019, methodology): “Some researchers use TPRs for modeling deductive reasoning processes both on a rule-based model and deep learning models in vector space (Lee et al., 2016; Smolensky et al., 2016; Schlag & Schmidhuber, 2018).”
Natural- to formal-language generation using Tensor Product Representations (2019, methodology): “Some researchers use TPRs for modeling deductive reasoning processes both on a rule-based model and deep learning models in vector space (Lee et al., 2016; Smolensky et al., 2016; Schlag & Schmidhuber, 2018).”
Learning Inference Rules with Neural TP-Reasoner (2020, methodology): “Some researchers also use TPRs for modeling deductive reasoning processes both on a rule-based model and deep learning models in vector space (Lee et al., 2016; Smolensky et al., 2016; Schlag and Schmidhuber, 2018).”
Neurosymbolic Programming (2021, background): “Neither do we consider Tensor Product Representations (Smolensky et al., 2016), which provide a symbolically inspired inductive bias in neural networks, to be an example of neurosymbolic programming.”
Learning to Reason with Third-Order Tensor Products (2018, methodology): “For further details, we refer to Smolensky’s work [18, 28, 29].”