Probabilistic Reasoning via Deep Learning: Neural Association Models

By QUAN LIU, Hui Jiang, Zhenhua Ling, Si Wei, Yu Hu. 2016

103 citations recorded by the source corpus.

In this paper, we propose a new deep learning approach, called neural association model (NAM), for probabilistic reasoning in artificial intelligence. We propose to use neural networks to model association between any two events in a domain. Neural networks take one event as input and compute a conditional probability of the other event to model how likely these two events are to be associated. The actual meaning of the conditional probabilities varies between applications and depends on how the models are trained. In this work, as two case studies, we have investigated two NAM structures, namely deep neural networks (DNN) and relation-modulated neural nets (RMNN), on several probabilistic reasoning tasks in AI, including recognizing textual entailment, triple classification in multi-relational knowledge bases and commonsense reasoning. Experimental results on several popular datasets derived from WordNet, FreeBase and ConceptNet have all demonstrated that both DNNs and RMNNs perform equally well and they can significantly outperform the conventional methods available for these reasoning tasks. Moreover, compared with DNNs, RMNNs are superior in knowledge transfer, where a pre-trained model can be quickly extended to an unseen relation after observing only a few training samples. To further prove the effectiveness of the proposed models, in this work, we have applied NAMs to solving challenging Winograd Schema (WS) problems. Experiments conducted on a set of WS problems prove that the proposed models have the potential for commonsense reasoning.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

A Simple Method for Commonsense Reasoning (2018, neutral): “3 % USSM + Cause-Effect Knowledge Base [34] 55.”
A Knowledge Hunting Framework for Common Sense Reasoning (2018, methodology): “Sharma et al. (2015)’s knowledge-hunting module focused on a subset of 71 instances that exhibit causal relationships; Liu et al. (2016)’s neural association model focused on a similar causal subset of 70 instances, for which events were extracted manually; and ﬁnally, a recent system by Huang and…”
Unsupervised Deep Structured Semantic Models for Commonsense Reasoning (2019, methodology): “Most of the performance in the top of the Table 2 are the models trained with external knowledge bases, such as Cause-Effect (Liu et al., 2016), Word-Net (Miller, 1995), ConceptNet (Liu and Singh, 2004) knowledge bases.”
How Reasonable are Common-Sense Reasoning Tasks: A Case-Study on the Winograd Schema Challenge and SWAG (2018, unknown): “This allows for simplifying assumptions (Liu et al., 2016) or schematizations (Emami et al., 2018).”
Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences (2019, background): “, dot production), while NAM [24] conducts the semantic matching with a deep architecture.”
A Generalized Knowledge Hunting Framework for the Winograd Schema Challenge (2018, background): “There have been efforts to encode such knowledge directly, using logical formalisms (Bailey et al., 2015) or by using deep learning models (Liu et al., 2016a); however, these approaches have so far solved only restricted subsets of WSC questions with high precision, and show limited ability to…”
Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs (2017, methodology): “Other sophisticated models include Holographic Embeddings (HoLE) (Nickel et al., 2016b) that employs circular correlation on entity embeddings and Neural Association Models (NAM) (Liu et al., 2016), a deep network used for probabilistic reasoning.”
On the Evaluation of Common-Sense Reasoning in Natural Language Understanding (2018, background): “Many WSC instances are composed of two clauses connected by a causal discourse connective like because (as in (1)), which allows for simplifying assumptions [4] or schematiza-tions [1].”
A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2020, methodology): “Representative neural models include multi-layer perceptron (MLP) [5], neural tensor network (NTN) [14], and neural association model (NAM) [46].”
Commonsense Knowledge Enhanced Embeddings for Solving Pronoun Disambiguation Problems in Winograd Schema Challenge (2016, methodology): “Therefore, in this work, we decide to use three commmonsense knowledge bases, including ConceptNet [8], WordNet [15] and CauseCom [16].”
Knowledge Graph Embedding: A Survey of Approaches and Applications (2017, background): “TABLE 2 Summary of Semantic Matching Models (See Section 3.2 for Details) Method Ent. embedding Rel. embedding Scoring function frðh; tÞ Constraints/Regularization RESCAL [13] h; t 2 Rd Mr 2 Rd d h>Mrt khk2 1; ktk2 1; kMrkF 1 Mr ¼ P i p i ruiv > i (required in [17]) TATEC [64] h; t 2 Rd r 2 Rd;Mr 2 ”
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications (2022, unknown): “Furthermore, the neural association model has been developed in [29] to conduct semantic matching with a deep architecture.”
The KnowRef Coreference Corpus: Removing Gender and Number Cues for Difficult Pronominal Anaphora Resolution (2018, background): “Previous approaches to common sense reasoning, based on logical formalisms (Bailey et al., 2015) or deep neural models (Liu et al., 2016), have solved only restricted subsets of the WSC with high precision.”
Combing Context and Commonsense Knowledge Through Neural Networks for Solving Winograd Schema Problems (2016, methodology): “…relationships between the nodes in the vocabulary cover common relations, e.g. cause-effect, entailment, etc. Based on these two requirements, this paper proposes to use the KB constructed by a recent work (Liu et al. 2016), which collects word pairs with cause-effect relationships automatically.”