RelNN: A Deep Neural Model for Relational Learning

By Seyed Mehran Kazemi, David Poole. 2017

55 citations recorded by the source corpus.

Statistical relational AI (StarAI) aims at reasoning and learning in noisy domains described in terms of objects and relationships by combining probability with first-order logic. With huge advances in deep learning in the current years, combining deep networks with first-order logic has been the focus of several recent studies. Many of the existing attempts, however, only focus on relations and ignore object properties. The attempts that do consider object properties are limited in terms of modelling power or scalability. In this paper, we develop relational neural networks (RelNNs) by adding hidden layers to relational logistic regression (the relational counterpart of logistic regression). We learn latent properties for objects both directly and through general rules. Back-propagation is used for training these models. A modular, layer-wise architecture facilitates utilizing the techniques developed within deep learning community to our architecture. Initial experiments on eight tasks over three real-world datasets show that RelNNs are promising models for relational learning.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

DeepProbLog: Neural Probabilistic Logic Programming (2018, methodology): “[51] treat symbolic rules expressed in first-order logic as a template for constructing a neural network, while Kazemi and Poole [52] compose a relational neural network by adding hidden layers to relational logistic regression [53].”
Diachronic Embedding for Temporal Knowledge Graph Completion (2019, background): “These approaches have been combined with embeddings (Kazemi and Poole 2018a) and also extended to temporal KGs (see, e.”
Representation Learning for Dynamic Graphs: A Survey (2019, methodology): “Realizing the information may not flow well between the two vectors of a node, SimplE adds another vector to the relation embeddings as EMB(r) = (zr, zr) where zr models the behaviour of the inverse of the relation.”
SimplE Embedding for Link Prediction in Knowledge Graphs (2018, methodology): “ generate many negative triples for a positive triple (Trouillon et al. [39] show that generating more negative triples improves accuracy), 5- combining SimplE with logic-based approaches (e.g., with [19]) to improve property prediction, 6- combining SimplE with (or use SimplE as a sub-component in)”
Relational Representation Learning for Dynamic (Knowledge) Graphs: A Survey (2019, background): “Sourek et al. [207] and Kazemi and Poole [114] propose other variants for Equation (28) where (roughly) the transformations are done using soft ﬁrst-order logic rules.”
Prototypical Representation Learning for Relation Extraction (2021, methodology): “…methods reason relations via symbolic logic rules, with adopting probabilistic graphical models or inductive logic systems to learn and infer logic rules orienting to relations Cohen & Hirsh 1994; Wang et al., 2015; Yang et al., 2017; Chen et al., 2018; Kazemi & Poole, 2018; Qu & Tang, 2019).”
Learning Relational Representations with Auto-encoding Logic Programs (2019, background): “Kazemi and Poole [2017] and Sourek et al. [2016] introduce symbolic variants of neural networks for relational data.”
Structure learning for relational logistic regression: an ensemble approach (2018, background): “They have generally ranged from directed models [15, 21, 10, 12, 22, 14] to undirected models [38, 40, 18].”