Adversarial Sets for Regularising Neural Link Predictors

By Pasquale Minervini, Thomas Demeester, Tim Rocktaschel, Sebastian Riedel. 2017

102 citations recorded by the source corpus.

In adversarial training, a set of models learn together by pursuing competing goals, usually defined on single data instances. However, in relational learning and other non-i.i.d domains, goals can also be defined over sets of instances. For example, a link predictor for the is-a relation needs to be consistent with the transitivity property: if is-a(x_1, x_2) and is-a(x_2, x_3) hold, is-a(x_1, x_3) needs to hold as well. Here we use such assumptions for deriving an inconsistency loss, measuring the degree to which the model violates the assumptions on an adversarially-generated set of examples. The training objective is defined as a minimax problem, where an adversary finds the most offending adversarial examples by maximising the inconsistency loss, and the model is trained by jointly minimising a supervised loss and the inconsistency loss on the adversarial examples. This yields the first method that can use function-free Horn clauses (as in Datalog) to regularise any neural link predictor, with complexity independent of the domain size. We show that for several link prediction models, the optimisation problem faced by the adversary has efficient closed-form solutions. Experiments on link prediction benchmarks indicate that given suitable prior knowledge, our method can significantly improve neural link predictors on all relevant metrics.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

DeepProbLog: Neural Probabilistic Logic Programming (2018, background): “[35], who investigate injecting rules by minimizing an inconsistency loss on adversarially-generated examples.”
Differentiable Reasoning on Large Knowledge Bases and Natural Language (2019, methodology): “2016), DistMult and ComplEx using Adversarial Sets (ASR) (Minervini et al. 2017)—a method for incorporating rules in neural link predictors via adversarial training—and the recently proposed KBLR (Garcı́a-Durán and Niepert 2018).”
MolGAN: An implicit generative model for small molecular graphs (2018, methodology): “For link prediction within graphs, a range of adversarial methods have been introduced in the literature (Minervini et al., 2017; Wang et al., 2018; Bojchevski et al., 2018).”
Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge (2018, methodology): “Similarly to Minervini et al. (2017), we use the Gödel t-norm, a continuous generalisation of the conjunction operator in logic (Gupta and Qi, 1991), for computing the probability of the body of a clause: where a 1 and a 2 are two clause atoms.”
Neuro-Symbolic = Neural + Logical + Probabilistic (2019, methodology): “, 2016] X Implication rules Adverserial Set Regularisation [Minervini et al., 2017] X Clausal logic Semantic Loss Function [Xu et al.”
Learning Relational Representations with Auto-encoding Logic Programs (2019, background): “Several works [Minervini et al. , 2017; Demeester et al. , 2016] impose logical constraints on embed-dings but do not retain the relational representation.”