Canonical Tensor Decomposition for Knowledge Base Completion

By Timothée Lacroix, Nicolas Usunier, Guillaume Obozinski. International Conference on Machine Learning, 2018

450 citations recorded by the source corpus.

The problem of Knowledge Base Completion can be framed as a 3rd-order binary tensor completion problem. In this light, the Canonical Tensor Decomposition (CP) (Hitchcock, 1927) seems like a natural solution; however, current implementations of CP on standard Knowledge Base Completion benchmarks are lagging behind their competitors. In this work, we attempt to understand the limits of CP for knowledge base completion. First, we motivate and test a novel regularizer, based on tensor nuclear $p$-norms. Then, we present a reformulation of the problem that makes it invariant to arbitrary choices in the inclusion of predicates or their reciprocals in the dataset. These two methods combined allow us to beat the current state of the art on several datasets with a CP decomposition, and obtain even better results using the more advanced ComplEx model.

Subfields: Artificial Intelligence, Computational Mathematics

View Paper: https://www.semanticscholar.org/paper/bbb2d7075141f6d638dd41c3ffe0d6bfb3d2dba9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mathematics

Citations in the corpus

PyTorch-BigGraph: A Large-scale Graph Embedding System (2019, methodology): “In addition, recent papers have reported even stronger results for FB15k (and other small knowledge graphs like WordNet) using ComplEx with very large embeddings of thousands of dimensions (Lacroix et al., 2018).”
Using Local Knowledge Graph Construction to Scale Seq2Seq Models to Multi-Document Inputs (2019, background): “, 2018b), the mechanism is termed DMCA as it is applied on the decoder side knowledge base completion (Lacroix et al., 2018; Bordes et al., 2011).”
Multi-relational Poincaré Graph Embeddings (2019, methodology): “We report results for ComplEx-N3 [16] at d = 100 to ensure comparability with MuRE and MuRP in terms of the overall number of parameters (original paper reports results at d = 2000).”
TuckER: Tensor Factorization for Knowledge Graph Completion (2019, background): “To our knowledge, ComplExN3 (Lacroix et al., 2018) is the only other linear link prediction model that benefits from multitask learning.”
HittER: Hierarchical Transformers for Knowledge Graph Embeddings (2020, background): “Early work uses translational distance-based scoring functions deﬁned on top of entity and relation simplify the modeling architecture, we also make the number of tokens known to all models. embeddings (Bordes et al., 2013; Wang et al., 2014; Lin et al., 2015; Ji et al., 2015).”
DRUM: End-To-End Differentiable Rule Mining On Knowledge Graphs (2019, background): “The results are taken from [21, 41] and [35]”
Tensor Decompositions for temporal knowledge base completion (2020, methodology): “∆ 3 matches the order used in the Ω 3 regularizer, and in previous work on knowledge base completion (Lacroix et al., 2018).”
Knowledge Graph Embedding for Link Prediction (2020, methodology): “is has resulted in the following choices: • For TransE [8], DistMult [71] and HolE [46] we have used the implementation provided by project Ampligraph [10] and available in their repository [1]; • For ComplEx [61] we have used Timothée Lacroix’s version with N3 regularization [30], available in Fac”
Relational Representation Learning for Dynamic (Knowledge) Graphs: A Survey (2019, unknown): “] https://github.com/soumith/net2net.torch Grover and Leskovec [83] https://github.com/aditya-grover/node2vec Wu et al. [234] https://github.com/RuidongZ/Recurrent_Recommender_Networks Lacroix et al. [130] https://github.com/facebookresearch/kbc Nickel et al. [173] https://github.com/mnick/holograph”
Question Answering Over Temporal Knowledge Graphs (2021, background): “Despite further developments, ComplEx, along with refined training protocols (Lacroix et al., 2018) remains among the strongest KB embedding approaches (Ruffinelli et al.”
Quaternion Knowledge Graph Embeddings (2019, methodology): “However, same as the method in [Lacroix et al., 2018], QuatE 2 requires a large embedding dimension.”
Low-Dimensional Hyperbolic Knowledge Graph Embeddings (2020, methodology): “Evaluation metrics At test time, we use the scoring function in Equation 10 to rank the correct tail or head entity against all possible entities, and use in use inverse relations for head prediction (Lacroix et al., 2018).”
Questions for Flat-Minima Optimization of Modern Neural Networks (2022, methodology): “…values from Hu et al. (2020) for GCN (Kipf & Welling, 2017), SAGE (Hamilton et al., 2017), and GIN (Xu et al., 2019), from Chen et al. (2021b) for (Lacroix et al., 2018) and ComplEx (Trouillon et al., 2016), and from Li et al. (2020) for DGCN. Due to high standard errors, we averaged the results…”
A Simple Convergence Proof of Adam and Adagrad (2020, unknown): “First-order methods with adaptive step sizes have proved useful in many ﬁelds of machine learning, be it for sparse optimization (Duchi et al., 2013), tensor factorization (Lacroix et al., 2018) or deep learning (Goodfel-low et al., 2016).”