Representation Learning for Visual-Relational Knowledge Graphs
By Daniel Oñoro-Rubio, Mathias Niepert, Alberto García-Durán, Roberto Gonzalez, Roberto Javier López-Sastre. 2017
15 citations recorded by the source corpus.
A visual-relational knowledge graph (KG) is a KG whose entities are associated with images. We propose representation learning for relation and entity prediction in visual-relational KGs as a novel machine learning problem. We introduce \textsc{ImageGraph}, a KG with 1,330 relation types, 14,870 entities, and 829,931 images. Visual-relational KGs lead to novel probabilistic query types treating images as first-class citizens. We approach the query answering problems by combining ideas from the areas of computer vision and embedding learning for KGs. The resulting ML models can answer queries such as \textit{"How are these two unseen images related to each other?"} We also explore a novel zero-shot learning scenario where an image of an entirely new entity is linked with multiple relations to entities of an existing KG. Our experiments show that the proposed deep neural networks are able to answer the visual-relational queries efficiently and accurately.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Yoshua Bengio
- Sergey Levine
- Ruslan Salakhutdinov
- Trevor Darrell
- Zhe Gan
- Mohit Bansal
- Kyunghyun Cho
- Abhinav Kumar Gupta
Citations in the corpus
- Relational inductive biases, deep learning, and graph networks (2018, background): “, 2018), to reason about knowledge graphs (Bordes et al., 2013; Oñoro-Rubio et al., 2017; Hamaguchi et al., 2017), to predict the chemical properties of molecules (Duvenaud et al.”
- MMKG: Multi-Modal Knowledge Graphs (2019, unknown): “In addition, recent work on visual-relational knowledge graphs [23] has introduced novel visual query types such as ”How are these two unseen images related to each other?” and has proposed novel machine learning methods to answer these queries.”
- Revisiting Evaluation of Knowledge Base Completion Models (2020, background): “Furthermore, a number of studies have examined the incorporation of extra types of evidence to achieve more informative embeddings, with extra modalities consisting of numerical values [Garcia-Duran and Niepert, 2017], images [Oñoro-Rubio et al., 2017], text [Toutanova et al.”