Deep Reasoning with Knowledge Graph for Social Relationship Understanding

By Zhouxia Wang, Tianshui Chen, Jimmy S. J. Ren, Weihao Yu, Hui Cheng, Liang Lin. 2018

196 citations recorded by the source corpus.

Social relationships (e.g., friends, couple etc.) form the basis of the social network in our daily life. Automatically interpreting such relationships bears a great potential for the intelligent systems to understand human behavior in depth and to better interact with people at a social level. Human beings interpret the social relationships within a group not only based on the people alone, and the interplay between such social relationships and the contextual information around the people also plays a significant role. However, these additional cues are largely overlooked by the previous studies. We found that the interplay between these two factors can be effectively modeled by a novel structured knowledge graph with proper message propagation and attention. And this structured knowledge can be efficiently integrated into the deep neural network architecture to promote social relationship understanding by an end-to-end trainable Graph Reasoning Model (GRM), in which a propagation mechanism is learned to propagate node message through the graph to explore the interaction between persons of interest and the contextual objects. Meanwhile, a graph attentional mechanism is introduced to explicitly reason about the discriminative objects to promote recognition. Extensive experiments on the public benchmarks demonstrate the superiority of our method over the existing leading competitors.

Subfields: Artificial Intelligence, Social Psychology, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Guosheng Lin
2. Chi Zhang
3. Weixin Liang
4. Zhou Yu
5. James Y. Zou
6. Liang Lin
7. Fayao Liu
8. Tianshui Chen

In Graph Learning

1. Chi Zhang
2. Guosheng Lin
3. Liang Lin
4. Tianshui Chen
5. Riquan Chen
6. Feng Xia
7. Weihao Yu

Citations in the corpus

• ALICE: Active Learning with Contrastive Natural Language Explanations (2020, methodology): “Explanations are reconstructed from Wang et al. (2018)”
• Pyramid Graph Networks With Connection Attentions for Region-Based One-Shot Semantic Segmentation (2019, methodology): “Graph attention mechanisms are also used in other computer vision tasks, such as few-shot image classification [6] and social relationship understanding [24].”
• Knowledge-Embedded Routing Network for Scene Graph Generation (2019, methodology): “We count these co-occurrence probabilities for all category pair and obtain a matrix Inspired by the Graph Gated Neural Networks [17, 3, 29], we adopt a gated recurrent update mechanism to iterative propagate node messages through the graph.”
• Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition (2019, methodology): “Inspired by the current graph propagation works [18, 1, 27, 3], we adopt a gated recurrent update mechanism to propagate message through the graph and learn contextualized node-level features.”
• Knowledge Graphs: Opportunities and Challenges (2023, background): “Wang et al. (2018a) proposed a knowledge graph-based information retrieval technology that extracts entities by mining entity information on web pages via an open-source relation extraction method.”