Learning relational object categories using behavioral exploration and multimodal perception

By Jivko Sinapov, Connor Schenck, Alexander Stoytchev. 2014

63 citations recorded by the source corpus.

Subfields: Cultural Studies, Developmental and Educational Psychology, Experimental and Cognitive Psychology, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cultural Studies

In Developmental and Educational Psychology

In Experimental and Cognitive Psychology

In Representation & Self-Supervised Learning

Citations in the corpus

Experience Grounds Language (2020, background): “Further, tactile senses lend meaning, both physical (Sinapov et al., 2014; Thomason et al., 2016) and abstract, to concepts like heavy and soft.”
Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completion (2021, background): “…full range of experience that can ground language in the world (Bisk et al., 2020), such as haptic feedback during object manipulation (Thomason et al., 2020, 2016; Sinapov et al., 2014), and audio (Chen et al., 2020a) and speech (Harwath et al., 2019; Ku et al., 2020) features of the environment.”
Interactive Perception: Leveraging Action in Perception and Perception in Action (2016, methodology): “[59], [60], and Sinapov and Stoytchev [62] let a robot interact with a set of objects that are characterized by different attributes, such as rigid or deformable, heavy or light, and slippery or not.”
Learning Multi-Modal Grounded Linguistic Semantics by Playing "I Spy" (2016, methodology): “Following previous work in multi-modal exploration [Sinapov et al., 2014b], for each context we calculated Cohen's Kappa κ c ∈ [0, 1] to measure the agreement across observations between the decisions of the M c classifier and the ground truth labels from the " I Spy " game.”
Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks (2018, background): “[3, 10, 19, 40] are examples of fusing visual and haptic data for grasp stability assessment, manipulation, material recognition, or object categorization.”
Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog (2020, methodology): “We connect these feature representations of objects to language labels by learning discriminative classifiers on the feature spaces for each perceptual language concept, as detailed in previous work [33, 38].”
Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks (2019, background): “The works in [24], [25], [44], and [45] fuse visual and haptic data for grasp stability assessment, manipulation, material recognition, or object categorization.”
Guiding Exploratory Behaviors for Multi-Modal Grounding of Linguistic Descriptions (2018, methodology): “Evaluation: colors, weights, contents[2]”