Grounding language in perception

By Jeffrey Mark Siskind. 1993

Subfields: Natural Language Processing, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Multimodal Learning

Citations in the corpus

Grounded Language Learning in a Simulated 3D World (2017, background): “orm, where the grounding is usually discrete and high level. This is in contrast to the work presented in this paper where we focus on learning to ground language in low level perception and actions. Siskind (1995) represents an early attempt to ground language in perception by seeking to link objec”
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances (2022, background): “A significant body of work has focused on grounding language [28, 29].”
Aibo''s first words. the social learning of language and meaning. Evolution of Communication (2002, methodology): “By using real world autonomous robots, our experiments differ from other computational experiments in word learning (such as [Siskind, 1995]) in which situation-word pairs are prepared in advance by the human experimenter, and even more from more traditional connectionist word learning experiments…”
Connectionist sentence processing in perspective (1999, background): “Work along these lines has already been sketched in more symbolic computational terms by Drescher (1991) and Siskind (1995).”
Exploring discriminative pose sub-patterns for effective action classification (2013, background): “In a much earlier work [30], Siskind animated a line-drawing of a person as pose input and classified actions such as dropping, throwing, picking up, and putting down by analyzing the event logic.”
Towards Understanding Situated Natural Language (2010, unknown): “Other works use visual grounding (Siskind, 1994; Yu & Ballard, 2004; Barnard & Johnson, 2005), or a representation of the intended meaning in some formal language (Fleischman & Roy, 2005; Kate & Mooney, 2007; Wong & Mooney, 2007; Chen & Mooney, 2008; Branavan et al.”
Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models (2019, background): “A thorough understanding of what a bird is involves understanding not just the characteristics of its visual and linguistic features individually, but also how they relate to each other (Barsalou, 2008; Siskind, 1994).”
Interactive Grounded Language Acquisition and Generalization in a 2D World (2018, background): “entence is simply a sequence of tokens with each token being equally meaningless in the beginning of learning. This is unlike some early pioneering systems (e.g., SHRDLU (Winograd, 1972) and ABIGAIL (Siskind, 1994)) that hard-coded the syntax or semantics to link language to a simulated world–an app”
The Recognition of Human Movement Using Temporal Templates (2001, methodology): “As for action recognition, Campbell and Bobick [8] used a commercially available system to obtain three-dimensional data of human body limb positions.”