Learning to Connect Language and Perception

By Raymond J. Mooney. 2008

93 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Multimodal Learning

Citations in the corpus

Experience Grounds Language (2020, background): “While the NLP community has played an important role in the history of grounding (Mooney, 2008), recently remarkable progress has taken place in the Computer Vision community.”
Learning to Interpret Natural Language Navigation Instructions from Observations (2011, background): “Our work also ﬁts into the broader area of grounded language acquisition , in which language is learned by simply observing its use in some naturally occurring perceptual context (see Mooney (2008) for a review).”
Learning to Win by Reading Manuals in a Monte-Carlo Framework (2011, background): “Our work fits into the broad area of grounded language acquisition where the goal is to learn linguistic analysis from a situated context (Oates, 2001; Siskind, 2001; Yu and Ballard, 2004; Fleischman and Roy, 2005; Mooney, 2008a; Mooney, 2008b; Branavan et al., 2009; Vogel and Jurafsky, 2010).”
Learning Rewards from Linguistic Feedback (2020, methodology): “To extract target features, we ﬁrst ground utterances to elements of the MDP (Harnad 1990; Mooney 2008).”
Neurosymbolic Programming (2021, background): “This symbol grounding problem (Harnad, 1990; Mooney, 2008) is well-established in many areas of AI.”
Grounded Language Learning: Where Robotics and NLP Meet (2018, methodology): “…meanings is the grounded language acquisition problem: taking linguistic tokens and learning to This material is based in part upon work supported by the National Science Foundation under Grant No. 1657469. interpret them by connecting them to real-world percepts and actions [Mooney, 2008].”
Reading between the Lines: Learning to Map High-Level Instructions to Commands (2010, background): “This assumption of a direct correspondence between the text and the environment is not unique to that paper, being inherent in other work on grounded language learning (Siskind, 2001; Oates, 2001; Yu and Ballard, 2004; Fleischman and Roy, 2005; Mooney, 2008; Liang et al., 2009; Matuszek et al., 2010”
Learning High-Level Planning from Text (2012, methodology): “Learning Semantics via Language Grounding Our work fits into the broad area of grounded language acquisition, where the goal is to learn linguistic analysis from a situated context (Oates, 2001; Siskind, 2001; Yu and Ballard, 2004; Fleischman and Roy, 2005; Mooney, 2008a; Mooney, 2008b; Branavan et ”
Learning to Mediate Perceptual Differences in Situated Human-Robot Dialogue (2015, background): “Recent years have seen an increasing amount of work that grounds language to shared environment (Mooney 2008; Qu and Chai 2010; Chen and Mooney 2011; Tellex et al. 2011; Matuszek et al. 2012).”
Distributional Semantics in Technicolor (2012, background): “Recent developments in computer vision make it possible to computationally model one vital human perceptual channel: vision (Mooney, 2008).”
Correcting Robot Plans with Natural Language Feedback (2022, background): “However, language also brings with it ambiguity and requires a model of symbol grounding and spatio-temporal reasoning to relate the concepts expressed in language to the robot’s state and action spaces [17, 25, 37, 22, 3, 30, 29].”
Robots That Use Language (2020, background): “Grounded language (also called situated language or physically situated language) has meaning in the context of the physical world—for example, by describing the environment, physical actions, or relationships between things (7, 19).”
Learning from Unscripted Deictic Gesture and Language for Human-Robot Interactions (2014, background): “The language learning component of this work fits into the general category of grounded language acquisition (Mooney 2008): The interpretation of human language into semantically informed structures in the context of perception and actuation.”
Learning to Parse Natural Language Commands to a Robot Control System (2012, background): “Our work falls also into the broader class of grounded language acquisition [24], in which language is learned from situated context, usually by learning over a corpus of parallel language and context data.”
Environment-Driven Lexicon Induction for High-Level Instructions (2015, unknown): “The broader problem of grounded language acquisition, involving connecting words to aspects of a situated context has been heavily studied (Duvallet et al., 2014; Yu and Siskind, 2013; Chu et al., 2013; Chen and Mooney, 2008; Mooney, 2008; Fleischman and Roy, 2005; Liang et al., 2009).”
Grounding Language in Play (2020, background): “This motivates the long-standing question in artiﬁcial intelligence of embodied language acquisition (Harnad, 1990; Mooney, 2008): how might intelligent agents ground language understanding in their own embodied perception?”
Situated Language Understanding with Human-like and Visualization-Based Transparency (2016, methodology): “methods with alternative language understanding techniques to resolve references to the scene [10, 17, 23, 25].”