What Can You Do with a Rock? Affordance Extraction via Word Embeddings

By Nancy Fulda, Daniel Ricks, Ben Murdoch, David Wingate. 2017

83 citations recorded by the source corpus.

Autonomous agents must often detect affordances: the set of behaviors enabled by a situation. Affordance detection is particularly helpful in domains with large action spaces, allowing the agent to prune its search space by avoiding futile behaviors. This paper presents a method for affordance extraction via word embeddings trained on a Wikipedia corpus. The resulting word vectors are treated as a common knowledge database which can be queried using linear algebra. We apply this method to a reinforcement learning agent in a text-only environment and show that affordance-based action selection improves performance most of the time. Our method increases the computational complexity of each learning step but significantly reduces the total number of steps needed. In addition, the agent's action selections begin to resemble those a human would choose.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Keep CALM and Explore: Language Models for Action Generation in Text-based Games (2020, methodology): “Fulda et al. (2017) used Word2vec (Mikolov et al., 2013) embeddings to infer affordance properties (i.e. verbs suitable for an object).”
Interactive Fiction Games: A Colossal Adventure (2019, background): “Also known as affordance extraction (Gibson 1977; Fulda et al. 2017), the problem of choosing which actions or verbs to pair with a particular noun is central to IF game playing.”
Graph Constrained Reinforcement Learning for Natural Language Action Spaces (2020, methodology): “Fulda et al. (2017) use Word2Vec (Mikolov et al., 2013) to aid in extracting affordances for items in these games and use this information to produce relevant action verbs.”
Toward Automated Quest Generation in Text-Adventure Games (2019, background): “Although there has been much work recently on text-adventure gameplay (Bordes et al., 2010; He et al., 2016; Narasimhan et al., 2015; Fulda et al., 2017; Yang et al., 2018; Haroush et al., 2018; Cˆot´e et al., 2018; Tao et al., 2018; Ammanabrolu and Riedl, 2019a; Hausknecht et al., 2019a;…”
Text-based adventures of the golovin AI agent (2017, methodology): “Also, we tested our agent against the winner of the last year competition [15].”
Situated Dialogue Learning through Procedural Environment Generation (2021, methodology): “Fulda et al. (2017) use word embeddings to model affordances for items in these games.”
Counting to Explore and Generalize in Text-based Games (2018, methodology): “Fulda et al. (2017) propose a method to reduce the action space for parser-based games by training word embeddings to be aware of verb-noun affordances.”
BYU-EVE: Mixed Initiative Dialog via Structured Knowledge Graph Traversal and Conversational Scaffolding (2018, background): “have shown and other researchers have confirmed that within these embedding spaces, simple mathematical operations are sufficient to solve analogical queries [20, 27, 11, 12].”
Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agents (2021, methodology): “Previous work has deployed a spectrum of methods for language processing in this domain, including word vectors (Fulda et al., 2017), recurrent neural networks (Narasimhan et al., 2015; Hausknecht et al., 2020), pre-trained language models (Yao ∗Work partly done during internship at Microsoft…”
NAIL: A General Interactive Fiction Agent (2019, background): “For those containing location information, commands are generated by extracting nouns from the text and using the BYU Agent’s word embedding approach to ﬁnd likely verbs.”
Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning (2018, methodology): “The same amount of steps that was used in previous work on Zork (Fulda et al., 2017; Kostka et al., 2017).”
Harvesting Common-sense Navigational Knowledge for Robotics from Uncurated Text Corpora (2017, methodology): “Recent work in our laboratory applies these concepts at a more abstract level by using word embeddings to automatically detect affordances [8], meaning the set of actions that can be performed on a given object.”
Guiding Exploratory Behaviors for Multi-Modal Grounding of Linguistic Descriptions (2018, background): “2016a), and unseen noun affordances from seen nouns (Fulda et al. 2017).”
How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds (2020, background): “(Fulda et al., 2017; Ammanabrolu and Riedl, 2019; Ammanabrolu et al., 2020; Murugesan et al., 2020) look at commonsense reasoning in interactive environments, with the former focusing on affordance extraction using word embeddings and the latter three on transferring text-game playing skills via pre”
Multi-Stage Episodic Control for Strategic Exploration in Text Games (2022, unknown): “Some approaches aim to navigate the large action space by ﬁltering inadmissible actions (Zahavy et al., 2018; Jain et al., 2020), leveraging pre-trained language models for action selection (Yao et al., 2020; Jang et al., 2020) or word embeddings for affordance detection (Fulda et al., 2017).”
Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning (2018, background): “A growing body of research has explored the challenges associated with text-based games (Bordes et al., 2010; Narasimhan et al., 2015; He et al., 2016; Fulda et al., 2017; Haroush et al., 2018; Cˆot´e et al., 2018; Tao et al., 2018).”
Learning Dynamic Belief Graphs to Generalize on Text-Based Games (2020, methodology): “[13] use pre-trained embeddings to reduce the action space.”
What can I do here? A Theory of Affordances in Reinforcement Learning (2020, methodology): “Different modalities have been used to detect affordances, such as visuo-motor simulation (Schenck et al., 2012), visual characteristics (Song et al., 2015), and text embeddings (Fulda et al., 2017).”