AllenAct: A Framework for Embodied AI Research

By Luca Weihs, Jordi Salvador, Klemen Kotar, Unnat Jain, Kuo-Hao Zeng, Roozbeh Mottaghi, Aniruddha Kembhavi. 2020

80 citations recorded by the source corpus.

The domain of Embodied AI, in which agents learn to complete tasks through interaction with their environment from egocentric observations, has experienced substantial growth with the advent of deep reinforcement learning and increased interest from the computer vision, NLP, and robotics communities. This growth has been facilitated by the creation of a large number of simulated environments (such as AI2-THOR, Habitat and CARLA), tasks (like point navigation, instruction following, and embodied question answering), and associated leaderboards. While this diversity has been beneficial and organic, it has also fragmented the community: a huge amount of effort is required to do something as simple as taking a model trained in one environment and testing it in another. This discourages good science. We introduce AllenAct, a modular and flexible learning framework designed with a focus on the unique requirements of Embodied AI research. AllenAct provides first-class support for a growing collection of embodied environments, tasks and algorithms, provides reproductions of state-of-the-art models and includes extensive documentation, tutorials, start-up code, and pre-trained models. We hope that our framework makes Embodied AI more accessible and encourages new researchers to join this exciting area. The framework can be accessed at: this https URL

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Continuous Scene Representations for Embodied AI (2022, methodology): “The network is trained using DD-PPO [44, 54] in the AllenAct framework [53], with a reward that encourages visiting more states and objects.”
A Survey of Embodied AI: From Simulators to Research Tasks (2021, methodology): “Based upon the seven features above and with reference to a study by Allen Institute of Artificial Intelligence [43], we propose secondary evaluation features for embodied AI simulators which consist of three key features: realism, scalability and interactivity shown in Table 1.”
CLIP on Wheels: Zero-Shot Object Navigation as Object Localization and Exploration (2022, methodology): “We use the AllenAct [66] framework to conduct exploration agent training.”
Large Batch Simulation for Deep Reinforcement Learning (2021, background): “…environment simulation by running many copies of off-the-shelf, unmodified simulators, such as simulators designed for video game engines (Bellemare et al., 2013; Kempka et al., 2016; Beattie et al., 2016; Weihs et al., 2020), on large numbers ∗Correspondence to bps@cs.stanford.edu ar X iv :2 10 3.”
Visual Room Rearrangement (2021, methodology): “5, 6 [46] Luca Weihs, Jordi Salvador, Klemen Kotar, Unnat Jain, KuoHao Zeng, Roozbeh Mottaghi, and Aniruddha Kembhavi.”
ManipulaTHOR: A Framework for Visual Object Manipulation (2021, methodology): “We use the AllenAct [32] framework to implement and train our models.”
ProcTHOR: Large-Scale Embodied AI Using Procedural Generation (2022, methodology): “All models are trained with the AllenAct [56] framework, see the Appendix for training details.”
Objaverse: A Universe of Annotated 3D Objects (2022, methodology): “We use the AllenAct [75] framework to train our agent.”
How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget (2020, unknown): “Relatively small CNNs are common place in visual navigation [27, 28, 25, 29, 6].”
Pushing it out of the Way: Interactive Visual Navigation (2021, methodology): “In this work, we use the AllenAct [37] framework to conduct experiments.”