A Survey of Embodied AI: From Simulators to Research Tasks

By Jiafei Duan, Samson Yu, Tangyao Li, Huaiyu Zhu, Cheston Tan. 2021

536 citations recorded by the source corpus.

There has been an emerging paradigm shift from the era of “internet AI” to “embodied AI,” where AI algorithms and agents no longer learn from datasets of images, videos or text curated primarily from the internet. Instead, they learn through interactions with their environments from an egocentric perception similar to humans. Consequently, there has been substantial growth in the demand for embodied AI simulators to support various embodied AI research tasks. This growing interest in embodied AI is beneficial to the greater pursuit of Artificial General Intelligence (AGI), but there has not been a contemporary and comprehensive survey of this field. This paper aims to provide an encyclopedic survey for the field of embodied AI, from its simulators to its research. By evaluating nine current embodied AI simulators with our proposed seven features, this paper aims to understand the simulators in their provision for use in embodied AI research and their limitations. Lastly, this paper surveys the three main research tasks in embodied AI – visual exploration, visual navigation and embodied question answering (QA), covering the state-of-the-art approaches, evaluation metrics and datasets. Finally, with the new insights revealed through surveying the field, the paper will provide suggestions for simulator-for-task selections and recommendations for the future directions of the field.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Robotics & Control

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Embodied AI [74], [105], [123], an emerging paradigm shift from the era of “internet AI”, aims to enable AI algorithms and agents to learn through interactions with their environments, no longer from datasets of images, videos or texts curated primarily from the internet.”
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge (2022, background): “[29] Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan.”
Voyager: An Open-Ended Embodied Agent with Large Language Models (2023, background): “[75] Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan.”
Tool Learning with Foundation Models (2023, unknown): “The fundamental framework for tool learning entails a sequence of action and observation, where the model can perceive changes in the environment, aligning with the fundamental concept of embodied learning (Duan et al., 2022).”
VIMA: General Robot Manipulation with Multimodal Prompts (2022, methodology): “For example, VIOLA (Zhu et al., 2022) achieves better performance with a pre-trained Region Proposal Network (Ren et al., 2015).”
The rise and potential of large language model based agents: a survey (2025, unknown): “Some studies [275, 276] highlighted the importance of spatial representation to obtain the precise localization of targets by leveraging the pre-trained VLM model to combine visual features from images with 3D reconstructions of the physical world [250].”
SPACE: A Simulator for Physical Interactions and Causal Learning in 3D Environments (2021, background): “But with the advancement and rise of deep learning, computer graphics, and embodied AI [8, 15, 18], there has been a paradigm shift towards generating synthetic datasets that range from simple 2D cartoons [10, 22, 35] to realistic interaction in 3D environments [3, 7, 9, 13, 29, 34], all with the ai”
Aligning Cyber Space With Physical World: A Comprehensive Survey on Embodied AI (2024, unknown): “Different from conversational agents like ChatGPT [4], embodied AI believes that the true AGI can be achieved by controlling physical embodiments and interacting with both simulated and physical environments [5], [6].”
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models (2025, unknown): “Efficient reasoning LLMs make embodied AI [27] much smarter by helping robots and smart devices understand and react to the world around them.”
PIP: Physical Interaction Prediction via Mental Imagery with Span Selection (2021, methodology): “With the rapid advancements of computer vision, deep learning and embodied AI [3, 4, 5], there are recent works attempted to model the prediction of physical interactions among objects using learning-based approaches.”
End-to-End Autonomous Driving: Challenges and Frontiers (2023, background): “Inareasrelatedtoautonomousdrivingsuchasembodied AI,incorporatingnaturallanguageasﬁne-grainedknowledge andinstructionstocontrolthevisuomotoragenthasachieved notable progress [151], [152], [153], [154].”
A Survey on Vision--Language--Action Models for Embodied AI (2024, unknown): “We define a VLA as any model capable of processing multimodal inputs from vision and language to produce robot actions that accomplish embodied tasks, typically following the architecture in Figure 1.”
STMA: A Spatio-Temporal Memory Agent for Long-Horizon Embodied Task Planning (2025, unknown): “Despite recent advances in Artificial Intelligence (AI) systems powered by Large Language Models (LLMs), such as open-world games [1], dialogue systems [2], and personalized recommendation systems [3], there remain significant challenges in their applications to embodied intelligence [4, 5].”