WebArena: A Realistic Web Environment for Building Autonomous Agents

By Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yonatan Bisk, Daniel Fried, Uri Alon and 1 more. International Conference on Learning Representations, 2023

1,666 citations recorded by the source corpus.

With advances in generative AI, there is now potential for autonomous agents to manage daily tasks via natural language commands. However, current agents are primarily created and tested in simplified synthetic environments, leading to a disconnect with real-world scenarios. In this paper, we build an environment for language-guided agents that is highly realistic and reproducible. Specifically, we focus on agents that perform tasks on the web, and create an environment with fully functional websites from four common domains: e-commerce, social forum discussions, collaborative software development, and content management. Our environment is enriched with tools (e.g., a map) and external knowledge bases (e.g., user manuals) to encourage human-like task-solving. Building upon our environment, we release a set of benchmark tasks focusing on evaluating the functional correctness of task completions. The tasks in our benchmark are diverse, long-horizon, and designed to emulate tasks that humans routinely perform on the internet. We experiment with several baseline agents, integrating recent techniques such as reasoning before acting. The results demonstrate that solving complex tasks is challenging: our best GPT-4-based agent only achieves an end-to-end task success rate of 14.41%, significantly lower than the human performance of 78.24%. These results highlight the need for further development of robust agents, that current state-of-the-art large language models are far from perfect performance in these real-life tasks, and that WebArena can be used to measure such progress.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/e41482f4ee984f17382f6cdd900df094d928be06

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

The rise and potential of large language model based agents: a survey (2025, positive): “…virtual simulation environments and the real physical world: Virtual environments are scenes-constrained, task-speciﬁc, and interacted with in a simulated manner [292,563], while real-world environments are boundless, accommodate a wide range of tasks, and interact with in a physical manner.”
SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents (2024, background): “Recent advances in Large Language Models (LLMs) such as GPT-4 (OpenAI, 2023) and LLaMA (Tou-vron et al., 2023) have significantly propelled the evolution of GUI agents (Gur et al., 2023a; Zhou et al., 2023).”
τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment (2025, neutral): “Following a long line of research into language agents [25, 27, 10, 13, 21], LLM tool use [24, 20, 9], and task-oriented dialog [5, 4, 1, 22, 6, 7, 8], τ -bench [26] is a recently introduced benchmark to measure the reliability of language agents in multi-turn task-oriented conversations such as…”
AppAgent: Multimodal Agents as Smartphone Users (2023, neutral): “Recent systems like Mind2Web [8], WebArena [53], WorkArena [9], and VisualWebArena [18] have leveraged LLMs to operate and interact with web pages, while others, such as OSWorld [41], extend similar capabilities to desktop PCs.”
Large Language Models for Information Retrieval: A Survey (2023, neutral): “The integration of LLMs into IR systems has brought about a fundamental change in how users engage with information and knowledge.”
WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models (2024, methodology): “Recent studies have explored the construction of text-based web browsing environments and how to instruct large language model agents to perform web navigation (Nakano et al., 2021; Gur et al., 2023; Zhou et al., 2023; Lu et al., 2023).”
AgentTuning: Enabling Generalized Agent Abilities for LLMs (2023, neutral): “While most tasks help others, Mind2Web shows little cross-task benefit, likely because it’s a single-round task, unlike others that are multi-round.”
MetaGPT: Meta Programming for Multi-Agent Collaborative Framework (2023, background): “This leads to challenges such as achieving coherent interactions, mitigating unproductive feedback loops, and guiding meaningful collaborative engagements [2, 3, 8, 9, 10].”
Cognitive Architectures for Language Agents (2023, background): “, 2023b), and websites (Shi et al., 2017; Nakano et al., 2021; Yao et al., 2022a; Zhou et al., 2023; Gur et al., 2023; Deng et al., 2023) as well as general code execution (Yang et al.”
SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (2023, background): “, 2023) or turned to the web as an interactive setting featuring tasks that require multiple steps to solve (Yao et al., 2022; Zhou et al., 2023; Deng et al., 2023; Liu et al., 2023d).”
τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (2024, background): “Existing benchmarks [27, 29, 12, 14, 16] for language agents often feature simplified instruction-following setups, where the agent autonomously interacts with an environment (web, code terminal, or APIs) given all the information upfront, without any human-in-the-loop interaction and without the…”
Understanding the planning of LLM agents: A survey (2024, methodology): “…include question-answering tasks based on the Wikipedia engine [Yao et al. , 2022] (such as HotPotQA [Yang et al. , 2018] and Fever [Thorne et al. , 2018]) and web browsing tasks to find specific information, including WebShop, Mind2Web [Deng et al. , 2023], and WebArena [Zhou et al. , 2023].”
GPT-4V(ision) is a Generalist Web Agent, if Grounded (2024, background): “GPT-4V also exhibits impressive capabilities, such as error correction and speculative planning.”
TravelPlanner: A Benchmark for Real-World Planning with Language Agents (2024, background): “…al., 2021; Patel et al., 2021); tool-use evaluating agents’ proficiency in employing tools and reporting results (Li et al., 2023; Xu et al., 2023; Zhuang et al., 2023); and web navigation, testing agents’ ability to locate specific websites (Deng et al., 2023; Zhou et al., 2024; Liu et al., 2024).”
AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors in Agents (2023, positive): “By harnessing the power of LLMs, autonomous agents can make more nuanced decisions and perform actions with an unprecedented degree of autonomy (Zhou et al., 2023).”