Language Models can Solve Computer Tasks
By Geunwoo Kim, Pierre Baldi, Stephen Marcus McAleer. 2023
522 citations recorded by the source corpus.
Agents capable of carrying out general tasks on a computer can improve efficiency and productivity by automating repetitive tasks and assisting in complex problem-solving. Ideally, such agents should be able to solve new computer tasks presented to them through natural language commands. However, previous approaches to this problem require large amounts of expert demonstrations and task-specific reward functions, both of which are impractical for new tasks. In this work, we show that a pre-trained large language model (LLM) agent can execute computer tasks guided by natural language using a simple prompting scheme where the agent Recursively Criticizes and Improves its output (RCI). The RCI approach significantly outperforms existing LLM methods for automating computer tasks and surpasses supervised learning (SL) and reinforcement learning (RL) approaches on the MiniWoB++ benchmark. We compare multiple LLMs and find that RCI with the InstructGPT-3+RLHF LLM is state-of-the-art on MiniWoB++, using only a handful of demonstrations per task rather than tens of thousands, and without a task-specific reward function. Furthermore, we demonstrate RCI prompting's effectiveness in enhancing LLMs' reasoning abilities on a suite of natural language reasoning tasks, outperforming chain of thought (CoT) prompting with external feedback. We find that RCI combined with CoT performs better than either separately. Our code can be found here: https://github.com/posgnu/rci-agent.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Code Generation & Program Synthesis
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jianfeng Gao
- Mohit Bansal
- Luke S. Zettlemoyer
- Percy Liang
- Graham Neubig
- Quoc V. Le
- Noah A. Smith
- Yejin Choi
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Jianfeng Gao
- Dacheng Tao
- Trevor Darrell
- Ming-Hsuan Yang
- Xiaogang Wang
- Jitendra Malik
- Andrew Zisserman
In Large Language Models
In Code Generation & Program Synthesis
Citations in the corpus
- Instruction Tuning with GPT-4 (2023, background): “Meanwhile, recent studies show that GPT-4 is capable of identifying and fixing its own mistakes, and accurately judging the quality of responses(Peng et al., 2023; Bai et al., 2022; Madaan et al., 2023; Kim et al., 2023).”
- Large Language Models as Optimizers (2023, background): “…(Shinn et al., 2023; Madaan et al., 2023) and code generation performance (Chen et al., 2023e; Olausson et al., 2023; Shinn et al., 2023; Chen et al., 2023b), dialogue applications (Nair et al., 2023; Madaan et al., 2023; Yuan et al., 2023), and so on (Kim et al., 2023; Wang et al., 2023).”
- Understanding the planning of LLM agents: A survey (2024, background): “Popular interactive programming environments involve issues related to operating systems, databases, etc., such as Agent Bench [Liu et al. , 2023c], MiniWoB++ [Kim and others, 2023].”
- Large Language Models Cannot Self-Correct Reasoning Yet (2023, methodology): “5-Turbo, the model examined in Kim et al. (2023); Shinn et al. (2023). Our goal is to reproduce the experiments described in the original papers. However, the papers do not specify which version was used1. We opt for gpt-3.5-turbo-0613, as we observed that the 0301 version almost always believes its”
- Training Language Models to Self-Correct via Reinforcement Learning (2024, unknown): “…in the first stage, it produces an initialization that is less susceptible to behavior collapse by training to correct second-attempt responses while constraining the first-turn distribution to be close to the base model; followed by training on both attempts to maximize reward in the second stage.”
- How Language Model Hallucinations Can Snowball (2023, background): “This solution is related to the “Review your previous answer and find problems with your answer.” prompt from Kim et al. (2023).”
- AgentBench: Evaluating LLMs as Agents (2023, background): “Later, a bunch of advanced reasoning strategies [40; 70; 88; 46; 98; 27] and applications [59; 64; 51; 1] for LLM-as-Agent have emerged and arouse much public interest.”
- AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents (2024, methodology): “The enthusiasm for building such agents is evident by the growing number of computer control agent systems [41, 9, 70, 24, 22, 16] and code repositories [13, 1, 2, 57].”
- The rise and potential of large language model based agents: a survey (2025, unknown): “To enable successful interactions between agents and realistic web pages, some researchers [294, 299] have started to leverage the powerful HTML reading and understanding abilities of LLMs.”
- Graph of Thoughts: Solving Elaborate Problems with Large Language Models (2023, methodology): “They are used to enhance different tasks, for example for code generation [17] or computer operation tasks [39].”
- WebArena: A Realistic Web Environment for Building Autonomous Agents (2023, background): “Existing works also incorporate failure recovery, self-correction (Shinn et al., 2023; Kim et al., 2023), observation summarization (Sridhar et al., 2023) to improve execution robustness.”
- Cognitive Architectures for Language Agents (2023, background): “…foreground a particular aspect of the problem (Bai et al., 2022; Jin et al., 2022; Ganguli et al., 2023; Madaan et al., 2023; Saunders et al., 2022; Kim et al., 2023; Kirk et al., 2023) or generate intermediate reasoning steps (Tafjord et al., 2021; Creswell et al., 2023; Yao et al., 2023) before…”
- GPT-4V(ision) is a Generalist Web Agent, if Grounded (2024, background): “Many works have focused on improving web agents relying on the HTML document (Deng et al., 2023; Gur et al., 2023; 2022; 2023; Kim et al., 2023; Sridhar et al., 2023).”
- Reflexion: language agents with verbal reinforcement learning (2023, methodology): “[10] use a retry pattern over a fixed number of steps without an evaluation step.”
- Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents (2024, unknown): “(Yan et al., 2023; Kim et al., 2023; Wang et al., 2024a; Zheng et al., 2024; Xie et al., 2024).”