AndroidEnv: A Reinforcement Learning Platform for Android

By Daniel Toyama, Philippe Hamel, Anita Gergely, Gheorghe Comanici, Amelia Glaese, Zafarali Ahmed, Tyler Jackson, Shibl Mourad, Doina Precup. 2021

108 citations recorded by the source corpus.

We introduce AndroidEnv, an open-source platform for Reinforcement Learning (RL) research built on top of the Android ecosystem. AndroidEnv allows RL agents to interact with a wide variety of apps and services commonly used by humans through a universal touchscreen interface. Since agents train on a realistic simulation of an Android device, they have the potential to be deployed on real devices. In this report, we give an overview of the environment, highlighting the significant features it provides for research, and we present an empirical evaluation of some popular reinforcement learning agents on a set of tasks built on this platform.

Subfields: Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

1. Sergey Levine
2. Ofir Nachum
3. Yuke Zhu
4. Li Fei-Fei
5. Pieter Abbeel
6. Ajay Mandlekar
7. Shixiang Shane Gu
8. Karthik Narasimhan

Citations in the corpus

• Augmented Language Models: a Survey (2023, background): “While there are other works on web navigation and computer-control, most of them assume the typical human interface, that takes as input images of a computer screen and output keyboard commands in order to solve digital tasks (Shi et al., 2017; Gur et al., 2019; 2021; Toyama et al., 2021; Humphreys ”
• WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents (2022, background): “While not directly operating on web pages, AndroidEnv [48] and MoTIF [8] provide environments to train agents for interacting with apps and services on mobile platforms.”
• MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge (2022, methodology): “Prior works include maze-style worlds [121, 129, 61], purely text-based game [69], grid worlds [21, 16], browser/GUI-based environments [108, 124], and indoor simulators for robotics [1, 107, 114, 34, 110, 99, 89].”
• Understanding HTML with Large Language Models (2022, methodology): “…HTML understanding is to train specialized models, i.e., gathering application-specific datasets and designing neural network (NN) architectures to leverage inductive biases of the HTML’s structure; see, e.g., Liu et al. (2018); Toyama et al. (2021); Gur et al. (2019, 2021); Humphreys et al. (2022).”
• OS-ATLAS: A Foundation Action Model for Generalist GUI Agents (2024, methodology): “For Android, we utilize AndroidEnv (Toyama et al., 2021) to create a simulation environment, and for Linux, we employ OSWorld (Xie et al., 2024).”
• A data-driven approach for learning to control computers (2022, background): “One domain that has these two ingredients but has received relatively less attention is digital device control (Chen et al., 2021; Gur et al., 2021; Li et al., 2020; Liu et al., 2018; Nakano et al., 2021; Shi et al., 2017; Shvo et al., 2021; Toyama et al., 2021), which comprises the use of digital d”
• WebArena: A Realistic Web Environment for Building Autonomous Agents (2023, background): “While AndroidEnv (Toyama et al., 2021) replicates an Android setup, it does not guarantee the reproducibility since live Android applications are used.”
• AgentBench: Evaluating LLMs as Agents (2023, background): “More recently, attempts on embodied agents [63; 35; 2] have employed complicated multi-modal simulators based on games [41; 22], GUI [69; 81], and indoor scenes [68; 74].”
• AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents (2024, methodology): “It connects agents to Android OS by leveraging the Python library AndroidEnv [53] to connect to the freely available Android Emulator.”
• VIMA: General Robot Manipulation with Multimodal Prompts (2022, background): “, 2023), powered by a collection of learning environments (Abramson et al., 2020; Shridhar et al., 2020; Savva et al., 2019; Puig et al., 2018; Team et al., 2021; Toyama et al., 2021; Shi et al., 2017).”
• Android in the Wild: A Large-Scale Dataset for Android Device Control (2023, methodology): “Our recording system uses AndroidEnv [40] with the Android Emulator.”
• A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis (2023, background): “Prior works have realized the web automation via reinforcement learning [23, 28, 29, 58], finetuned [19, 24] or prompted LLMs [30, 63, 82, 86] on the simulated websites [59, 69, 81].”