ActioNet: An Interactive End-To-End Platform For Task-Based Data Collection And Augmentation In 3D Environment

By Jiafei Duan, Samson Yu, Hui Li Tan, Cheston Tan. 2020

9 citations recorded by the source corpus.

The problem of task planning for artificial agents remains largely unsolved. While there has been increasing interest in data-driven approaches for the study of task planning for artificial agents, a significant remaining bottleneck is the dearth of large-scale comprehensive task-based datasets. In this paper, we present ActioNet, an interactive end-to-end platform for data collection and augmentation of task-based dataset in 3D environment. Using ActioNet, we collected a large-scale comprehensive task-based dataset, comprising over 3000 hierarchical task structures and videos. Using the hierarchical task structures, the videos are further augmented across 50 different scenes to give over 150,000 video. To our knowledge, ActioNet is the first interactive end-to-end platform for such task-based dataset generation and the accompanying dataset is the largest task-based dataset of such comprehensive nature. The ActioNet platform and dataset will be made available to facilitate research in hierarchical task planning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jiafei Duan
2. Samson Yu
3. Cheston Tan
4. Bolei Zhou
5. Animesh Garg
6. Bo Lu
7. Tangyao Li
8. Jiankai Sun

In Computer Vision and Pattern Recognition

1. De-An Huang
2. Cheston Tan
3. Animesh Garg
4. Jiafei Duan
5. Samson Yu
6. Yunhui Liu
7. Bolei Zhou
8. Li Fei-Fei

Citations in the corpus

• A Survey of Embodied AI: From Simulators to Research Tasks (2021, background): “These embodied AI simulators also facilitate the collection of of task-based dataset [10, 11] which are tedious to collect in real-world as”
• SPACE: A Simulator for Physical Interactions and Causal Learning in 3D Environments (2021, background): “But with the advancement and rise of deep learning, computer graphics, and embodied AI [22, 23, 24], there has been a paradigm shift towards generating synthetic datasets that range from simple 2D cartoons [25, 26, 27] to realistic interaction in 3D environments [13, 28, 14, 29, 30, 15], all with th”
• PlaTe: Visually-Grounded Planning With Transformers in Procedural Tasks (2021, methodology): “We evaluate PlaTe on an instructional video dataset CrossTask [16] and an interactive dataset ActioNet [30], which is based on AI2-THOR [31].”