Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems

By Boyi Liu, Lujia Wang, Ming Liu. 2019

276 citations recorded by the source corpus.

This letter was motivated by the problem of how to make robots fuse and transfer their experience so that they can effectively use prior knowledge and quickly adapt to new environments. To address the problem, we present a learning architecture for navigation in cloud robotic systems: Lifelong Federated Reinforcement Learning (LFRL). In the letter, we propose a knowledge fusion algorithm for upgrading a shared model deployed on the cloud. Then, effective transfer learning methods in LFRL are introduced. LFRL is consistent with human cognitive science and fits well in cloud robotic systems. Experiments show that LFRL greatly improves the efficiency of reinforcement learning for robot navigation. The cloud robotic system deployment also shows that LFRL is capable of fusing prior knowledge. In addition, we release a cloud robotic navigation-learning website to provide the service based on LFRL: www.shared-robotics.com.

Subfields: Computer Networks and Communications, Control and Systems Engineering, Mechanical Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Control and Systems Engineering

In Mechanical Engineering

Citations in the corpus

FedML: A Research Library and Benchmark for Federated Machine Learning (2020, background): “…computer vision [77, 78], natural language processing [79, 80, 81, 82, 83], ﬁnance [74, 35, 84], transportation [85, 86, 87, 88, 89, 90, 91, 92, 93, 94], digital health [95, 96, 97, 98, 99, 100, 101], recommendation [102, 103, 104, 105, 106, 107], robotics [108, 109], and smart cities [110, 111].”
A Survey on Federated Learning: The Journey From Centralized to Distributed On-Site Learning and Beyond (2021, methodology): “found its way into many other areas, such as edge computing [74], networking [75], robotics [12], grid-world [76], FL enhancement [77], recommender system [78], cybersecu-”
A Survey on Federated Learning for Resource-Constrained IoT Devices (2021, unknown): “Smart Robotics: A lifelong reinforcement FL framework for mobile robots is proposed in [50].”
A Tutorial on Ultrareliable and Low-Latency Communications in 6G: Integrating Domain Knowledge Into Deep Learning (2021, background): “For example, by combining the federated learning framework with lifelong learning, a lifelong federated reinforcement learning framework for cloud robotic systems was proposed by Liu et al. [111].”
Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges (2020, background): “, [40] Reinforcement learning Smart mobile robotics”
Federated Learning for Internet of Things: A Comprehensive Survey (2021, background): “The authors in [163], [164] investigate a federated imitation learning scheme for cloud robotics.”
Federated Learning on Non-IID Data: A Survey (2021, background): “combine lifelong and reinforcement learning to form a lifelong federated reinforcement learning (LFRL) architecture [87].”
Federated Imitation Learning: A Privacy Considered Imitation Learning Framework for Cloud Robotic Systems with Heterogeneous Sensor Data (2019, background): “To realize the goal, we focus on cloud robotic systems [4].”
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection (2019, methodology): “horizontal Federated MTL [169] multi-task learning Federated Meta-Learning [33] NN meta-learning Personalized FedAvg [81] LFRL [115] reinforcement learning FBO [44] LM Bayesian optimization Structure Updates [95]”
Federated Learning for 6G: Applications, Challenges, and Opportunities (2021, background): “In [19], the goal of FRL is to enable wireless devices to”
Federated Transfer Reinforcement Learning for Autonomous Driving (2019, background): “[14] studied the FRL settings in the autonomous navigation where the main task is to make the robots fuse and transfer their experience so that they can effectively use prior knowledge and quickly adapt to new environments.”