Online Learning for Offloading and Autoscaling in Energy Harvesting Mobile Edge Computing

By Jie Xu, Lixing Chen, Shaolei Ren. IEEE Transactions on Cognitive Communications and Networking, 2017

317 citations recorded by the source corpus.

Mobile edge computing (also known as fog computing) has recently emerged to enable in-situ processing of delay-sensitive applications at the edge of mobile networks. Providing grid power supply in support of mobile edge computing, however, is costly and even infeasible (in certain rugged or under-developed areas), thus mandating on-site renewable energy as a major or even sole power supply in increasingly many scenarios. Nonetheless, the high intermittency and unpredictability of renewable energy make it very challenging to deliver a high quality of service to users in energy harvesting mobile edge computing systems. In this paper, we address the challenge of incorporating renewables into mobile edge computing and propose an efficient reinforcement learning-based resource management algorithm, which learns on-the-fly the optimal policy of dynamic workload offloading (to the centralized cloud) and edge server provisioning to minimize the long-term system cost (including both service delay and operational cost). Our online learning algorithm uses a decomposition of the (offline) value iteration and (online) reinforcement learning, thus achieving a significant improvement of learning rate and run-time performance when compared to standard reinforcement learning algorithms such as ${Q}$ -learning. We prove the convergence of the proposed algorithm and analytically show that the learned policy has a simple monotone structure amenable to practical implementation. Our simulation results validate the efficacy of our algorithm, which significantly improves the edge computing performance compared to fixed or myopic optimization schemes and conventional reinforcement learning algorithms.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/e6134e7ae32e7b9b3337e47e8354dd43102850a4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

Optimized Computation Offloading Performance in Virtual Edge Computing Systems Via Deep Reinforcement Learning (2018, background): “In this context, the explosion in state space makes the conventional reinforcement learning algorithms [21]–[23] infeasible.”
Edge-Computing-Enabled Smart Cities: A Comprehensive Survey (2019, unknown): “There are significant variations in the harvested energy from both natural sources and radio-frequency sources [133].”
Price-Based Distributed Offloading for Mobile-Edge Computing With Computation Capacity Constraints (2017, background): “In general, the MEC paradigm can be divided into two categories: binary offloading [5]–[9] and partial offloading [10]–[12].”
Dynamic Task Offloading and Resource Allocation for Ultra-Reliable Low-Latency Edge Computing (2018, background): “powered by solar or wind energy, whereas the cloud server has a constant grid power [13].”
A Survey of Multi-Access Edge Computing in 5G and Beyond: Fundamentals, Technology Integration, and State-of-the-Art (2019, unknown): “The literature [106] jointly studied the offloading and autoscaling policy (i.”
Edge Computing with Artificial Intelligence: A Machine Learning Perspective (2022, methodology): “[91] propose an algorithm that uses the special structure of state transitions of the considered EC system to overcome the curse-of-dimensionality problem.”
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2019, background): “At this background, large state and action spaces in the whole offloading problem make the conventional learning algorithms [245], [256], [247] infeasible actually.”
Ensemble deep learning: A review (2021, methodology): “With the limitation of getting the complete data in real-world problems, online/incremental learning has been applied to various tasks like learning social representations [174], fog computing [175], identifying suspicious URLs [176], etc.”
Joint Service Caching and Task Offloading for Mobile Edge Computing in Dense Networks (2018, background): “stochastic task arrivals [7], [8], energy efficiency [9], [10], [11], collaboration [12], [13], [14] etc.”
Learning-Based Computation Offloading for IoT Devices With Energy Harvesting (2017, methodology): “processing speed of the edge server [15].”
Computation Rate Maximization in UAV-Enabled Wireless-Powered Mobile-Edge Computing Systems (2018, methodology): “Although resource allocation problems have been well studied in MEC systems [5]–[10], MEC systems relying on energy harvesting [11]–[16] and UAV-enabled MEC systems [21]–[25], few investigations have been conducted for designing resource allocation schemes in the UAV-enabled wireless powered MEC sys”