Optimized Computation Offloading Performance in Virtual Edge Computing Systems Via Deep Reinforcement Learning

By Xianfu Chen, Honggang Zhang, Celimuge Wu, Shiwen Mao, Yusheng Ji, Medhi Bennis. IEEE Internet of Things Journal, 2018

524 citations recorded by the source corpus.

To improve the quality of computation experience for mobile devices, mobile-edge computing (MEC) is a promising paradigm by providing computing capabilities in close proximity within a sliced radio access network (RAN), which supports both traditional communication and MEC services. Nevertheless, the design of computation offloading policies for a virtual MEC system remains challenging. Specifically, whether to execute a computation task at the mobile device or to offload it for MEC server execution should adapt to the time-varying network dynamics. This paper considers MEC for a representative mobile user in an ultradense sliced RAN, where multiple base stations (BSs) are available to be selected for computation offloading. The problem of solving an optimal computation offloading policy is modeled as a Markov decision process, where our objective is to maximize the long-term utility performance whereby an offloading decision is made based on the task queue state, the energy queue state as well as the channel qualities between mobile user and BSs. To break the curse of high dimensionality in state space, we first propose a double deep ${Q}$ -network (DQN)-based strategic computation offloading algorithm to learn the optimal policy without knowing a priori knowledge of network dynamics. Then motivated by the additive structure of the utility function, a ${Q}$ -function decomposition technique is combined with the double DQN, which leads to a novel learning algorithm for the solving of stochastic computation offloading. Numerical experiments show that our proposed learning algorithms achieve a significant improvement in computation offloading performance compared with the baseline policies.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/8c673cb6de4c38ba295e2557319c4e63c3654dd2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues (2018, methodology): “For surveyed literatures, this motivation applies to [15]–[17], [55], [56], [63], [64], [76]–[81], [85], [91], [97], [109], [110], [113], [118], [128]–[130], [133]–[135].”
A Review of Deep Reinforcement Learning for Smart Building Energy Management (2020, background): “, games [88] [31], autonomous driving [32]–[35], autonomous IoT [36], smart buildings [19] [20] [37] [38], smart city [39], wireless networks [40], Internet of energy [41], unmanned aerial vehicles [42], smart microgrids [43], edge computing [44], and manufacturing systems [45].”
Multi-Hop RIS-Empowered Terahertz Communications: A DRL-Based Hybrid Beamforming Design (2021, background): “In terms of dynamic and mobile wireless scenarios, deep reinforcement learning (DRL) provides an effective solution by leveraging the advantages of deep learning, iterative updating and interacting with environments over the time [50], [51], [53]–[64].”
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey (2018, methodology): “In contrast to [102], multiple BSs in an ultra-dense network is considered in [103] and [104], as shown in Fig.”
Edge Computing with Artificial Intelligence: A Machine Learning Perspective (2022, methodology): “[95] Q-function decomposition technique combined with double DQN Double deep Q-network to obtain optimal computing offloading without prior knowledge; a new function approximator-based DNN model to deal with high dimensional state spaces”
Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence (2019, background): “In [32] and [56], the computation offloading problem is formulated as an MDP with finite states and actions.”
Machine Learning for 5G/B5G Mobile and Wireless Communications: Potential, Limitations, and Future Directions (2019, background): “big data analytics [9], [10], mobile edge computing [11]–[13],”
Toward Edge Intelligence: Multiaccess Edge Computing for 5G and Internet of Things (2020, background): “A number of studies have been conducted on the problems of joint 4C (communication, computation, control, and caching) optimization [115], [116], mobile crowdsensing [117], joint big data analytics [118], caching [119], [120], and computation offloading [121], [122].”
Holistic Network Virtualization and Pervasive Network Intelligence for 6G (2023, background): “Network Operation [161] Computation offloading Minimizing average computing time of services and maximizing user computing experience Deep Q network”
Federated Learning-Based Computation Offloading Optimization in Edge Computing-Supported Internet of Things (2019, methodology): “…power with maximum limitation p trmax , which satisfies Thus, the total time for transmitting the input data µ is According to the proof in [4], given the association c i 2 N and the allocated energy units e i > 0 at a epoch i , the transmitting rate should remain a constant for achieving…”
Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks (2018, methodology): “There exists limited work on deep reinforcement learning-based offloading for MEC networks [18], [19], [20], [21], [22].”
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2019, methodology): “IoT edge environments powered by Energy Harvesting (EH) is investigated in [251], [254].”
A Survey of Multi-Access Edge Computing in 5G and Beyond: Fundamentals, Technology Integration, and State-of-the-Art (2019, unknown): “In [242], the DQN was deployed to learn the offloading decision and energy allocation of a representative mobile user in ultra-dense sliced RAN.”
Fast Adaptive Task Offloading in Edge Computing Based on Meta Reinforcement Learning (2020, background): “The task offloading problem in MEC has attracted significant research interests [5], [10], [11], [13], [29], [30], [31], [32], Fig.”
Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues (2020, background): “Considering the limited communication resources in vehicular environments, mobile edge computing (MEC) [4], [5], [6] has been widely discussed in providing a short delay for end users by conducting data caching and computation offloading to end users nearby.”