Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space-Time Popularities

By Alireza Sadeghi, Fatemeh Sheikholeslami, Georgios B. Giannakis. 2017

231 citations recorded by the source corpus.

Small basestations (SBs) equipped with caching units have potential to handle the unprecedented demand growth in heterogeneous networks. Through low-rate, backhaul connections with the backbone, SBs can prefetch popular files during off-peak traffic hours, and service them to the edge at peak periods. To intelligently prefetch, each SB must learn what and when to cache, while taking into account SB memory limitations, the massive number of available contents, the unknown popularity profiles, as well as the space-time popularity dynamics of user file requests. In this paper, local and global Markov processes model user requests, and a reinforcement learning (RL) framework is put forth for finding the optimal caching policy when the transition probabilities involved are unknown. Joint consideration of global and local popularity demands along with cache-refreshing costs allow for a simple, yet practical asynchronous caching approach. The novel RL-based caching relies on a Q-learning algorithm to implement the optimal policy in an online fashion, thus, enabling the cache control unit at the SB to learn, track, and possibly adapt to the underlying dynamics. To endow the algorithm with scalability, a linear function approximation of the proposed Q-learning scheme is introduced, offering faster convergence as well as reduced complexity and memory requirements. Numerical tests corroborate the merits of the proposed approach in various realistic settings.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning (2018, methodology): “Particularly, related studies on edge computing and caching, such as [7] [8], have shown that reinforcement”
Machine Learning for 5G/B5G Mobile and Wireless Communications: Potential, Limitations, and Future Directions (2019, methodology): “The caching problem has been formulated as a Markov process with an unknown transition probability and is solved using reinforcement learning to find the optimal caching policy to adapt to the underlying dynamics of 5G [83].”