Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning

By Chongwen Huang, Ronghong Mo, Chau Yuen. IEEE Journal on Selected Areas in Communications, 2020

677 citations recorded by the source corpus.

Recently, the reconfigurable intelligent surface (RIS), benefited from the breakthrough on the fabrication of programmable meta-material, has been speculated as one of the key enabling technologies for the future six generation (6G) wireless communication systems scaled up beyond massive multiple input multiple output (Massive-MIMO) technology to achieve smart radio environments. Employed as reflecting arrays, RIS is able to assist MIMO transmissions without the need of radio frequency chains resulting in considerable reduction in power consumption. In this paper, we investigate the joint design of transmit beamforming matrix at the base station and the phase shift matrix at the RIS, by leveraging recent advances in deep reinforcement learning (DRL). We first develop a DRL based algorithm, in which the joint design is obtained through trial-and-error interactions with the environment by observing predefined rewards, in the context of continuous state and action. Unlike the most reported works utilizing the alternating optimization techniques to alternatively obtain the transmit beamforming and phase shifts, the proposed DRL based algorithm obtains the joint design simultaneously as the output of the DRL neural network. Simulation results show that the proposed algorithm is not only able to learn from the environment and gradually improve its behavior, but also obtains the comparable performance compared with two state-of-the-art benchmarks. It is also observed that, appropriate neural network parameter settings will improve significantly the performance and convergence rate of the proposed algorithm.

Subfields: Aerospace Engineering, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/376052e5d050db370592f687bc44c565b805f100

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Electrical and Electronic Engineering

Citations in the corpus

Multi-Hop RIS-Empowered Terahertz Communications: A DRL-Based Hybrid Beamforming Design (2021, background): “In terms of dynamic and mobile wireless scenarios, deep reinforcement learning (DRL) provides an effective solution by leveraging the advantages of deep learning, iterative updating and interacting with environments over the time [50],”
STAR: Simultaneous Transmission and Reflection for 360° Coverage by Intelligent Surfaces (2021, background): “Given these beneficial RIS characteristics, extensive industrial and academic research efforts have been devoted to the investigation of RISs, including but not limited to the design of energy efficient communication [5], the mitigation of blockages in millimeter wave (mmWave)/terahertz (THz) commun”
Smart Radio Environments Empowered by Reconfigurable Intelligent Surfaces: How it Works, State of Research, and Road Ahead (2020, background): “In [269], the authors propose a joint design of transmit”
Reconfigurable Intelligent Surfaces for Wireless Communications: Principles, Challenges, and Opportunities (2020, methodology): “ment learning (DRL) framework to jointly optimize transmitter precoding and RIS phase shifts has been developed in [51] and found to achieve comparable performance to the alternating optimization techniques proposed earlier in the literature.”
Deep Reinforcement Learning-Based Intelligent Reflecting Surface for Secure Wireless Communications (2020, background): “the authors in [28] and [29] proposed deep reinforcement learning (DRL) based approach to address the non-convex optimization problem, and the phase shifts at the IRS are optimized effectively.”
Physics-Based Modeling and Scalable Optimization of Large Intelligent Reflecting Surfaces (2020, background): “, the reinforcement learning-based IRS optimization in [43].”
Modeling and Architecture Design of Reconfigurable Intelligent Surfaces Using Scattering Parameter Network Analysis (2020, background): “In addition, research on optimizing RIS aided wireless systems with discrete phase shifts [25], [26], statistic and imperfect channel state information (CSI) [27], [28] and deep learning [29], [30] has also been conducted.”
Smart Radio Environments Empowered by Reconfigurable Intelligent Surfaces: How It Works, State of Research, and The Road Ahead (2020, methodology): “In [271], the authors propose a joint design of transmit beamforming and phase shifts in a RIS-assisted MIMO system, by using a deep reinforcement learning framework.”
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications (2021, background): “By considering CSI variations [275], [55], [276] and stochastic task arrivals [85], [84], the dynamic”
Channel Estimation for RIS-Empowered Multi-User MISO Wireless Communications (2020, methodology): “To lower the protocol overhead for CE in RIS-empowered systems, machine learning approaches have been lately presented [31]–[34].”
Intelligent Reflecting Surface Aided Multigroup Multicast MISO Communication Systems (2019, methodology): “A deep reinforcement learning (DRL)-based algorithm [7] and a mobile edge computing-based algorithm [13] were proposed”
Holographic MIMO Communications: Theoretical Foundations, Enabling Technologies, and Future Directions (2022, background): “[307]–[311], federated learning [312]–[314], and machine learning aided applications [315]–[318].”
Exploiting Intelligent Reflecting Surfaces in NOMA Networks: Joint Beamforming Optimization (2019, methodology): “With the target of maximizing the sum rate of the IRS-aided system, the joint optimization of active and passive beamforming was solved by utilizing the deep reinforcement learning in [35].”
An Overview of Signal Processing Techniques for RIS/IRS-Aided Wireless Systems (2021, neutral): “Examples of application of deep reinforcement learning to the optimization of RIS-aided communications can be found in [138]–[140].”
Intelligent Reflecting Surface-Aided Wireless Communications: A Tutorial (2020, background): “important to develop more advanced and computationally efficient algorithms such as machine learning based methods for IRS reflection design, especially for practically large IRSs [69], [70], [94], [95].”
Reconfigurable Intelligent Surfaces: Principles and Opportunities (2020, methodology): “[127] applied a deep deterministic policy gradient (DDPG) based algorithm for maximizing the throughput by utilizing the sum rate as instant rewards for training the DDPG model.”
Edge Learning for B5G Networks With Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing (2022, neutral): “ML methods need to centrally train data on a specific data center [2], [3], [4], [5], [6], [7].”
Simultaneously Transmitting And Reflecting (STAR) RIS Aided Wireless Communications (2021, methodology): “The authors of [18] invoked deep reinforcement learning to optimize the active and passive beamforming for maximization of the system sum rate based on partial CSI.”