Semisupervised Deep Reinforcement Learning in Support of IoT and Smart City Services

By M. Mohammadi, Ala Al-Fuqaha, Mohsen Guizani, Jun-Seok Oh. 2018

376 citations recorded by the source corpus.

Smart services are an important element of the smart cities and the Internet of Things (IoT) ecosystems where the intelligence behind the services is obtained and improved through the sensory data. Providing a large amount of training data is not always feasible; therefore, we need to consider alternative ways that incorporate unlabeled data as well. In recent years, deep reinforcement learning (DRL) has gained great success in several application domains. It is an applicable method for IoT and smart city scenarios where auto-generated data can be partially labeled by users’ feedback for training purposes. In this paper, we propose a semisupervised DRL model that fits smart city applications as it consumes both labeled and unlabeled data to improve the performance and accuracy of the learning agent. The model utilizes variational autoencoders as the inference engine for generalizing optimal policies. To the best of our knowledge, the proposed model is the first investigation that extends DRL to the semisupervised paradigm. As a case study of smart city applications, we focus on smart buildings and apply the proposed model to the problem of indoor localization based on Bluetooth low energy signal strength. Indoor localization is the main component of smart city services since people spend significant time in indoor environments. Our model learns the best action policies that lead to a close estimation of the target locations with an improvement of 23% in terms of distance to the target and at least 67% more received rewards compared to the supervised DRL model.

Subfields: Building and Construction, Computer Science Applications, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Building and Construction

In Computer Science Applications

In Electrical and Electronic Engineering

Citations in the corpus

Deep Learning for IoT Big Data and Streaming Analytics: A Survey (2017, methodology): “In the ﬁeld of IoT, the work presented in [40] uses DRL in a semi-supervised setting for localization in smart campus environments.”
Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems (2021, background): “software agent on what actions to take in a certain environment so as to maximize the cumulative reward [132], [133].”
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey (2018, background): “Then, the authors in [181] introduce the semi-supervised DRL framework that utilizes variational auto-encoders [182] as an inference engine to infer the classification of unlabeled data.”
Deep Learning in Mobile and Wireless Networking: A Survey (2018, background): “In particular, their framework envisions a virtual agent in indoor environments [319], which can constantly receive state information during training, including signal strength indicators, current agent location, and the real (labeled data) and inferred (via a VAE) distance to the target.”
The Deep Learning Compiler: A Comprehensive Survey (2020, background): “It has not only demonstrated remarkable value in artificial intelligence such as natural language processing (NLP) [1] and computer vision (CV) [2], but also proved great success in broader applications such as e-commerce [3], smart city [4] and drug discovery [5].”
Enabling Cognitive Smart Cities Using Big Data and Machine Learning: Approaches and Challenges (2018, methodology): “The proposed approach was applied in a smart campus project as part of a smart city [11].”
Security and Privacy for Green IoT-Based Agriculture: Review, Blockchain Solutions, and Challenges (2020, background): “INTRODUCTION The Internet of Things (IoT) has been applied in many areas, such as smart farming [1], smart home [2], wearables [3], smart city [4], connected health [5], connected car [6], connected drones [7], among other areas.”
A Review of Deep Reinforcement Learning for Smart Building Energy Management (2020, background): “, games [88] [31], autonomous driving [32]–[35], autonomous IoT [36], smart buildings [19] [20] [37] [38], smart city [39], wireless networks [40], Internet of energy [41], unmanned aerial vehicles [42], smart microgrids [43], edge computing [44], and manufacturing systems [45].”
A Survey of Deep Learning: Platforms, Applications and Emerging Research Trends (2018, background): “) across centralized and distributed cloud computing frameworks, utilizing IoT devices and some novel mechanisms [67], [85], [95], [101], [134], [142], [160].”
Future Trends and Current State of Smart City Concepts: A Survey (2020, background): “Rapid development in IoT technologies motivates researchers and scientists in terms of creating new application areas and new IoT services [6], and these new smart services should highly meet the citizens’ needs worldwide.”
Low-Latency Federated Learning and Blockchain for Edge Association in Digital Twin Empowered 6G Networks (2020, background): “cases, such as intelligent transportation [2], [3] and smart city [4], are envisioned to provide high-quality services to users in the IIoT network.”