Constrained EV Charging Scheduling Based on Safe Deep Reinforcement Learning
By Hepeng Li, Zhiqiang Wan, Haibo He. IEEE Transactions on Smart Grid, 2020
353 citations recorded by the source corpus.
Electric vehicles (EVs) have been popularly adopted and deployed over the past few years because they are environment-friendly. When integrated into smart grids, EVs can operate as flexible loads or energy storage devices to participate in demand response (DR). By taking advantage of time-varying electricity prices in DR, the charging cost can be reduced by optimizing the charging/discharging schedules. However, since there exists randomness in the arrival and departure time of an EV and the electricity price, it is difficult to determine the optimal charging/discharging schedules to guarantee that the EV is fully charged upon departure. To address this issue, we formulate the EV charging/discharging scheduling problem as a constrained Markov Decision Process (CMDP). The aim is to find a constrained charging/discharging scheduling strategy to minimize the charging cost as well as guarantee the EV can be fully charged. To solve the CMDP, a model-free approach based on safe deep reinforcement learning (SDRL) is proposed. The proposed approach does not require any domain knowledge about the randomness. It directly learns to generate the constrained optimal charging/discharging schedules with a deep neural network (DNN). Unlike existing reinforcement learning (RL) or deep RL (DRL) paradigms, the proposed approach does not need to manually design a penalty term or tune a penalty coefficient. Numerical experiments with real-world electricity prices demonstrate the effectiveness of the proposed approach.
Subfields: Automotive Engineering, Electrical and Electronic Engineering
View Paper: https://www.semanticscholar.org/paper/bed9daf903a3a2a1fb9607812f7dfde9456339ae
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Automotive Engineering
In Electrical and Electronic Engineering
Citations in the corpus
- Enabling Massive IoT Toward 6G: A Comprehensive Survey (2021, background): “Unlike previous works [125], with more practical assumptions, e.g., unknown electricity market prices, the authors of [96] adopted a model-free RL approach to learn an optimized control policy for storage charging/discharging.”
- Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges (2021, methodology): “Some common methods to deal with constraints include 1) formulating the constraint violation as a penalty term to the reward, 2) training the control policy based on a constrained MDP [93], [122], 3) adding a heuristic safety layer to adjust the actions such that the constraints are respected.”
- Reinforcement Learning and Its Applications in Modern Power and Energy Systems: A Review (2020, background): “To ensure that the battery operates within the allowable ranges, [82] models the charging of EV as a constrained MDP and solves it by the safe DRL.”
- A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, background): “, autonomous driving [98, 122, 144, 130, 133, 188, 189, 288], robotics [90, 92, 214, 216, 256, 272], video compression [177], vehicle schedule [22, 159], etc.”
- A Review of Safe Reinforcement Learning: Methods, Theories, and Applications (2024, neutral): “…and robotics of safe RL applications, safe RL is also adopted to ensure safety in recommender systems [231], video compression [70], video transmissions [232], wireless security [233], satellite docking [234], edge computing [235], chemical processes [43] and vehicle schedule [1], [71], and so on.”