Magnetic control of tokamak plasmas through deep reinforcement learning
By Jonas Degrave, F. Felici, Jonas Buchli, Michael Neunert, Brendan D. Tracey, F. Carpanese, T. Ewalds, Roland Hafner, Abbas Abdolmaleki, D. de Las Casas and 21 more. 2022
923 citations recorded by the source corpus.
Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. In this work, we introduce a previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils. This architecture meets control objectives specified at a high level, at the same time satisfying physical and operational constraints. This approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations. We successfully produce and control a diverse set of plasma configurations on the Tokamak à Configuration Variable1,2, including elongated, conventional shapes, as well as advanced configurations, such as negative triangularity and ‘snowflake’ configurations. Our approach achieves accurate tracking of the location, current and shape for these configurations. We also demonstrate sustained ‘droplets’ on TCV, in which two separate plasmas are maintained simultaneously within the vessel. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied. A newly designed control architecture uses deep reinforcement learning to learn to command the coils of a tokamak, and successfully stabilizes a wide variety of fusion plasma configurations.
Subfields: Nuclear and High Energy Physics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
Citations in the corpus
- Acme: A Research Framework for Distributed Reinforcement Learning (2020, neutral): “, 2020), nuclear fusion (Degrave et al., 2022), and more.”
- Vygotskian Autotelic Artificial Intelligence: Language and Culture Internalization for Human-Like AI (2022, background): “5 But human chess world champions can also run, cook, draw a cat, or make a friend laugh.”
- Scientific discovery in the age of artificial intelligence (2023, methodology): “a, Leveraging AI for nuclear fusion control of complex and dynamic systems: Degrave et al.(166) developed an AI controller to regulate nuclear fusion”
- MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning (2021, background): “Recent progress shows the promise of applying RL to real-world applications, such as autonomous driving [16], [18], [50], the power system optimization in smart building [29], the surgical robotics arm [40], and even nuclear fusion [9].”
- Tool Learning with Foundation Models (2023, positive): “…showing great potential in various scientific scenarios, such as HyperTree Proof Search for proving Metamath theorems (Lample et al., 2022), protein structure prediction in structural biology (Jumper et al., 2021) and magnetic actuator coils controlling in nuclear physics (Degrave et al., 2022).”
- Reaching the limit in autonomous racing: Optimal control versus reinforcement learning (2023, background): “Yet there is a gap: RL can handle problems of high dimensionality and long planning horizons, albeit mostly in simulation [7, 14–17], and has only recently [6] been shown to outperform humans in a real-world sport.”