Efficient collective swimming by harnessing vortices through deep reinforcement learning

By Siddhartha Verma, Guido Novati, Petros Koumoutsakos. Proceedings of the National Academy of Sciences of the United States of America, 2018

428 citations recorded by the source corpus.

Significance Can fish reduce their energy expenditure by schooling? We answer affirmatively this longstanding question by combining state-of-the-art direct numerical simulations of the 3D Navier–Stokes equations with reinforcement learning, using recurrent neural networks with long short-term memory cells to account for the unsteadiness of the flow field. Surprisingly, we find that swimming behind a leader is not always associated with energetic benefits for the follower. In turn, we demonstrate that fish can improve their sustained propulsive efficiency by placing themselves at appropriate locations in the wake of other swimmers and intercepting their wake vortices judiciously. The results show that autonomous, “smart” swimmers may exploit unsteady flow fields to reap substantial energetic benefits and have promising implications for robotic swarms. Fish in schooling formations navigate complex flow fields replete with mechanical energy in the vortex wakes of their companions. Their schooling behavior has been associated with evolutionary advantages including energy savings, yet the underlying physical mechanisms remain unknown. We show that fish can improve their sustained propulsive efficiency by placing themselves in appropriate locations in the wake of other swimmers and intercepting judiciously their shed vortices. This swimming strategy leads to collective energy savings and is revealed through a combination of high-fidelity flow simulations with a deep reinforcement learning (RL) algorithm. The RL algorithm relies on a policy defined by deep, recurrent neural nets, with long–short-term memory cells, that are essential for capturing the unsteadiness of the two-way interactions between the fish and the vortical flow field. Surprisingly, we find that swimming in-line with a leader is not associated with energetic benefits for the follower. Instead, “smart swimmer(s)” place themselves at off-center positions, with respect to the axis of the leader(s) and deform their body to synchronize with the momentum of the oncoming vortices, thus enhancing their swimming efficiency at no cost to the leader(s). The results confirm that fish may harvest energy deposited in vortices and support the conjecture that swimming in formation is energetically advantageous. Moreover, this study demonstrates that deep RL can produce navigation algorithms for complex unsteady and vortical flow fields, with promising implications for energy savings in autonomous robotic swarms.

Subfields: Aerospace Engineering, Artificial Intelligence, Condensed Matter Physics

View Paper: https://www.semanticscholar.org/paper/c390dd6031632805a190c472e29349f28ec81ba5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Artificial Intelligence

In Condensed Matter Physics

Citations in the corpus

Machine Learning for Fluid Mechanics (2019, methodology): “Generative adversarial networks (GANs) (Goodfellow et al. 2014) are also being used to infer dynamical systems from data (Wu et al. 2018).”
A review on Deep Reinforcement Learning for Fluid Mechanics (2019, background): “It should also be noted that the successful use of LSTM cells 709 instead of regular fully-connected networks was advertised in swimming applications [13, 52], and that 710”
Deep learning methods for super-resolution reconstruction of turbulent flows (2020, background): “13,14 Deep learning technology has become a promising tool in these fields.”
Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control (2018, background): “…also to fluid mechanics (Kutz, 2017), but until this day those have been limited to a few applications, such as the definition of reduced order models (Wang et al., 2018), the effective control of swimmers (Verma et al., 2018), or performing Particle Image Velocimetry (PIV) (Rabault et al., 2017).”
Unsupervised deep learning for super-resolution reconstruction of turbulence (2020, methodology): “Recently, deep-reinforcement learning has been applied to fluid dynamics, such as observations of how swimmers efficiently use energy (Verma et al. 2018) and the development of a new flow-control scheme (Rabault et al. 2019).”