A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks

By Hermann Georg Mayer, Faustino J. Gomez, Daan Wierstra, Istvan Nagy, Alois C. Knoll, J. Urgen Schmidhuber. 2006

241 citations recorded by the source corpus.

Subfields: Biomedical Engineering, Cardiology and Cardiovascular Medicine, Electrical and Electronic Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Cardiology and Cardiovascular Medicine

In Electrical and Electronic Engineering

In Reinforcement Learning

Citations in the corpus

End-to-End Training of Deep Visuomotor Policies (2015, methodology): “ work on neural network control used 3 Levine, Finn, Darrell, and Abbeel small, simple networks (Pomerleau, 1989; Hunt et al., 1992; Bekey and Goldberg, 1992; Lewis et al., 1998; Bakker et al., 2003; Mayer et al., 2006), and has largely been supplanted by methods that use carefully designed policies”
Deep learning in neural networks: An overview (2014, methodology): “LSTM is also applicable to robot localization (Förster, Graves, & Schmidhuber, 2007), robot control (Mayer et al., 2008), online driver distraction detection (Wöllmer et al., 2011), and many other tasks.”
Differential Recurrent Neural Networks for Action Recognition (2015, background): “Recently, Recurrent Neural Networks (RNNs) [28], especially Long Short-Term Memory (LSTM) model [14], have gained significant attention in solving many challenging problems involving time-series data, such as action recognition [12, 8, 13], multilingual machine translation [30, 4], multimodal transl”
Combining self-supervised learning and imitation for vision-based rope manipulation (2017, background): “Prior works have considered problems such as surgical suturing [8], [9], towel folding [7], knot tying and rope manipulation among many others.”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, background): “Many recent state-of-the-art results in sequence processing are based on LSTM, which learned to control robots [159], and was used to set benchmark records in prosody contour prediction [55] (IBM), textto-speech synthesis [54] (Microsoft), large vocabulary speech recognition [213] (Google), and mach”
Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-to-End Learning from Demonstration (2017, methodology): “In [19], LSTMs are used for a robot to learn to autonomously tie knots after a pre-processing step to reduce the noise from the training data.”
Evolution of autonomous and semi‐autonomous robotic surgical systems: a review of the literature (2011, methodology): “The authors used LSTMs for their experiments and trained them to learn to control the movement of a surgical manipulator to successfully tie a knot.”
Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions (2016, background): “published a series of works to this end [115,118,119,121].”
Cable manipulation with a tactile-reactive gripper (2019, background): “Cable/rope manipulation Manipulating wire-like objects has attracted attention in the robotics community [2], [3] with tasks including tying knots [4], surgical suturing [5], or wire reshaping [6], [7].”
A case study of trajectory transfer through non-rigid registration for a simplified suturing scenario (2013, methodology): “[9] used a recurrent neural network to as a controller for part of a surgical knot-tying task, based on demonstrations.”
Motion generation of robotic surgical tasks: Learning from expert demonstrations (2010, methodology): “The work in [ 3 ] presented promising results from a system that automated the wind up portion of a knot tying task using recurrent neural networks trained from expert surgeons.”
A guide to deep learning in healthcare (2019, background): “Modern genomic technologies collect a wide variety of measurements, from an individual’s DNA sequence to the quantity of”
Recent Advances in Robot Learning from Demonstration (2020, background): “Thanks to the recent developments in deep learning and the availability of impressive computational power, a class of end-to-end methods have recently been introduced (41, 43, 92, 93, 94).”
Imitation Learning (2017, methodology): “In Mayer et al. [2008], a robot is trained to automate part of a surgical procedure from surgeons’ demonstrations.”
Learning Recurrent Neural Networks with Hessian-Free Optimization (2011, background): “…exhibiting longterm dependencies, Hochreiter and Schmidhuber (1997) proposed a modified architecture called the Long Shortterm Memory (LSTM) and successfully applied it to speech and handwritten text recognition (Graves and Schmidhuber , 2009; 2005) and robotic control (Mayer et al., 2007).”
Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Data (2020, background): “Manipulating deformable objects is valuable for a wide variety of applications from surgery and manufacturing to household robotics [2, 8, 11, 14, 15, 19, 26, 37–39].”