Soft robot perception using embedded soft sensors and recurrent neural networks

By Thomas George Thuruthel, Benjamin Shih, Cecilia Laschi, Michael Thomas Tolley. Science Robotics, 2019

595 citations recorded by the source corpus.

Recurrent neural networks with an unstructured redundant soft sensor topology allow robust multimodal proprioceptive capabilities. Recent work has begun to explore the design of biologically inspired soft robots composed of soft, stretchable materials for applications including the handling of delicate materials and safe interaction with humans. However, the solid-state sensors traditionally used in robotics are unable to capture the high-dimensional deformations of soft systems. Embedded soft resistive sensors have the potential to address this challenge. However, both the soft sensors—and the encasing dynamical system—often exhibit nonlinear time-variant behavior, which makes them difficult to model. In addition, the problems of sensor design, placement, and fabrication require a great deal of human input and previous knowledge. Drawing inspiration from the human perceptive system, we created a synthetic analog. Our synthetic system builds models using a redundant and unstructured sensor topology embedded in a soft actuator, a vision-based motion capture system for ground truth, and a general machine learning approach. This allows us to model an unknown soft actuated system. We demonstrate that the proposed approach is able to model the kinematics of a soft continuum actuator in real time while being robust to sensor nonlinearities and drift. In addition, we show how the same system can estimate the applied forces while interacting with external objects. The role of action in perception is also presented. This approach enables the development of force and deformation models for soft robotic systems, which can be useful for a variety of applications, including human-robot interaction, soft orthotics, and wearable robotics.

Subfields: Biomedical Engineering, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/e6d1c54df5561132b748c3e727ba5f21bd93c061

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Control and Systems Engineering

Citations in the corpus

ReSkin: versatile, replaceable, lasting tactile skins (2021, background): “Several works on soft sensors have used neural networks for sensor characterizations [27, 28], but these models are often trained on single sensor prototypes, and do not necessarily transfer to new copies of the sensor.”
Haptic-feedback smart glove as a creative human-machine interface (HMI) for virtual/augmented reality applications (2020, background): “By leveraging the proper learning model for specific sensing application, more comprehensive information can be extracted for those simply designed sensors, such as motion sequences, touch forces, and slip-ping ( 30 ).”
Electronic skins and machine learning for intelligent soft robots (2020, background): “In addition, researchers have also focused on recurrent neural networks, which have been shown to be advantageous for learning patterns in time series data (36, 39, 61, 62).”