Enabling always-available input with muscle-computer interfaces

By T. Scott Saponas, Desney S. Tan, Dan Morris, Ravin Balakrishnan, Jim Turner, James A. Landay. ACM Symposium on User Interface Software and Technology, 2009

360 citations recorded by the source corpus.

Previous work has demonstrated the viability of applying offline analysis to interpret forearm electromyography (EMG) and classify finger gestures on a physical surface. We extend those results to bring us closer to using muscle-computer interfaces for always-available input in real-world applications. We leverage existing taxonomies of natural human grips to develop a gesture set covering interaction in free space even when hands are busy with other objects. We present a system that classifies these gestures in real-time and we introduce a bi-manual paradigm that enables use in interactive systems. We report experimental results demonstrating four-finger classification accuracies averaging 79% for pinching, 85% while holding a travel mug, and 88% when carrying a weighted bag. We further show generalizability across different arm postures and explore the tradeoffs of providing real-time visual feedback.

Subfields: Biomedical Engineering, Cognitive Neuroscience, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/3fd89cf73d1705befe7a08dcc5df2abc6271d496

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Cognitive Neuroscience

In Human-Computer Interaction

Citations in the corpus

Tomo: Wearable, Low-Cost Electrical Impedance Tomography for Hand Gesture Recognition (2015, methodology): “The EMG-based system created by Saponas et al. [21] supported the pinch gesture set at 77.0% accuracy (and required conductive gel to provide sufficient SNR).”
Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor (2012, background): “Given their lightweight form-factor, a variety of on-body placements including forearm [34] and wrist-worn [18, 19, 32] have been demonstrated.”
EMG feature evaluation for improving myoelectric pattern recognition robustness (2013, background): “The stable feature should succeed in maintaining high classification accuracy (>80”
OmniTouch: wearable multitouch interaction everywhere (2011, background): “The hand requires buttons to be 22.5mm in diameter, offering the highest un-instrumented, on-body button density to date [8,24,32].”
3D Gestural Interaction: The State of the Field (2013, background): “A device that contains EMG sensors is attached to a user’s wrist and collects muscle data about fingertip movement and can then detect a variety of different finger gestures [21, 22].”
Soli (2016, background): “As mobile computing grows, new modes of interaction are emerging and becoming feasible, including touch and touchless gestures using camera-based tracking or capacitive field sensors, voice and gaze input, and a multitude of sensors embedded in various objects, the human body, clothes and environmen”
ViBand: High-Fidelity Bio-Acoustic Sensing Using Commodity Smartwatch Accelerometers (2016, background): “Approaches can be passive, such as electromyography [45, 46], where gestures are classified by measuring the electrical signals caused by muscle activation, or active [9, 41], as in Touché [47] and Tomo [55], where a signal is injected into the body to detect hand gestures.”
Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum (2016, neutral): “6% [13], 85% [29]).”
Feature Extraction and Selection for Myoelectric Control Based on Wearable EMG Sensors (2018, methodology): “To create EMG-based human–computer interfaces to be used in an everyday context, they should be simple and non-invasive, such as a watch, an armband, jewelry, or concealed beneath clothing [3].”