Perceptual user interfaces: multimodal interfaces that process what comes naturally

By Sharon L. Oviatt, Philip R. Cohen. CACM, 2000

462 citations recorded by the source corpus.

more transparent experience than ever before. Our voice, hands, and entire body, once augmented by sensors such as microphones and cameras, are becoming the ultimate transparent and mobile multimodal input devices. The area of multimodal systems has expanded rapidly during the past five years. Since Bolt’s [1] original “Put That There” concept demonstration, which processed speech and manual pointing during object manipulation, significant achievements have been made in developing more general multimodal systems. State-of-the-art multimodal speech and gesture systems now process complex gestural input other than pointing, and new systems have been extended to process different mode combinations—the most noteworthy being speech and pen input [9], and speech and lip movements [10]. As a foundation for advancing new multimodal systems, proactive empirical work has generated predictive information on human-computer multimodal interaction, which is being used to P U I Sharon Oviatt and Philip Cohen

Subfields: Artificial Intelligence, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/60f7303e11a26367c160dccf5bb48183f7934d29

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Human-Computer Interaction

Citations in the corpus

Multimodal interaction: A review (2014, background): “…eye towards more powerful ''butler-like'' interaction took on life in the push for ''perceptual interfaces'' (Turk, 1998; Turk and Robertson, 2000; Oviatt and Cohen, 2000; Turk and Kölsch, 2004), which seek to make the user interface more natural and compelling by taking advantage of the ways in…”
Eviza: A Natural Language Interface for Visual Analysis (2016, background): “[16] demonstrated that multi-modal input is richer than any one modality [27].”
The Gesture Pendant: A Self-illuminating, Wearable, Infrared Computer Vision System for Home Automation Control and Medical Monitoring (2000, methodology): “This multi-modal approach has already been implemented successfully in the QuickSet system, which enables users to create and position objects on a virtual map using speech and pen-based gestures [12].”
Cognitive and Usability Issues in Geovisualization (2001, background): “Oviatt and Cohen (2000) make the same contention from the perspective of computer science.”
Multimodal human discourse (2002, background): “[Oviatt et al. 1999; Oviatt and Cohen 2000; Oviatt 1999] extended this research direction by combining speech and natural language processing and pen-based gestures.”
Designing the User Interface for Multimodal Speech and Pen-Based Gesture Applications: State-of-the-Art Systems and Future Research Directions (2000, background): “To date, multimodal systems that combine either speech and pen input (Oviatt & Cohen, 2000) or speech and lip movements (Benoit, Martin, Pelachaud, Schomaker, & Suhm, 2000; Rubin, Vatikiotis-Bateson, & Benoit, 1998; Stork & Hennecke, 1995) constitute the two major research areas within the field.”