Machine Learning Glove Using Self‐Powered Conductive Superhydrophobic Triboelectric Textile for Gesture Recognition in VR/AR Applications

By Feng Wen, Zhongda Sun, Tianyiyi He, Qiongfeng Shi, Minglu Zhu, Zixuan Zhang, Lianhui Li, Ting Zhang, Chengkuo Lee. Advancement of science, 2020

403 citations recorded by the source corpus.

The rapid progress of Internet of things (IoT) technology raises an imperative demand on human machine interfaces (HMIs) which provide a critical linkage between human and machines. Using a glove as an intuitive and low‐cost HMI can expediently track the motions of human fingers, resulting in a straightforward communication media of human–machine interactions. When combining several triboelectric textile sensors and proper machine learning technique, it has great potential to realize complex gesture recognition with the minimalist‐designed glove for the comprehensive control in both real and virtual space. However, humidity or sweat may negatively affect the triboelectric output as well as the textile itself. Hence, in this work, a facile carbon nanotubes/thermoplastic elastomer (CNTs/TPE) coating approach is investigated in detail to achieve superhydrophobicity of the triboelectric textile for performance improvement. With great energy harvesting and human motion sensing capabilities, the glove using the superhydrophobic textile realizes a low‐cost and self‐powered interface for gesture recognition. By leveraging machine learning technology, various gesture recognition tasks are done in real time by using gestures to achieve highly accurate virtual reality/augmented reality (VR/AR) controls including gun shooting, baseball pitching, and flower arrangement, with minimized effect from sweat during operation.

Subfields: Biomedical Engineering, Cognitive Neuroscience, Polymers and Plastics

View Paper: https://www.semanticscholar.org/paper/1f399026772d6823a85e3883970fdfcde075cadf

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

1. Chengkuo Lee
2. Xinge Guo
3. Long Liu
4. Tianyiyi He
5. Feng Wen
6. Qiongfeng Shi
7. Zixuan Zhang
8. Zhou Li

In Cognitive Neuroscience

1. Chengkuo Lee
2. Xinge Guo
3. Long Liu
4. Tianyiyi He
5. Feng Wen
6. Qiongfeng Shi
7. Zixuan Zhang
8. Jianxiong Zhu

In Polymers and Plastics

1. Chengkuo Lee
2. Long Liu
3. Xinge Guo
4. Tianyiyi He
5. Qiongfeng Shi
6. Zhou Li
7. Minyi Xu
8. Xiu Xiao

Citations in the corpus

• Recent progress in blue energy harvesting for powering distributed sensors in ocean (2021, background): “The subsequent studies have confirmed that TENG can obtain electric power from "disordered" energy in various environments, such as wave energy, human motion, and vibration energy in the surrounding [191-198].”
• Artificial Intelligence Meets Flexible Sensors: Emerging Smart Flexible Sensing Systems Driven by Machine Learning and Artificial Synapses (2023, background): “CNN can solve signal processing problems in flexible electronics and apply them to a variety of real-world scenarios [58].”
• Multidiscipline Applications of Triboelectric Nanogenerators for the Intelligent Era of Internet of Things (2022, methodology): “In addition to smart socks and insoles in the wearable field, Lee’s group also proposed a smart glove assisted with ML method, which uses self-powered conductive and superhydrophobic triboelectric textile for the gesture recognition in VR/AR applications [48].”
• AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove (2021, methodology): “As a proof of concept, our previous work demonstrated an AI-enabled TENG glove capable of recognizing similar and complex 11 gestures for advanced virtual reality/ augmented reality (VR/AR) controls 53 .”
• Progress in TENG technology—A journey from energy harvesting to nanoenergy and nanosystem (2020, methodology): “Copyright 2020, Wiley-VCH.(226) J, Haptic-feedback smart glove for virtual/augmented reality applications.”
• Promoting smart cities into the 5G era with multi-field Internet of Things (IoT) applications powered with advanced mechanical energy harvesters (2021, background): “6(j) [366].”