MediaPipe Hands: On-device Real-time Hand Tracking

By Fan Zhang, Valentin Bazarevsky, Andrey Vakunov, A. Tkachenka, George Sung, Chuo-Ling Chang, Matthias Grundmann. arXiv.org, 2020

1,111 citations recorded by the source corpus.

We present a real-time on-device hand tracking pipeline that predicts hand skeleton from single RGB camera for AR/VR applications. The pipeline consists of two models: 1) a palm detector, 2) a hand landmark model. It's implemented via MediaPipe, a framework for building cross-platform ML solutions. The proposed model and pipeline architecture demonstrates real-time inference speed on mobile GPUs and high prediction quality. MediaPipe Hands is open sourced at https://mediapipe.dev.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/84b19524609ad75f309be7f87bcea783e6ecd337

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Yu Li
2. Chun Yuan
3. Zhendong Yang
4. Ailing Zeng
5. Wenhan Yang
6. Jiaying Liu

In Human-Computer Interaction

1. Ailing Zeng

Citations in the corpus

• Effective Whole-body Pose Estimation with Two-stages Distillation (2023, methodology): “Due to the lack of whole-body annotations, most previous models are designed for bodyonly [22, 40, 47, 53], hand-only [6, 32, 62, 68], or faceonly [21, 46, 67].”