Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison

By Dongxu Li, Cristian Rodriguez-Opazo, Xin Yu, Hongdong Li. IEEE Workshop/Winter Conference on Applications of Computer Vision, 2019

648 citations recorded by the source corpus.

Vision-based sign language recognition aims at helping the deaf people to communicate with others. However, most existing sign language datasets are limited to a small number of words. Due to the limited vocabulary size, models learned from those datasets cannot be applied in practice. In this paper, we introduce a new large-scale Word-Level American Sign Language (WLASL) video dataset, containing more than 2000 words performed by over 100 signers. This dataset will be made publicly available to the research community. To our knowledge,it is by far the largest public ASL dataset to facilitate word-level sign recognition research.Based on this new large-scale dataset, we are able to experiment with several deep learning methods for word-level sign recognition and evaluate their performances in large scale scenarios. Specifically we implement and compare two different models,i.e., (i) holistic visual appearance based approach, and (ii) 2D human pose based approach. Both models are valuable baselines that will benefit the community for method benchmarking. Moreover, we also propose a novel pose-based temporal graph convolution networks (Pose-TGCN) that model spatial and temporal dependencies in human pose trajectories simultaneously, which has further boosted the performance of the pose-based method. Our results show that pose-based and appearance-based models achieve comparable performances up to 62.63% at top-10 accuracy on 2,000 words/glosses, demonstrating the validity and challenges of our dataset. Our dataset and baseline deep models are available at https://dxli94.github.io/WLASL/.

Subfields: Computer Vision and Pattern Recognition, Developmental and Educational Psychology, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/874063b6c71aef8382eb66a66d8a1c1188e78a9a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Lin Ma
  2. Luc Van Gool
  3. Hongdong Li
  4. Andrew Zisserman
  5. Wei David Liu
  6. Xavier Giró-i-Nieto
  7. Wenhan Luo
  8. Dacheng Tao

In Human-Computer Interaction

  1. Xavier Giró-i-Nieto
  2. B. Stenger
  3. Deepti Ghadiyaram
  4. Kaihao Zhang
  5. Hongdong Li
  6. Wei David Liu

Citations in the corpus