MS-ASL: A Large-Scale Data Set and Benchmark for Understanding American Sign Language

By Hamid Reza Vaezi Joze, Oscar Koller. British Machine Vision Conference, 2018

332 citations recorded by the source corpus.

Sign language recognition is a challenging and often underestimated problem comprising multi-modal articulators (handshape, orientation, movement, upper body and face) that integrate asynchronously on multiple streams. Learning powerful statistical models in such a scenario requires much data, particularly to apply recent advances of the field. However, labeled data is a scarce resource for sign language due to the enormous cost of transcribing these unwritten languages. We propose the first real-life large-scale sign language data set comprising over 25,000 annotated videos, which we thoroughly evaluate with state-of-the-art methods from sign and related action recognition. Unlike the current state-of-the-art, the data set allows to investigate the generalization to unseen individuals (signer-independent test) in a realistic setting with over 200 signers. Previous work mostly deals with limited vocabulary tasks, while here, we cover a large class count of 1000 signs in challenging and unconstrained real-life recording conditions. We further propose I3D, known from video classifications, as a powerful and suitable architecture for sign language recognition, outperforming the current state-of-the-art by a large margin. The data set is publicly available to the community.

Subfields: Biomedical Engineering, Developmental and Educational Psychology, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/310a8e2c9f2650fa2e44fdf0d82d11c0cb3e387e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Developmental and Educational Psychology

In Human-Computer Interaction

Citations in the corpus

MMTM: Multimodal Transfer Module for CNN Fusion (2019, background): “There are multiple multimodal datasets available for this task [37, 41, 46, 47] and several previous fusion methods have reported their results on these datasets [36–39].”
Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective (2019, background): “, YouTube [62]) where signer provenance and skill is unknown.”
Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison (2019, background): “1We notice that an unpublished paper [32] aims at providing an ASL dataset containing 1,000 glosses.”
Sign Language Transformers: Joint End-to-End Sign Language Recognition and Translation (2020, background): “However, most of the research to date has mainly focused on Isolated Sign Language Recognition [35, 75, 72, 10, 63, 67], working on application specific datasets [11, 71, 23], thus limiting the applicability of such technologies.”