Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective

By Danielle Bragg, Oscar Koller, Mary Bellard, Larwan Berke, Patrick Boudreault, Annelies Braffort, Naomi K. Caselli, Matt Huenerfauth, Hernisa Kacorri, T. Verhoef and 2 more. International ACM SIGACCESS Conference on Computers and Accessibility, 2019

457 citations recorded by the source corpus.

Developing successful sign language recognition, generation, and translation systems requires expertise in a wide range of fields, including computer vision, computer graphics, natural language processing, human-computer interaction, linguistics, and Deaf culture. Despite the need for deep interdisciplinary knowledge, existing research occurs in separate disciplinary silos, and tackles separate portions of the sign language processing pipeline. This leads to three key questions: 1) What does an interdisciplinary view of the current landscape reveal? 2) What are the biggest challenges facing the field? and 3) What are the calls to action for people working in the field? To help answer these questions, we brought together a diverse group of experts for a two-day workshop. This paper presents the results of that interdisciplinary workshop, providing key background that is often overlooked by computer scientists, a review of the state-of-the-art, a set of pressing challenges, and a call to action for the research community.

Subfields: Cognitive Neuroscience, Developmental and Educational Psychology, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/c7fd5973a2cb3f7c6a5df0120400bc47b2f2bacf

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Developmental and Educational Psychology

In Human-Computer Interaction

Citations in the corpus

Artificial intelligence in the creative industries: a review (2020, background): “Moreover, visual and linguistic modelling has been combined to enable translation between spoken/written language and continuous sign language videos (Bragg et al. 2019).”
Improving Sign Language Translation with Monolingual Data by Sign Back-Translation (2021, unknown): “Due to the signiﬁcant social impact and the cross-modality challenge, sign language understanding has been attracting more and more research attention [1, 4, 10, 17, 22, 26, 37, 48].”
Designing an Online Infrastructure for Collecting AI Data From People With Disabilities (2021, background): “To the Deaf community, a sign language is considered not only a means of communication but also a cherished cultural element that identifes the community [8].”
How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language (2020, methodology): “One of the most important factors that has hindered the progress of sign language processing research is the absence of large scale annotated datasets [6].”
IncluSet: A Data Surfacing Repository for Accessibility Datasets (2020, background): “However, scarcity of large datasets generated from people with disabilities that can be used in AI-infused technologies remain one of the feld’s biggest challenges [2].”
Sign Language Transformers: Joint End-to-End Sign Language Recognition and Translation (2020, background): “The end goal of computational sign language research is to build translation and production systems [16], that are capable of translating sign language videos to spoken language sentences and vice versa, to ease the daily lives of the Deaf [15, 6].”