Related papers: Recognising BSL Fingerspelling in Continuous Signing Sequences

Recognising BSL Fingerspelling in Continuous Signing Sequences

URL: http://arxiv.org/abs/2603.19523v1
Date: Thu, 19 Mar 2026 23:29:31 GMT
Title: Recognising BSL Fingerspelling in Continuous Signing Sequences
Abstract summary: Fingerspelling is a critical component of British Sign Language (BSL)<n>Existing BSL fingerspelling datasets are either small in scale or temporally and letter-wise inaccurate.<n>We introduce a new large-scale BSL fingerspelling dataset, FS23K, constructed using an iterative annotation framework.
Score: 48.21890372724633
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Fingerspelling is a critical component of British Sign Language (BSL), used to spell proper names, technical terms, and words that lack established lexical signs. Fingerspelling recognition is challenging due to the rapid pace of signing and common letter omissions by native signers, while existing BSL fingerspelling datasets are either small in scale or temporally and letter-wise inaccurate. In this work, we introduce a new large-scale BSL fingerspelling dataset, FS23K, constructed using an iterative annotation framework. In addition, we propose a fingerspelling recognition model that explicitly accounts for bi-manual interactions and mouthing cues. As a result, with refined annotations, our approach halves the character error rate (CER) compared to the prior state of the art on fingerspelling recognition. These findings demonstrate the effectiveness of our method and highlight its potential to support future research in sign language understanding and scalable, automated annotation pipelines. The project page can be found at https://taeinkwon.com/projects/fs23k/.

Related papers

SignSeek: Learning Transferable Representations for Sign Dictionary Retrieval [20.105725837533722]
textbfSignSeek sets a new state-of-the-art performance in cross-corpus retrieval on sign languages.<n>It achieves zero-shot generalisation to an entirely unseen British Sign Language (BSL)
arXiv Detail & Related papers (2026-09-03T11:31:47Z)
OpenFS: Multi-Hand-Capable Fingerspelling Recognition with Implicit Signing-Hand Detection and Frame-Wise Letter-Conditioned Synthesis [7.940141427569929]
Fingerspelling recognition plays a crucial role in bridging the communication gap between Deaf and hearing communities.<n>We develop OpenFS, an open-source approach for fingerspelling recognition and synthesis.
arXiv Detail & Related papers (2026-02-26T12:41:24Z)
Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generator [55.94334001112357]
We introduce a multilingual sign language model, Signs as Tokens (SOKE), which can generate 3D sign avatars autoregressively from text inputs.<n>We propose a retrieval-enhanced SLG approach, which incorporates external sign dictionaries to provide accurate word-level signs.
arXiv Detail & Related papers (2024-11-26T18:28:09Z)
Fingerspelling within Sign Language Translation [0.9790236766474201]
Fingerspelling poses challenges for sign language processing due to its high-frequency motion and use for open-vocabulary terms. We evaluate how well sign language translation models understand fingerspelling in the context of entire sentences.
arXiv Detail & Related papers (2024-08-13T17:57:14Z)
Toward American Sign Language Processing in the Real World: Data, Tasks, and Methods [15.77894358993113]
I study automatic sign language processing in the wild, using signing videos collected from the Internet. I present three new large-scale ASL datasets in the wild: ChicagoFSWild, ChicagoFSWild+, and OpenASL. I propose two tasks for building real-world fingerspelling-based applications: fingerspelling detection and search.
arXiv Detail & Related papers (2023-08-23T20:38:19Z)
Weakly-supervised Fingerspelling Recognition in British Sign Language Videos [85.61513254261523]
Previous fingerspelling recognition methods have not focused on British Sign Language (BSL) In contrast to previous methods, our method only uses weak annotations from subtitles for training. We propose a Transformer architecture adapted to this task, with a multiple-hypothesis CTC loss function to learn from alternative annotation possibilities.
arXiv Detail & Related papers (2022-11-16T15:02:36Z)
Automatic dense annotation of large-vocabulary sign language videos [85.61513254261523]
We propose a simple, scalable framework to vastly increase the density of automatic annotations. We make these annotations publicly available to support the sign language research community.
arXiv Detail & Related papers (2022-08-04T17:55:09Z)
Fingerspelling Detection in American Sign Language [32.79935314131377]
We consider the task of fingerspelling detection in raw, untrimmed sign language videos. This is an important step towards building real-world fingerspelling recognition systems. We propose a benchmark and a suite of evaluation metrics, some of which reflect the effect of detection on the downstream fingerspelling recognition task.
arXiv Detail & Related papers (2021-04-03T02:11:09Z)
Skeleton Based Sign Language Recognition Using Whole-body Keypoints [71.97020373520922]
Sign language is used by deaf or speech impaired people to communicate. Skeleton-based recognition is becoming popular that it can be further ensembled with RGB-D based method to achieve state-of-the-art performance. Inspired by the recent development of whole-body pose estimation citejin 2020whole, we propose recognizing sign language based on the whole-body key points and features.
arXiv Detail & Related papers (2021-03-16T03:38:17Z)
BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues [106.21067543021887]
We show how to use mouthing cues from signers to obtain high-quality annotations from video data. The BSL-1K dataset is a collection of British Sign Language (BSL) signs of unprecedented scale.
arXiv Detail & Related papers (2020-07-23T16:59:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.