Geometry-Aware Metric Learning for Cross-Lingual Few-Shot Sign Language Recognition on Static Hand Keypoints

• URL: http://arxiv.org/abs/2603.09213v1
• Date: Tue, 10 Mar 2026 05:31:46 GMT
• Title: Geometry-Aware Metric Learning for Cross-Lingual Few-Shot Sign Language Recognition on Static Hand Keypoints
• Abstract summary: Cross-lingual few-shot transfer, pretraining on a data-rich source language, offers a scalable alternative.<n>We propose a geometry-aware metric-learning framework centered on a compact 20-dimensional inter-joint angle descriptor.<n>These angles are invariant to SO(3) rotation, translation, and isotropic scaling, eliminating the dominant sources of cross-dataset shift.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Sign language recognition (SLR) systems typically require large labeled corpora for each language, yet the majority of the world's 300+ sign languages lack sufficient annotated data. Cross-lingual few-shot transfer, pretraining on a data-rich source language and adapting with only a handful of target-language examples, offers a scalable alternative, but conventional coordinate-based keypoint representations are susceptible to domain shift arising from differences in camera viewpoint, hand scale, and recording conditions. This shift is particularly detrimental in the few-shot regime, where class prototypes estimated from only K examples are highly sensitive to extrinsic variance. We propose a geometry-aware metric-learning framework centered on a compact 20-dimensional inter-joint angle descriptor derived from MediaPipe static hand keypoints. These angles are invariant to SO(3) rotation, translation, and isotropic scaling, eliminating the dominant sources of cross-dataset shift and yielding tighter, more stable class prototypes. Evaluated on four fingerspelling alphabets spanning typologically diverse sign languages, ASL, LIBRAS, Arabic Sign Language, and Thai Sign Language, the proposed angle features improve over normalized-coordinate baselines by up to 25 percentage points within-domain and enable frozen cross-lingual transfer that frequently exceeds within-domain accuracy, using a lightweight MLP encoder with about 10^5 parameters. These findings demonstrate that invariant hand-geometry descriptors provide a portable and effective foundation for cross-lingual few-shot SLR in low-resource settings.

Related papers

• One Patch Is Enough: Reinforcement-Optimized Visual Token Grounding for MLLM-Based Scene Text Spotting [51.09550363815034]
  Scene text spotting requires high-precision alignment between textual recognition and spatial localization.<n>We propose Single-Patch Text Spotting (SPaTS), a vision-centric framework that routes each text instance through a single anchor visual token.<n>We show that SPaTS consistently and significantly outperforms both frontier closed-source MLLMs and OCR MLLMs.
  arXiv  Detail & Related papers  (2026-07-30T09:17:48Z)
• SIGNET: Motion-Level Knowledge Transfer for Cross-Language Sign Language Translation [20.105725837533722]
  We present textbf SIGNET, a framework that enables motion-level knowledge for cross-language sign language translation.<n>Our key is that, although sign languages differ in grammar and lexicon, pretrained models capture motion-level visual patterns that can be reused across datasets and languages.
  arXiv  Detail & Related papers  (2026-06-26T21:58:16Z)
• Bilingual Text-to-Motion Generation: A New Benchmark and Baselines [52.71312720094036]
  We introduce BiHumanML3D, the first bilingual text-to-motion benchmark, constructed via LLM-assisted annotation and rigorous manual correction.<n>We also propose a simple yet effective baseline, Bilingual Motion Diffusion synthesis (BiMD), featuring Cross-Lingual Alignment (CLA)<n>BiMD with CLA achieves an FID of 0.045 vs. 0.169 and R@3 of 82.8% vs. 80.8%, significantly outperforms monolingual diffusion models and translation baselines on BiHumanML3D.
  arXiv  Detail & Related papers  (2026-03-26T08:48:27Z)
• SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning [54.232148007248874]
  Current Sign Language Production (SLP) frameworks face a stark trade-off: direct text-to-pose models suffer from regression-to-the-mean effects.<n>We propose a novel training paradigm that leverages sparses to capture the true underlying distribution of human signing.<n>By predicting dense motion from these discrete anchors, our approach mitigates regression-to-the-mean while ensuring fluid articulation.
  arXiv  Detail & Related papers  (2026-03-11T06:02:36Z)
• Multilinguality as Sense Adaptation [24.548610248136352]
  SENse-based Symmetric Interlingual Alignment (SENSIA)<n>We introduce SENse-based Symmetric Interlingual Alignment (SENSIA)<n>It adapts a Backpack language model from one language to another by explicitly aligning sense-level mixtures and contextual representations on parallel data.
  arXiv  Detail & Related papers  (2026-01-15T11:44:01Z)
• PolyLingua: Margin-based Inter-class Transformer for Robust Cross-domain Language Detection [6.237314351305131]
  PolyLingua is a lightweight Transformer-based model for in-domain language detection and fine-grained language classification.<n>It employs a two-level contrastive learning framework combining instance-level separation and class-level alignment with adaptive margins.<n>It achieves 99.25% F1 and 98.15% F1, respectively, surpassing Sonnet 3.5 while using 10x fewer parameters.
  arXiv  Detail & Related papers  (2025-12-09T00:39:18Z)
• Language steering in latent space to mitigate unintended code-switching [1.1330938617817454]
  Large Language Models (LLMs) often exhibit unintended code-switching, reducing reliability in downstream tasks.<n>We propose latent-space language steering, a lightweight inference-time method that identifies language directions via PCA on parallel translations.<n>Our approach mitigates code-switching while preserving semantics with negligible computational overhead.
  arXiv  Detail & Related papers  (2025-10-11T19:49:38Z)
• Enhancing Robustness of Autoregressive Language Models against Orthographic Attacks via Pixel-based Approach [51.95266411355865]
  Autoregressive language models are vulnerable to orthographic attacks.<n>This vulnerability stems from the out-of-vocabulary issue inherent in subword tokenizers and their embeddings.<n>We propose a pixel-based generative language model that replaces the text-based embeddings with pixel-based representations by rendering words as individual images.
  arXiv  Detail & Related papers  (2025-08-28T20:48:38Z)
• Multi-Stream Keypoint Attention Network for Sign Language Recognition and Translation [3.976851945232775]
  Current approaches for sign language recognition rely on RGB video inputs, which are vulnerable to fluctuations in the background. We propose a multi-stream keypoint attention network to depict a sequence of keypoints produced by a readily available keypoint estimator. We carry out comprehensive experiments on well-known benchmarks like Phoenix-2014, Phoenix-2014T, and CSL-Daily to showcase the efficacy of our methodology.
  arXiv  Detail & Related papers  (2024-05-09T10:58:37Z)
• Efficient Spoken Language Recognition via Multilabel Classification [53.662747523872305]
  We show that our models obtain competitive results while being orders of magnitude smaller and faster than current state-of-the-art methods. Our multilabel strategy is more robust to unseen non-target languages compared to multiclass classification.
  arXiv  Detail & Related papers  (2023-06-02T23:04:19Z)
• Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts [56.72488923420374]
  Pre-trained language models (LMs) have shown great potential for cross-lingual transfer in low-resource settings. We show the few-shot cross-lingual transfer property of LMs for named recognition (NER) and apply it to solve a low-resource and real-world challenge of code-mixed (Spanish-Catalan) clinical notes de-identification in the stroke.
  arXiv  Detail & Related papers  (2022-04-10T21:46:52Z)
• Multi-Modal Zero-Shot Sign Language Recognition [51.07720650677784]
  We propose a multi-modal Zero-Shot Sign Language Recognition model. A Transformer-based model along with a C3D model is used for hand detection and deep features extraction. A semantic space is used to map the visual features to the lingual embedding of the class labels.
  arXiv  Detail & Related papers  (2021-09-02T09:10:39Z)
• FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding [85.29270319872597]
  We propose an enhanced fusion method that takes cross-lingual data as input for XLM finetuning. During inference, the model makes predictions based on the text input in the target language and its translation in the source language. To tackle this issue, we propose an additional KL-divergence self-teaching loss for model training, based on auto-generated soft pseudo-labels for translated text in the target language.
  arXiv  Detail & Related papers  (2020-09-10T22:42:15Z)
• Inducing Language-Agnostic Multilingual Representations [61.97381112847459]
  Cross-lingual representations have the potential to make NLP techniques available to the vast majority of languages in the world. We examine three approaches for this: (i) re-aligning the vector spaces of target languages to a pivot source language; (ii) removing language-specific means and variances, which yields better discriminativeness of embeddings as a by-product; and (iii) increasing input similarity across languages by removing morphological contractions and sentence reordering.
  arXiv  Detail & Related papers  (2020-08-20T17:58:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.