SignAgent: Agentic LLMs for Linguistically-Grounded Sign Language Annotation and Dataset Curation

• URL: http://arxiv.org/abs/2603.19059v1
• Date: Thu, 19 Mar 2026 15:52:49 GMT
• Title: SignAgent: Agentic LLMs for Linguistically-Grounded Sign Language Annotation and Dataset Curation
• Abstract summary: SignAgent is a novel agentic framework for scalable, linguistically-grounded Sign Language (SL) annotation and dataset curation.<n>Our results demonstrate that our agentic approach achieves strong performance for large-scale, linguistically-aware data annotation and curation.
• Score: 20.300369465395786
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This paper introduces SignAgent, a novel agentic framework that utilises Large Language Models (LLMs) for scalable, linguistically-grounded Sign Language (SL) annotation and dataset curation. Traditional computational methods for SLs often operate at the gloss level, overlooking crucial linguistic nuances, while manual linguistic annotation remains a significant bottleneck, proving too slow and expensive for the creation of large-scale, phonologically-aware datasets. SignAgent addresses these challenges through SignAgent Orchestrator, a reasoning LLM that coordinates a suite of linguistic tools, and SignGraph, a knowledge-grounded LLM that provides lexical and linguistic grounding. We evaluate our framework on two downstream annotation tasks. First, on Pseudo-gloss Annotation, where the agent performs constrained assignment, using multi-modal evidence to extract and order suitable gloss labels for signed sequences. Second, on ID Glossing, where the agent detects and refines visual clusters by reasoning over both visual similarity and phonological overlap to correctly identify and group lexical sign variants. Our results demonstrate that our agentic approach achieves strong performance for large-scale, linguistically-aware data annotation and curation.

Related papers

• SIGNET: Motion-Level Knowledge Transfer for Cross-Language Sign Language Translation [20.105725837533722]
  We present textbf SIGNET, a framework that enables motion-level knowledge for cross-language sign language translation.<n>Our key is that, although sign languages differ in grammar and lexicon, pretrained models capture motion-level visual patterns that can be reused across datasets and languages.
  arXiv  Detail & Related papers  (2026-06-26T21:58:16Z)
• AgenticTagger: Structured Item Representation for Recommendation with LLM Agents [58.12004213978182]
  AgenticTagger is a framework that queries LLMs for representing items with sequences of text descriptors.<n>To effectively and efficiently ground vocabulary in the item corpus of interest, we design a multi-agent reflection mechanism.<n>Experiments on public and private data show AgenticTagger brings consistent improvements across diverse recommendation scenarios.
  arXiv  Detail & Related papers  (2026-02-05T18:01:37Z)
• LinguistAgent: A Reflective Multi-Model Platform for Automated Linguistic Annotation [0.0]
  This paper introduces LinguistAgent, an integrated platform that leverages a reflective multi-model architecture to automate linguistic annotation.<n>The system implements a dual-agent workflow, comprising an Annotator and a Reviewer, to simulate a professional peer-review process.<n>We demonstrate LinguistAgent's efficacy using the task of metaphor identification as an example, providing real-time token-level evaluation.
  arXiv  Detail & Related papers  (2026-02-05T09:55:19Z)
• Bridging the Knowledge Void: Inference-time Acquisition of Unfamiliar Programming Languages for Coding Tasks [22.908904483320953]
  Large Language Models (LLMs) in coding tasks are often a reflection of their extensive pre-training corpora.<n>We propose ILA-agent, a general ILA framework that equips LLMs with a set of behavioral primitives.<n>We instantiate ILA-agent for Cangjie and evaluate its performance across code generation, translation, and program repair tasks.
  arXiv  Detail & Related papers  (2026-01-16T09:06:47Z)
• Lost in Translation, Found in Embeddings: Sign Language Translation and Alignment [84.39962912136525]
  We develop a model for sign language understanding that performs sign language translation (SLT) and sign-subtitle alignment (SSA)<n>Our approach is built upon three components: (i) a lightweight visual backbone that captures manual and non-manual cues from human keypoints and lip-region images; (ii) a Sliding Perceiver mapping network that aggregates consecutive visual features into word-level embeddings; and (iii) a multi-task scalable training strategy that jointly optimises SLT and SSA.
  arXiv  Detail & Related papers  (2025-12-08T21:05:46Z)
• Sign Spotting Disambiguation using Large Language Models [29.79050316749927]
  We introduce a training-free framework that integrates Large Language Models (LLMs) to significantly enhance sign spotting quality.<n>Our approach extracts global-temporal and hand shape features, which are then matched against a large-scale sign dictionary.<n>This dictionary-based matching inherently offers superior vocabulary flexibility without requiring model retraining.
  arXiv  Detail & Related papers  (2025-07-04T16:38:09Z)
• Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation [48.20483623444857]
  Sign Language Translation aims to map sign language videos to spoken language text.<n>A common approach relies on gloss annotations as an intermediate representation.<n>We propose a gloss-free pseudo gloss generation framework that eliminates the need for human-annotated glosses.
  arXiv  Detail & Related papers  (2025-05-21T12:19:55Z)
• A Tale of Two Languages: Large-Vocabulary Continuous Sign Language Recognition from Spoken Language Supervision [74.972172804514]
  We introduce a multi-task Transformer model, CSLR2, that is able to ingest a signing sequence and output in a joint embedding space between signed language and spoken language text. New dataset annotations provide continuous sign-level annotations for six hours of test videos, and will be made publicly available. Our model significantly outperforms the previous state of the art on both tasks.
  arXiv  Detail & Related papers  (2024-05-16T17:19:06Z)
• Gloss-free Sign Language Translation: Improving from Visual-Language Pretraining [56.26550923909137]
  Gloss-Free Sign Language Translation (SLT) is a challenging task due to its cross-domain nature. We propose a novel Gloss-Free SLT based on Visual-Language Pretraining (GFSLT-) Our approach involves two stages: (i) integrating Contrastive Language-Image Pre-training with masked self-supervised learning to create pre-tasks that bridge the semantic gap between visual and textual representations and restore masked sentences, and (ii) constructing an end-to-end architecture with an encoder-decoder-like structure that inherits the parameters of the pre-trained Visual and Text Decoder from
  arXiv  Detail & Related papers  (2023-07-27T10:59:18Z)
• Prompting Language Models for Linguistic Structure [73.11488464916668]
  We present a structured prompting approach for linguistic structured prediction tasks. We evaluate this approach on part-of-speech tagging, named entity recognition, and sentence chunking. We find that while PLMs contain significant prior knowledge of task labels due to task leakage into the pretraining corpus, structured prompting can also retrieve linguistic structure with arbitrary labels.
  arXiv  Detail & Related papers  (2022-11-15T01:13:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.