SignRAG: A Retrieval-Augmented System for Scalable Zero-Shot Road Sign Recognition

• URL: http://arxiv.org/abs/2512.12885v1
• Date: Sun, 14 Dec 2025 23:56:34 GMT
• Title: SignRAG: A Retrieval-Augmented System for Scalable Zero-Shot Road Sign Recognition
• Abstract summary: This paper introduces a novel zero-shot recognition framework that adapts the Retrieval-Augmented Generation paradigm to address this challenge.<n>We validate this approach on a comprehensive set of 303 regulatory signs from the Ohio MUTCD.<n> Experimental results demonstrate the framework's effectiveness, achieving 95.58% accuracy on ideal reference images and 82.45% on challenging real-world road data.
• Score: 3.9692145330341764
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated road sign recognition is a critical task for intelligent transportation systems, but traditional deep learning methods struggle with the sheer number of sign classes and the impracticality of creating exhaustive labeled datasets. This paper introduces a novel zero-shot recognition framework that adapts the Retrieval-Augmented Generation (RAG) paradigm to address this challenge. Our method first uses a Vision Language Model (VLM) to generate a textual description of a sign from an input image. This description is used to retrieve a small set of the most relevant sign candidates from a vector database of reference designs. Subsequently, a Large Language Model (LLM) reasons over the retrieved candidates to make a final, fine-grained recognition. We validate this approach on a comprehensive set of 303 regulatory signs from the Ohio MUTCD. Experimental results demonstrate the framework's effectiveness, achieving 95.58% accuracy on ideal reference images and 82.45% on challenging real-world road data. This work demonstrates the viability of RAG-based architectures for creating scalable and accurate systems for road sign recognition without task-specific training.

Related papers

• Achieving Text-based Person Retrieval with Any Granularity [80.69427598925718]
  This paper introduces a new paradigm, Text-based Person Retrieval with Any Granularity.<n>We formalize a five-level granularity spectrum and construct UFine6926-MG, a high-quality multi-grained annotated dataset.<n>We then propose the Cross-modal Multi-grained Aligning and Matching framework.
  arXiv  Detail & Related papers  (2026-07-23T08:42:25Z)
• A Robust Deep Learning Framework for Bangla License Plate Recognition Using YOLO and Vision-Language OCR [0.0]
  The detection of Bangla license plates remains challenging because of the complicated character scheme and uneven layouts.<n>This paper presents a robust Bangla License Plate Recognition system that integrates a deep learning-based object detection model for license plate localization with Optical Character Recognition for text extraction.
  arXiv  Detail & Related papers  (2026-03-10T22:55:59Z)
• Pose-Based Sign Language Spotting via an End-to-End Encoder Architecture [0.4083182125683813]
  We present a first step toward sign language retrieval by addressing the challenge of detecting the presence or absence of a query sign video.<n>Unlike conventional approaches that rely on intermediate gloss recognition or text-based matching, we propose an end-to-end model that directly operates on pose keypoints extracted from sign videos.<n>Our architecture employs an encoder-only backbone with a binary classification head to determine whether the query sign appears within the target sequence.
  arXiv  Detail & Related papers  (2025-12-09T15:49:23Z)
• Cross-domain Few-shot In-context Learning for Enhancing Traffic Sign Recognition [49.20086587208214]
  We propose a cross-domain few-shot in-context learning method based on the MLLM for enhancing traffic sign recognition. By using description texts, our method reduces the cross-domain differences between template and real traffic signs. Our approach requires only simple and uniform textual indications, without the need for large-scale traffic sign images and labels.
  arXiv  Detail & Related papers  (2024-07-08T10:51:03Z)
• Instruction-Guided Scene Text Recognition [51.853730414264625]
  We propose a novel instruction-guided scene text recognition (IGTR) paradigm that formulates STR as an instruction learning problem.<n>IGTR first devises $left langle condition,question,answerright rangle$ instruction triplets, providing rich and diverse descriptions of character attributes.<n>To effectively learn these attributes through question-answering, IGTR develops a lightweight instruction encoder, a cross-modal feature fusion module and a multi-task answer head.
  arXiv  Detail & Related papers  (2024-01-31T14:13:01Z)
• Sign Languague Recognition without frame-sequencing constraints: A proof of concept on the Argentinian Sign Language [42.27617228521691]
  This paper presents a general probabilistic model for sign classification that combines sub-classifiers based on different types of features. The proposed model achieved an accuracy rate of 97% on an Argentinian Sign Language dataset.
  arXiv  Detail & Related papers  (2023-10-26T14:47:11Z)
• Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition [57.08108545219043]
  Recognizing novel sub-categories with scarce samples is an essential and challenging research topic in computer vision. Existing literature addresses this challenge by employing local-based representation approaches. This article proposes a novel model, Robust Saliency-aware Distillation (RSaD), for few-shot fine-grained visual recognition.
  arXiv  Detail & Related papers  (2023-05-12T00:13:17Z)
• SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding [132.78015553111234]
  Hand gesture serves as a crucial role during the expression of sign language. Current deep learning based methods for sign language understanding (SLU) are prone to over-fitting due to insufficient sign data resource. We propose the first self-supervised pre-trainable SignBERT+ framework with model-aware hand prior incorporated.
  arXiv  Detail & Related papers  (2023-05-08T17:16:38Z)
• Gait Recognition in the Wild: A Large-scale Benchmark and NAS-based Baseline [95.88825497452716]
  Gait benchmarks empower the research community to train and evaluate high-performance gait recognition systems. GREW is the first large-scale dataset for gait recognition in the wild. SPOSGait is the first NAS-based gait recognition model.
  arXiv  Detail & Related papers  (2022-05-05T14:57:39Z)
• Towards Zero-shot Sign Language Recognition [11.952300437658703]
  This paper tackles the problem of zero-shot sign language recognition. The goal is to leverage models learned over the seen sign classes to recognize the instances of unseen sign classes.
  arXiv  Detail & Related papers  (2022-01-15T19:26:36Z)
• SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition [94.30084702921529]
  Hand gesture serves as a critical role in sign language. Current deep-learning-based sign language recognition methods may suffer insufficient interpretability. We introduce the first self-supervised pre-trainable SignBERT with incorporated hand prior for SLR.
  arXiv  Detail & Related papers  (2021-10-11T16:18:09Z)
• Accelerating Road Sign Ground Truth Construction with Knowledge Graph and Machine Learning [5.226306460380354]
  We propose a novel approach using knowledge graphs and a machine learning algorithm to assist human annotators in classifying road signs effectively. Annotators can query the Road Sign Knowledge Graph using visual attributes and receive closest matching candidates suggested by the VPE model. We show that our knowledge graph approach can reduce sign search space by 98.9%.
  arXiv  Detail & Related papers  (2020-12-04T15:42:08Z)
• Automatic Counting and Identification of Train Wagons Based on Computer Vision and Deep Learning [70.84106972725917]
  The proposed solution is cost-effective and can easily replace solutions based on radiofrequency identification (RFID) The system is able to automatically reject some of the train wagons successfully counted, as they have damaged identification codes.
  arXiv  Detail & Related papers  (2020-10-30T14:56:54Z)
• Improving Road Signs Detection performance by Combining the Features of Hough Transform and Texture [5.620334754517149]
  Detection of road signs present in the scene is the one of the main stages of the traffic sign detection and recognition. In this paper, an efficient solution to enhance road signs detection, including Arabic context, has been made. The Hough Transform (RHT) is used to detect the circular and octagonal shapes.
  arXiv  Detail & Related papers  (2020-10-13T15:09:29Z)
• BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues [106.21067543021887]
  We show how to use mouthing cues from signers to obtain high-quality annotations from video data. The BSL-1K dataset is a collection of British Sign Language (BSL) signs of unprecedented scale.
  arXiv  Detail & Related papers  (2020-07-23T16:59:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.