TraRA: Trajectory-level Recognition Aggregation for Video Text Spotting in Urban Surveillance

• URL: http://arxiv.org/abs/2606.07161v1
• Date: Fri, 05 Jun 2026 11:23:16 GMT
• Title: TraRA: Trajectory-level Recognition Aggregation for Video Text Spotting in Urban Surveillance
• Abstract summary: Video Text Spotting (VTS) is essential for urban surveillance and intelligent transportation systems.<n>We propose TraRA, a plug-and-play method that performs trajectory-level text recognition by leveraging temporal and multimodal consistency.<n>TraRA consistently improves tracking and recognition performance over state-of-the-art VTS methods.
• Score: 9.382380971789575
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Video Text Spotting (VTS) is essential for urban surveillance and intelligent transportation systems, enabling automated reading of street signs, vehicle markings, and scene text in video streams. However, reliable recognition remains challenging due to dynamic video factors common in surveillance scenarios, including motion blur, occlusion, and scale variation, which degrade frame-level recognition. Existing VTS methods typically perform recognition independently on each frame, leading to inconsistent and inaccurate results across sequences. To address these limitations, we propose TraRA (Trajectory-level Recognition Aggregation for VTS), a plug-and-play method that performs trajectory-level text recognition by leveraging temporal and multimodal consistency. TraRA integrates two key modules: (1) the Temporal Clustering and (2) the Vision-Language Aggregation. The former refines noisy trajectories by grouping temporally and visually coherent text instances, while the latter employs a Low-Rank Adaptation-enhanced Vision-Language model to fuse visual cues with linguistic context across frames. By aggregating information over entire text trajectories, TraRA achieves robust text recognition even under challenging surveillance conditions. Extensive experiments on four public benchmarks, including road and urban scene datasets (RoadText, BOVText, ArTVideo, and ICDAR15), demonstrate that TraRA consistently improves tracking and recognition performance over state-of-the-art VTS methods. The source code is available at https://github.com/trid2912/TraRA.

Related papers

• Hierarchical Semantic-Constrained Heterogeneous Graph for Audio-Visual Event Localization [69.67186845167568]
  Open-vocabulary audio-visual event localization (OV-AVEL) jointly models audio-visual cues to recognize and temporally localize events.<n>Existing methods primarily learn joint audio-visual representations in Euclidean space.<n>We propose a hierarchical semantic constrained heterogeneous graph (HSCHG) for audio-visual event localization framework.
  arXiv  Detail & Related papers  (2026-06-05T08:23:58Z)
• Empower Words: DualGround for Structured Phrase and Sentence-Level Temporal Grounding [30.223279362023337]
  Video Grounding (VTG) aims to localize temporal segments in long, untrimmed videos that align with a given natural language query.<n>Existing approaches commonly treat all text tokens uniformly during crossmodal attention, disregarding their distinct semantic roles.<n>We propose DualGround, a dual-branch architecture that explicitly separates global and local semantics.
  arXiv  Detail & Related papers  (2025-10-23T05:53:01Z)
• TGT: Text-Grounded Trajectories for Locally Controlled Video Generation [33.989722489622075]
  We introduce Text-Grounded Trajectories (TGT), a framework that conditions video generation on trajectories paired with localized text descriptions.<n>TGT achieves higher visual quality, more accurate text alignment, and improved motion controllability compared with prior approaches.
  arXiv  Detail & Related papers  (2025-10-16T19:45:27Z)
• Taming Text-to-Sounding Video Generation via Advanced Modality Condition and Interaction [28.20791917022439]
  This study focuses on a challenging yet promising task, Text-to-Sounding-Video (T2SV) generation.<n>It aims to generate a video with synchronized audio from text conditions, ensuring both modalities are aligned with text.<n>Two critical challenges still remain unaddressed: (1) a single, shared text caption where the text for video is equal to the text for audio often creates modal interference, and (2) the optimal mechanism for cross-modal feature interaction remains unclear.
  arXiv  Detail & Related papers  (2025-10-03T15:43:56Z)
• GAID: Frame-Level Gated Audio-Visual Integration with Directional Perturbation for Text-Video Retrieval [12.483734449829235]
  GAID is a framework that integrates audio and visual features under textual guidance.<n>DASP injects structure-aware perturbations into text embeddings, enhancing robustness and discrimination without incurring multi-pass inference.<n>Experiments on MSR-VTT, DiDeMo, LSMDC, and VATEX show consistent state-of-the-art results with notable efficiency gains.
  arXiv  Detail & Related papers  (2025-08-03T10:44:24Z)
• Video-Level Language-Driven Video-Based Visible-Infrared Person Re-Identification [47.40091830500585]
  Video-based Visible-basedInfrared Person Re-Identification (VVIReID) aims to match pedestrian sequences across modalities by extracting modality-in sequence-level features.<n>A framework, video-level language-driven VVI-ReID (VLD), consists of two core modules: inmodality language (IMLP) and spatialtemporal aggregation.
  arXiv  Detail & Related papers  (2025-06-03T04:49:08Z)
• SignVTCL: Multi-Modal Continuous Sign Language Recognition Enhanced by Visual-Textual Contrastive Learning [51.800031281177105]
  SignVTCL is a continuous sign language recognition framework enhanced by visual-textual contrastive learning. It integrates multi-modal data (video, keypoints, and optical flow) simultaneously to train a unified visual backbone. It achieves state-of-the-art results compared with previous methods.
  arXiv  Detail & Related papers  (2024-01-22T11:04:55Z)
• Towards Generalisable Video Moment Retrieval: Visual-Dynamic Injection to Image-Text Pre-Training [70.83385449872495]
  The correlation between the vision and text is essential for video moment retrieval (VMR) Existing methods rely on separate pre-training feature extractors for visual and textual understanding. We propose a generic method, referred to as Visual-Dynamic Injection (VDI), to empower the model's understanding of video moments.
  arXiv  Detail & Related papers  (2023-02-28T19:29:05Z)
• TVLT: Textless Vision-Language Transformer [89.31422264408002]
  We present the Textless Vision-Language Transformer (TVLT), where homogeneous transformer blocks take raw visual and audio inputs. TVLT attains performance comparable to its text-based counterpart, on various multimodal tasks. Our findings suggest the possibility of learning compact and efficient visual-linguistic representations from low-level visual and audio signals.
  arXiv  Detail & Related papers  (2022-09-28T15:08:03Z)
• End-to-End Video Text Spotting with Transformer [86.46724646835627]
  We propose a simple, but effective end-to-end video text DEtection, Tracking, and Recognition framework (TransDETR) TransDETR is the first end-to-end trainable video text spotting framework, which simultaneously addresses the three sub-tasks (e.g., text detection, tracking, recognition)
  arXiv  Detail & Related papers  (2022-03-20T12:14:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.