Video-Text Temporal Localization via Multi-Scale Convolution and Dynamic Routing

• URL: http://arxiv.org/abs/2607.05093v2
• Date: Tue, 07 Jul 2026 12:28:04 GMT
• Title: Video-Text Temporal Localization via Multi-Scale Convolution and Dynamic Routing
• Abstract summary: Video-text temporal localization requires precise alignment between natural language queries and corresponding video segments.<n>We present a novel framework that addresses two critical limitations of existing methods.<n>Our results validate that combining hierarchical temporal modeling with structured semantic routing provides an effective solution for fine-grained video-language understanding.
• Score: 7.704586969939547
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video-text temporal localization requires precise alignment between natural language queries and corresponding video segments, a fundamental challenge in multimodal understanding. We present a novel framework that addresses two critical limitations of existing methods: inadequate modeling of hierarchical temporal structure and inability to handle complex many-to-many correspondences between modalities. Our approach introduces a multi-scale temporal convolutional encoder that captures motion patterns across different temporal granularities - from instantaneous frame transitions to extended action sequences. We further propose a capsule-based dynamic routing mechanism that iteratively refines segment-query associations through structured agreement updates, enabling flexible modeling of non-monotonic alignments. These components are unified through a multi-task learning objective that jointly optimizes temporal boundary regression, cross-modal semantic alignment, and capsule diversity. Extensive experiments on ActivityNet Captions demonstrate significant improvements, achieving 42.9% Recall@0.5 and 41.1% mean IoU, surpassing strong transformer-based baselines while maintaining computational efficiency. Our results validate that combining hierarchical temporal modeling with structured semantic routing provides an effective solution for fine-grained video-language understanding.

Related papers

• Turing Patterns for Multimedia: Reaction-Diffusion Multi-Modal Fusion for Language-Guided Video Moment Retrieval [59.55233913340214]
  Video-language models often struggle to capture dynamic, non-linear interactions between temporal video sequences and textual semantics.<n>We propose textbfReaction-Diffusion Multimodal Fusion (RDMF), a novel framework that reimagines video-language alignment as a reaction-diffusion process.<n>Preliminary experiments demonstrate its potential to outperform existing methods in identifying salient video moments.
  arXiv  Detail & Related papers  (2026-06-01T03:02:35Z)
• Beyond Boundary Frames: Audio-Visual Semantic Guidance for Context-Aware Video Interpolation [14.00347197658315]
  BBF is a context-aware video frame framework guided by audio/visual semantics.<n>We show that BBF outperforms specialized state-of-the-art methods on both generic and audio-visual synchronized tasks.
  arXiv  Detail & Related papers  (2025-12-03T09:22:13Z)
• Complementary and Contrastive Learning for Audio-Visual Segmentation [74.11434759171199]
  We present Complementary and Contrastive Transformer (CCFormer), a novel framework adept at processing both local and global information.<n>Our method sets new state-of-the-art benchmarks across the S4, MS3 and AVSS datasets.
  arXiv  Detail & Related papers  (2025-10-11T06:36:59Z)
• MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editing [53.98607267063729]
  MotionVerse is a framework to comprehend, generate, and edit human motion in both single-person and multi-person scenarios.<n>We employ a motion tokenizer with residual quantization, which converts continuous motion sequences into multi-stream discrete tokens.<n>We also introduce a textitDelay Parallel Modeling strategy, which temporally staggers the encoding of residual token streams.
  arXiv  Detail & Related papers  (2025-09-28T04:20:56Z)
• DAMS:Dual-Branch Adaptive Multiscale Spatiotemporal Framework for Video Anomaly Detection [7.117824587276951]
  This study offers a dual-path architecture called the Dual-Branch Adaptive Multiscale Stemporal Framework (DAMS), which is based on multilevel feature and decoupling fusion.<n>The main processing path integrates the Adaptive Multiscale Time Pyramid Network (AMTPN) with the Convolutional Block Attention Mechanism (CBAM)
  arXiv  Detail & Related papers  (2025-07-28T08:42:00Z)
• OptiCorNet: Optimizing Sequence-Based Context Correlation for Visual Place Recognition [2.3093110834423616]
  This paper presents OptiCorNet, a novel sequence modeling framework.<n>It unifies spatial feature extraction and temporal differencing into a differentiable, end-to-end trainable module.<n>Our approach outperforms state-of-the-art baselines under challenging seasonal and viewpoint variations.
  arXiv  Detail & Related papers  (2025-07-19T04:29:43Z)
• LoViC: Efficient Long Video Generation with Context Compression [68.22069741704158]
  We introduce LoViC, a DiT-based framework trained on million-scale open-domain videos.<n>At the core of our approach is FlexFormer, an expressive autoencoder that jointly compresses video and text into unified latent representations.
  arXiv  Detail & Related papers  (2025-07-17T09:46:43Z)
• Vision Transformer Based Semantic Communications for Next Generation Wireless Networks [3.8095664680229935]
  This paper presents a Vision Transformer (ViT)-based semantic communication framework.<n>By equipping ViT as the encoder-decoder framework, the proposed architecture can proficiently encode images into a high semantic content.<n>The architecture based on the proposed ViT network achieves the Peak Signal-versato-noise Ratio (PSNR) of 38 dB.
  arXiv  Detail & Related papers  (2025-03-21T16:23:02Z)
• Take What You Need: Flexible Multi-Task Semantic Communications with Channel Adaptation [51.53221300103261]
  This article introduces a novel channel-adaptive and multi-task-aware semantic communication framework based on a masked auto-encoder architecture.<n>A channel-aware extractor is employed to dynamically select relevant information in response to real-time channel conditions.<n> Experimental results demonstrate the superior performance of our framework compared to conventional methods in tasks such as image reconstruction and object detection.
  arXiv  Detail & Related papers  (2025-02-12T09:01:25Z)
• Adaptive Semantic Token Selection for AI-native Goal-oriented Communications [11.92172357956248]
  We propose a novel design for AI-native goal-oriented communications. We exploit transformer neural networks under dynamic inference constraints on bandwidth and computation. We show that our model improves over state-of-the-art token selection mechanisms.
  arXiv  Detail & Related papers  (2024-04-25T13:49:50Z)
• Agent-driven Generative Semantic Communication with Cross-Modality and Prediction [57.335922373309074]
  We propose a novel agent-driven generative semantic communication framework based on reinforcement learning. In this work, we develop an agent-assisted semantic encoder with cross-modality capability, which can track the semantic changes, channel condition, to perform adaptive semantic extraction and sampling. The effectiveness of the designed models has been verified using the UA-DETRAC dataset, demonstrating the performance gains of the overall A-GSC framework.
  arXiv  Detail & Related papers  (2024-04-10T13:24:27Z)
• Communication-Efficient Framework for Distributed Image Semantic Wireless Transmission [68.69108124451263]
  Federated learning-based semantic communication (FLSC) framework for multi-task distributed image transmission with IoT devices. Each link is composed of a hierarchical vision transformer (HVT)-based extractor and a task-adaptive translator. Channel state information-based multiple-input multiple-output transmission module designed to combat channel fading and noise.
  arXiv  Detail & Related papers  (2023-08-07T16:32:14Z)
• Deeply Interleaved Two-Stream Encoder for Referring Video Segmentation [87.49579477873196]
  We first design a two-stream encoder to extract CNN-based visual features and transformer-based linguistic features hierarchically. A vision-language mutual guidance (VLMG) module is inserted into the encoder multiple times to promote the hierarchical and progressive fusion of multi-modal features. In order to promote the temporal alignment between frames, we propose a language-guided multi-scale dynamic filtering (LMDF) module.
  arXiv  Detail & Related papers  (2022-03-30T01:06:13Z)
• Video Frame Interpolation Transformer [86.20646863821908]
  We propose a Transformer-based video framework that allows content-aware aggregation weights and considers long-range dependencies with the self-attention operations. To avoid the high computational cost of global self-attention, we introduce the concept of local attention into video. In addition, we develop a multi-scale frame scheme to fully realize the potential of Transformers.
  arXiv  Detail & Related papers  (2021-11-27T05:35:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.