Length Matters: Length-Aware Transformer for Temporal Sentence Grounding

• URL: http://arxiv.org/abs/2508.04299v1
• Date: Wed, 06 Aug 2025 10:34:58 GMT
• Title: Length Matters: Length-Aware Transformer for Temporal Sentence Grounding
• Authors: Yifan Wang, Ziyi Liu, Xiaolong Sun, Jiawei Wang, Hongmin Liu,
• Abstract summary: Temporal sentence grounding is a challenging task aiming to localize the temporal segment within an untrimmed video.<n>We introduce the Length-Aware Transformer (LATR) for TSG, which assigns different queries to handle predictions based on varying temporal lengths.<n>Experiments demonstrate the effectiveness of our LATR, achieving state-of-the-art performance on three public benchmarks.
• Score: 19.652239319193413
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Temporal sentence grounding (TSG) is a highly challenging task aiming to localize the temporal segment within an untrimmed video corresponding to a given natural language description. Benefiting from the design of learnable queries, the DETR-based models have achieved substantial advancements in the TSG task. However, the absence of explicit supervision often causes the learned queries to overlap in roles, leading to redundant predictions. Therefore, we propose to improve TSG by making each query fulfill its designated role, leveraging the length priors of the video-description pairs. In this paper, we introduce the Length-Aware Transformer (LATR) for TSG, which assigns different queries to handle predictions based on varying temporal lengths. Specifically, we divide all queries into three groups, responsible for segments with short, middle, and long temporal durations, respectively. During training, an additional length classification task is introduced. Predictions from queries with mismatched lengths are suppressed, guiding each query to specialize in its designated function. Extensive experiments demonstrate the effectiveness of our LATR, achieving state-of-the-art performance on three public benchmarks. Furthermore, the ablation studies validate the contribution of each component of our method and the critical role of incorporating length priors into the TSG task.

Related papers

• Respecting Temporal-Causal Consistency: Entity-Event Knowledge Graphs for Retrieval-Augmented Generation [69.45495166424642]
  We develop a robust and discriminative QA benchmark to measure temporal, causal, and character consistency understanding in narrative documents.<n>We then introduce Entity-Event RAG (E2RAG), a dual-graph framework that keeps separate entity and event subgraphs linked by a bipartite mapping.<n>Across ChronoQA, our approach outperforms state-of-the-art unstructured and KG-based RAG baselines, with notable gains on causal and character consistency queries.
  arXiv  Detail & Related papers  (2025-06-06T10:07:21Z)
• On the Reproducibility of Learned Sparse Retrieval Adaptations for Long Documents [2.186901738997927]
  We reproduce and examine the mechanisms of adapting Learned Sparse Retrieval (LSR) for long documents.<n>Our experiments confirmed the importance of specific segments, with the first segment consistently dominating document retrieval performance.<n>We re-evaluated recently proposed methods -- ExactSDM and SoftSDM -- across varying document lengths.
  arXiv  Detail & Related papers  (2025-03-31T08:19:31Z)
• Emulating Retrieval Augmented Generation via Prompt Engineering for Enhanced Long Context Comprehension in LLMs [23.960451986662996]
  This paper proposes a method that emulates Retrieval Augmented Generation (RAG) through specialized prompt engineering and chain-of-thought reasoning.<n>We evaluate our approach on selected tasks from BABILong, which interleaves standard bAbI QA problems with large amounts of distractor text.
  arXiv  Detail & Related papers  (2025-02-18T02:49:40Z)
• Diversifying Query: Region-Guided Transformer for Temporal Sentence Grounding [30.33362992577831]
  Temporal sentence grounding is a challenging task that aims to localize the moment spans relevant to a language description.<n>Recent DETR-based models have achieved notable progress by leveraging multiple learnable moment queries.<n>We present a Region-Guided TRansformer (RGTR) for temporal sentence grounding.
  arXiv  Detail & Related papers  (2024-05-31T19:13:09Z)
• TE-TAD: Towards Full End-to-End Temporal Action Detection via Time-Aligned Coordinate Expression [25.180317527112372]
  normalized coordinate expression is a key factor as reliance on hand-crafted components in query-based detectors for temporal action detection (TAD) We propose modelname, a full end-to-end temporal action detection transformer that integrates time-aligned coordinate expression. Our approach not only simplifies the TAD process by eliminating the need for hand-crafted components but also significantly improves the performance of query-based detectors.
  arXiv  Detail & Related papers  (2024-04-03T02:16:30Z)
• LongHeads: Multi-Head Attention is Secretly a Long Context Processor [49.1661870007655]
  LongHeads is a training-free framework that enhances large language models' long context ability. Instead of allowing each head to attend to the full sentence, we allow each head to process in-distribution length by selecting and attending to context chunks. LongHeads achieves 100% accuracy at the 128k length on passkey retrieval task.
  arXiv  Detail & Related papers  (2024-02-16T13:39:34Z)
• Simultaneous Task Allocation and Planning for Multi-Robots under Hierarchical Temporal Logic Specifications [8.471147498059235]
  We introduce a hierarchical structure for a widely used specification type -- on finite traces (LTL$_f$)<n>The resulting language, termed HLTL$_f$, is defined with both its syntax and semantics.<n>We also develop a search-based approach to synthesize plans for multi-robot systems.
  arXiv  Detail & Related papers  (2024-01-08T16:35:13Z)
• Tracking Objects and Activities with Attention for Temporal Sentence Grounding [51.416914256782505]
  Temporal sentence (TSG) aims to localize the temporal segment which is semantically aligned with a natural language query in an untrimmed segment. We propose a novel Temporal Sentence Tracking Network (TSTNet), which contains (A) a Cross-modal Targets Generator to generate multi-modal and search space, and (B) a Temporal Sentence Tracker to track multi-modal targets' behavior and to predict query-related segment.
  arXiv  Detail & Related papers  (2023-02-21T16:42:52Z)
• Efficient Long Sequence Encoding via Synchronization [29.075962393432857]
  We propose a synchronization mechanism for hierarchical encoding. Our approach first identifies anchor tokens across segments and groups them by their roles in the original input sequence. Our approach is able to improve the global information exchange among segments while maintaining efficiency.
  arXiv  Detail & Related papers  (2022-03-15T04:37:02Z)
• End-to-end Temporal Action Detection with Transformer [86.80289146697788]
  Temporal action detection (TAD) aims to determine the semantic label and the boundaries of every action instance in an untrimmed video. Here, we construct an end-to-end framework for TAD upon Transformer, termed textitTadTR. Our method achieves state-of-the-art performance on HACS Segments and THUMOS14 and competitive performance on ActivityNet-1.3.
  arXiv  Detail & Related papers  (2021-06-18T17:58:34Z)
• Interpretable Time-series Representation Learning With Multi-Level Disentanglement [56.38489708031278]
  Disentangle Time Series (DTS) is a novel disentanglement enhancement framework for sequential data. DTS generates hierarchical semantic concepts as the interpretable and disentangled representation of time-series. DTS achieves superior performance in downstream applications, with high interpretability of semantic concepts.
  arXiv  Detail & Related papers  (2021-05-17T22:02:24Z)
• Temporal Query Networks for Fine-grained Video Understanding [88.9877174286279]
  We cast this into a query-response mechanism, where each query addresses a particular question, and has its own response label set. We evaluate the method extensively on the FineGym and Diving48 benchmarks for fine-grained action classification and surpass the state-of-the-art using only RGB features.
  arXiv  Detail & Related papers  (2021-04-19T17:58:48Z)
• Query Resolution for Conversational Search with Limited Supervision [63.131221660019776]
  We propose QuReTeC (Query Resolution by Term Classification), a neural query resolution model based on bidirectional transformers. We show that QuReTeC outperforms state-of-the-art models, and furthermore, that our distant supervision method can be used to substantially reduce the amount of human-curated data required to train QuReTeC.
  arXiv  Detail & Related papers  (2020-05-24T11:37:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.