TBSG-Net: Temporal Bipartite Scene Graph Network for Fine-Grained Video Moment Retrieval

• URL: http://arxiv.org/abs/2608.02056v1
• Date: Mon, 03 Aug 2026 10:52:32 GMT
• Title: TBSG-Net: Temporal Bipartite Scene Graph Network for Fine-Grained Video Moment Retrieval
• Abstract summary: Video Moment Retrieval (VMR) has highlighted the effectiveness of Static Scene Graphs (SSGs)<n> integrating SSGs into VM remains constrained by two inherent limitations.<n>We propose Explicit Bipartite Scene Graph Network (TBSG-Net)
• Score: 6.81967623770995
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in proposal-free Video Moment Retrieval (VMR) have highlighted the effectiveness of Static Scene Graphs (SSGs). By modeling objects and their relations at the frame level, SSGs enrich retrieval-oriented video representations. However, integrating SSGs into VMR remains constrained by two inherent limitations: (1) Lack of Temporal Dynamics. SSGs fail to model how objects and their relationships evolve over time, leading to the loss of essential temporal dependencies in video representation; and (2) Lack of Explicit Temporal Span Encoding. SSGs do not explicitly encode the duration of relationships, making precise localization challenging. To address these limitations, we propose Temporal Bipartite Scene Graph Network (TBSG-Net)---to the best of our knowledge, the first Dynamic Scene Graph (DSG) based proposal-free VMR model. Specifically, TBSG-Net leverages DSGs to extract event-centric graph representations of the input video, enabling the modeling of object interactions over time and thus addressing limitation (1). These DSGs are then processed by a novel Dynamic Scene Graph Embedding (DSG-E) module to capture both Temporal Span and spatio-temporal information. First, DSG-E utilizes a TBSG Constructor to transform DSGs into TBSGs, explicitly encoding objects, relationships, and time spans to tackle limitation (2). Second, the resultant TBSGs are passed into a hybrid TBSG Encoder that integrates a Transformer variant for global event modeling and a Graph Convolutional Network for detailed relational reasoning, ultimately producing a more comprehensive spatio-temporal representation. Our experiments demonstrate substantial improvements of TBSG-Net over all baselines.

Related papers

• Static and Dynamic Graph Alignment Network for Temporal Video Grounding [17.14274381541407]
  Temporal Video Grounding aims to localize temporal moments in an untrimmed video that semantically correspond to given natural language queries.<n>Graph Convolutional Networks (GCN) have been widely adopted in TVG to model temporal relations among video clips.
  arXiv  Detail & Related papers  (2026-05-01T14:16:42Z)
• RS-Net: Context-Aware Relation Scoring for Dynamic Scene Graph Generation [1.7188280334580195]
  Dynamic Scene Graph Generation (DSGG) models how object relations evolve over time in videos.<n>Existing methods are trained only on annotated object pairs and lack guidance for non-related pairs, making it difficult to identify meaningful relations during inference.<n>We propose Relation Scoring Network (RS-Net), a modular framework that scores the contextual importance of object pairs using both spatial interactions and long-range temporal context.
  arXiv  Detail & Related papers  (2025-11-11T05:37:21Z)
• DIFFVSGG: Diffusion-Driven Online Video Scene Graph Generation [61.59996525424585]
  DIFFVSGG is an online VSGG solution that frames this task as an iterative scene graph update problem.<n>We unify the decoding of object classification, bounding box regression, and graph generation three tasks using one shared feature embedding.<n>DIFFVSGG further facilitates continuous temporal reasoning, where predictions for subsequent frames leverage results of past frames as the conditional inputs of LDMs.
  arXiv  Detail & Related papers  (2025-03-18T06:49:51Z)
• Weakly Supervised Video Scene Graph Generation via Natural Language Supervision [27.97296273461145]
  Existing Video Scene Graph Generation (VidSGG) studies are trained in a fully supervised manner.<n>We propose a Natural Language-based Video Scene Graph Generation framework that only utilizes the readily available video captions.
  arXiv  Detail & Related papers  (2025-02-21T10:42:04Z)
• Supra-Laplacian Encoding for Transformer on Dynamic Graphs [14.293220696079919]
  We present a new-temporal encoding for GT architecture while keeping temporal information. Specifically, we transform Time Dynamic Graphplas into multi-layer graphs and take advantage of the spectral properties of their associated supra-lacian matrix. Our second contribution explicitly model nodes pairwise with a cross-attention mechanism, providing an accurate edge representation for dynamic link prediction.
  arXiv  Detail & Related papers  (2024-09-26T15:56:40Z)
• Enhancing Video-Language Representations with Structural Spatio-Temporal Alignment [130.15775113897553]
  Finsta is a fine-grained structural-temporal alignment learning method. It consistently improves the existing 13 strong-tuning video-language models.
  arXiv  Detail & Related papers  (2024-06-27T15:23:36Z)
• Constructing Holistic Spatio-Temporal Scene Graph for Video Semantic Role Labeling [96.64607294592062]
  Video Semantic Label Roleing (VidSRL) aims to detect salient events from given videos. Recent endeavors have put forth methods for VidSRL, but they can be subject to two key drawbacks.
  arXiv  Detail & Related papers  (2023-08-09T17:20:14Z)
• Transform-Equivariant Consistency Learning for Temporal Sentence Grounding [66.10949751429781]
  We introduce a novel Equivariant Consistency Regulation Learning framework to learn more discriminative representations for each video. Our motivation comes from that the temporal boundary of the query-guided activity should be consistently predicted. In particular, we devise a self-supervised consistency loss module to enhance the completeness and smoothness of the augmented video.
  arXiv  Detail & Related papers  (2023-05-06T19:29:28Z)
• RU-Net: Regularized Unrolling Network for Scene Graph Generation [92.95032610978511]
  Scene graph generation (SGG) aims to detect objects and predict the relationships between each pair of objects. Existing SGG methods usually suffer from several issues, including 1) ambiguous object representations, and 2) low diversity in relationship predictions. We propose a regularized unrolling network (RU-Net) to address both problems.
  arXiv  Detail & Related papers  (2022-05-03T04:21:15Z)
• Temporal Relational Modeling with Self-Supervision for Action Segmentation [38.62057004624234]
  We introduce Dilated Temporal Graph Reasoning Module (DTGRM) to model temporal relations in video. In particular, we capture and model temporal relations via constructing multi-level dilated temporal graphs. Our model outperforms state-of-the-art action segmentation models on three challenging datasets.
  arXiv  Detail & Related papers  (2020-12-14T13:41:28Z)
• Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks [150.5425122989146]
  This work proposes a novel attentive graph neural network (AGNN) for zero-shot video object segmentation (ZVOS) AGNN builds a fully connected graph to efficiently represent frames as nodes, and relations between arbitrary frame pairs as edges. Experimental results on three video segmentation datasets show that AGNN sets a new state-of-the-art in each case.
  arXiv  Detail & Related papers  (2020-01-19T10:45:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.