MOSA: Motion-Guided Semantic Alignment for Dynamic Scene Graph Generation

• URL: http://arxiv.org/abs/2604.19631v1
• Date: Tue, 21 Apr 2026 16:18:26 GMT
• Title: MOSA: Motion-Guided Semantic Alignment for Dynamic Scene Graph Generation
• Abstract summary: This paper proposes a motion-guided semantic alignment method for Dynamic Scene Graph Generation (DSGG)<n>MoSA encodes object-pair motion attributes such as distance, velocity, motion persistence, and directional consistency.<n>Cross-modal Action Semantic Matching mechanism aligns visual relationship features with text embeddings of relationship categories.
• Score: 27.7774056956553
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dynamic Scene Graph Generation (DSGG) aims to structurally model objects and their dynamic interactions in video sequences for high-level semantic understanding. However, existing methods struggle with fine-grained relationship modeling, semantic representation utilization, and the ability to model tail relationships. To address these issues, this paper proposes a motion-guided semantic alignment method for DSGG (MoSA). First, a Motion Feature Extractor (MFE) encodes object-pair motion attributes such as distance, velocity, motion persistence, and directional consistency. Then, these motion attributes are fused with spatial relationship features through the Motion-guided Interaction Module (MIM) to generate motion-aware relationship representations. To further enhance semantic discrimination capabilities, the cross-modal Action Semantic Matching (ASM) mechanism aligns visual relationship features with text embeddings of relationship categories. Finally, a category-weighted loss strategy is introduced to emphasize learning of tail relationships. Extensive and rigorous testing shows that MoSA performs optimally on the Action Genome dataset.

Related papers

• Following Motion for Sequential Modeling in Video Frame Interpolation [22.71397762551783]
  State Space Models (SSMs) have surfaced as a promising architecture in Video Frame Interpolation (VFI)<n>We propose Motion-Guided Mamba for Video Frame Interpolation (MGMVFI), an adaptation of the selective state space model tailored explicitly for VFI.
  arXiv  Detail & Related papers  (2026-08-24T06:43:48Z)
• PRISM: Iterative Cross-Modal Posterior Refinement for Dynamic Text-Attributed Graphs [5.2307029116231165]
  multimodal learning has emerged as a promising yet underexplored direction for enhancing DyTAG representation learning.<n>We propose textbfPRISM, an iterative cross-modal posterior refinement framework for DyTAG representation learning.<n>PRISM organizes DyTAG information into semantic and behavioral modalities, providing a more intrinsic alternative to carrier-level modality partitions.
  arXiv  Detail & Related papers  (2026-05-07T11:58:47Z)
• InstrAct: Towards Action-Centric Understanding in Instructional Videos [12.356484522873577]
  InstrAction is a pretraining framework for instructional videos' action-centric representations.<n>We first introduce a data-driven strategy, which filters noisy captions and generates action-centric hard negatives.<n>At the visual feature level, an Action Perceiver extracts motion-relevant tokens from redundant video encodings.
  arXiv  Detail & Related papers  (2026-04-09T20:51:13Z)
• IM-Animation: An Implicit Motion Representation for Identity-decoupled Character Animation [58.297199313494]
  Implicit methods capture motion semantics directly from driving video, but suffer from identity leakage and entanglement between motion and appearance.<n>We propose a novel implicit motion representation that compresses per-frame motion into compact 1D motion tokens.<n>Our methodology employs a three-stage training strategy to enhance the training efficiency and ensure high fidelity.
  arXiv  Detail & Related papers  (2026-02-07T11:17:20Z)
• IRG-MotionLLM: Interleaving Motion Generation, Assessment and Refinement for Text-to-Motion Generation [54.36300724708094]
  Assessment and refinement tasks act as crucial bridges to enable bidirectional knowledge flow between understanding and generation.<n>We introduce IRG-MotionLLM, the first model that seamlessly interleaves motion generation, assessment, and refinement to improve generation performance.
  arXiv  Detail & Related papers  (2025-12-11T15:16:06Z)
• A Renaissance of Explicit Motion Information Mining from Transformers for Action Recognition [87.12969639957441]
  Action recognition has been dominated by transformer-based methods, thanks to their contextual aggregation capacities.<n>We propose to integrate those effective motion modeling properties into the existing transformer in a unified and neat way.<n>Our method performs better than existing state-of-the-art approaches, especially on motion-sensitive datasets.
  arXiv  Detail & Related papers  (2025-10-21T15:01:48Z)
• MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editing [53.98607267063729]
  MotionVerse is a framework to comprehend, generate, and edit human motion in both single-person and multi-person scenarios.<n>We employ a motion tokenizer with residual quantization, which converts continuous motion sequences into multi-stream discrete tokens.<n>We also introduce a textitDelay Parallel Modeling strategy, which temporally staggers the encoding of residual token streams.
  arXiv  Detail & Related papers  (2025-09-28T04:20:56Z)
• SynMotion: Semantic-Visual Adaptation for Motion Customized Video Generation [56.90807453045657]
  SynMotion is a motion-customized video generation model that jointly leverages semantic guidance and visual adaptation.<n>At the semantic level, we introduce the dual-em semantic comprehension mechanism which disentangles subject and motion representations.<n>At the visual level, we integrate efficient motion adapters into a pre-trained video generation model to enhance motion fidelity and temporal coherence.
  arXiv  Detail & Related papers  (2025-06-30T10:09:32Z)
• Text-Derived Relational Graph-Enhanced Network for Skeleton-Based Action Segmentation [14.707224594220264]
  We propose a Text-Derived Graph Network (TRG-Net) to enhance both modeling and supervision.<n>For modeling, the Dynamic Spatio-Temporal Fusion Modeling (D) method incorporates Text-Derived Joint Graphs (JGT) with channel adaptation.<n>For supervision, the Absolute-Relative Inter-Class Supervision (ARIS) method employs contrastive learning between action features and text embeddings to regularize the absolute class.
  arXiv  Detail & Related papers  (2025-03-19T11:38:14Z)
• SemanticBoost: Elevating Motion Generation with Augmented Textual Cues [73.83255805408126]
  Our framework comprises a Semantic Enhancement module and a Context-Attuned Motion Denoiser (CAMD) The CAMD approach provides an all-encompassing solution for generating high-quality, semantically consistent motion sequences. Our experimental results demonstrate that SemanticBoost, as a diffusion-based method, outperforms auto-regressive-based techniques.
  arXiv  Detail & Related papers  (2023-10-31T09:58:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.