Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphs

• URL: http://arxiv.org/abs/2606.14172v1
• Date: Fri, 12 Jun 2026 06:53:46 GMT
• Title: Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphs
• Abstract summary: MultimodalAttributed Graphs (MAGs) model real-world entities by coupling graph topology with heterogeneous attributes such as text and images.<n>Existing MAG methods often rely on fixed graph contexts or uniformly fused representations, causing task-agnostic propagation and over-compressed fusion.<n>We propose CoMAG, a unified MAG that learns task-adaptive reliable contexts and modality-preserving alignment within them.
• Score: 8.922335341563363
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal Attributed Graphs (MAGs) model real-world entities by coupling graph topology with heterogeneous attributes such as text and images. They support graph-centric tasks requiring structural and class-discriminative representations, and modality-centric tasks requiring fine-grained cross-modal correspondence. However, existing MAG methods often rely on fixed graph contexts or uniformly fused representations, causing task-agnostic propagation and over-compressed fusion that hinder diverse task requirements and modality-specific evidence preservation. To address this, we propose CoMAG, a unified MAG backbone that learns task-adaptive reliable contexts and modality-preserving alignment within them. CoMAG first conducts Reliable Context Learning by estimating edge reliability from multimodal semantic consistency, complementing raw topology with semantic neighbors, and selecting context components through a task-aware gate. It then performs Modality-preserving Hop-token Alignment by maintaining modality-specific multi-hop trajectories, matching modality-hop tokens across modalities, and decoupling shared and private representations. Thus, CoMAG produces graph and modality representations from one forward pass while retaining modality-specific cues. We further analyze stable propagation, over-smoothing mitigation, and modality-collapse control. Experiments on nine OpenMAG datasets compare CoMAG with feature-only, graph-only, multimodal, and unified MAG baselines across graph-level prediction, modality matching, and graph-conditioned generation. Results show that CoMAG achieves the best reported performance, demonstrating that task-adaptive reliable contexts and modality-preserving alignment improve structural prediction, cross-modal matching, and graph-conditioned generation while retaining sparse edge-linear complexity.

Related papers

• SMGFM: Spectral Multimodal Graph Pretraining for Multimodal-Attributed Graphs [38.400117494307615]
  Multimodal-attributed graphs (MAGs) couple graph topology with node semantics from text, images, and other modalities.<n>We propose SMGFM, a spectral multimodal graph pretraining framework that decomposes each modality-specific node signal into graph-frequency bands.<n>SMGFM achieves state-of-the-art performance across graph-level and modality-level tasks.
  arXiv  Detail & Related papers  (2026-06-11T03:54:35Z)
• GOMA: Toward Structure-Driven Multimodal Alignment from a Graph Signal Smoothing Perspective [39.44851011793204]
  We propose a structure-driven post-alignment framework that views frozen multimodal embeddings as graph signals.<n>GOMA decouples three key design choices: where messages should flow, how multimodal evidence should propagate, and which smoothing depth should be retained.<n>All experiments follow a transductive MAG retrieval protocol where the graph serves only as unlabeled context and diagonal self-pair edges are removed.
  arXiv  Detail & Related papers  (2026-05-15T08:21:22Z)
• STAGE: Tackling Semantic Drift in Multimodal Federated Graph Learning [48.96526409527749]
  Federated graph learning (FGL) enables collaborative training on graph data across multiple clients.<n>The key challenge is that clients from different modality domains may not share a common semantic space.<n>We propose textbfSTAGE, a protocol-first framework for MM-FGL.
  arXiv  Detail & Related papers  (2026-05-12T10:35:43Z)
• Robust Multimodal Recommendation via Graph Retrieval-Enhanced Modality Completion [71.20734649881258]
  Real-world multimodal datasets often suffer from modality incompleteness due to sensor failures, annotation scarcity, or privacy constraints.<n>One effective solution is modality completion, which reconstructs missing features to provide modality-complete graphs for downstream tasks.<n>We propose GRE-MC, a Graph Retrieval-Enhanced Modality Completion framework, to overcome these limitations.
  arXiv  Detail & Related papers  (2026-05-01T13:50:52Z)
• TMTE: Effective Multimodal Graph Learning with Task-aware Modality and Topology Co-evolution [12.027339810975079]
  Task-aware Modality and Topology co-Evolution (TMTE) is a novel MGL framework that jointly and iteratively optimize graph topology and multimodal representations toward the target task.<n>TMTE consistently achieves state-of-the-art performance across 6 graph-centric and modality-centric tasks.
  arXiv  Detail & Related papers  (2026-03-29T14:54:59Z)
• OptiMAG: Structure-Semantic Alignment via Unbalanced Optimal Transport [37.640303159988015]
  Multimodal Attributed Graphs (MAGs) have been widely adopted for modeling complex systems by integrating multi-modal information, such as text and images, on nodes.<n>We identify a discrepancy between the implicit semantic structure induced by different modality embeddings and the explicit graph structure.<n>Since existing methods typically perform message passing over the fixed explicit graph structure, they inadvertently aggregate dissimilar features.<n>We propose OptiMAG, an Unbalanced Optimal Transport-based regularization framework.
  arXiv  Detail & Related papers  (2026-01-30T11:29:03Z)
• GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning [92.19531718298744]
  Graph Foundation Models (GFMs) hold promise for broad applicability across diverse graph tasks and domains.<n>Existing GFMs struggle with unstable few-shot fine-tuning.<n>We propose GRAVER, a novel Generative gRAph VocabulariEs for Robust GFM fine-tuning framework.
  arXiv  Detail & Related papers  (2025-11-05T13:07:26Z)
• EGRA:Toward Enhanced Behavior Graphs and Representation Alignment for Multimodal Recommendation [50.848374648774374]
  MultiModal Recommendation (MMR) systems have emerged as a promising solution for improving recommendation quality by leveraging rich item-side modality information.<n>We propose EGRA, which incorporates into the behavior graph an item-item graph built from representations generated by a pretrained MMR model.<n>It also introduces a novel bi-level dynamic alignment weighting mechanism to improve modality-behavior representation alignment.
  arXiv  Detail & Related papers  (2025-08-22T07:47:54Z)
• Refining Contrastive Learning and Homography Relations for Multi-Modal Recommendation [19.01114538768217]
  We propose a novel framework for textbfRtextbfEfining multi-modtextbfAl conttextbfRastive learning and hotextbfMography relations.<n>Our experiments on three real-world datasets demonstrate the superiority of REARM to various state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-08-19T11:35:48Z)
• A Pure Transformer Pretraining Framework on Text-attributed Graphs [50.833130854272774]
  We introduce a feature-centric pretraining perspective by treating graph structure as a prior. Our framework, Graph Sequence Pretraining with Transformer (GSPT), samples node contexts through random walks. GSPT can be easily adapted to both node classification and link prediction, demonstrating promising empirical success on various datasets.
  arXiv  Detail & Related papers  (2024-06-19T22:30:08Z)
• Jointly Cross- and Self-Modal Graph Attention Network for Query-Based Moment Localization [77.21951145754065]
  We propose a novel Cross- and Self-Modal Graph Attention Network (CSMGAN) that recasts this task as a process of iterative messages passing over a joint graph. Our CSMGAN is able to effectively capture high-order interactions between two modalities, thus enabling a further precise localization.
  arXiv  Detail & Related papers  (2020-08-04T08:25:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.