Towards Text-free Graph Foundation Models: Rethinking Multi-Domain Graph Contrastive Learning

• URL: http://arxiv.org/abs/2506.22510v1
• Date: Thu, 26 Jun 2025 03:14:50 GMT
• Title: Towards Text-free Graph Foundation Models: Rethinking Multi-Domain Graph Contrastive Learning
• Authors: Zihao Zhao, Xinlong Zhai, Jinyu Yang, Chuan Shi,
• Abstract summary: We propose a novel multi-domain pre-training and cross-domain transfer framework, namely MDGCL.<n>In the pre-training stage, we design a contrastive learning strategy to substantially recognize and capture domain differences.<n>In the downstream stage, we introduce a domain attention mechanism to enable fine-grained domain knowledge transfer.
• Score: 40.56379624114316
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Foundation models have achieved great success in natural language processing (NLP) and computer vision (CV). Their success largely stems from the ability to integrate multi-domain knowledge in pre-training and transfer it to target domains. Considering graph data, especially graphs without textual features, is ubiquitous in real-world applications such as social networks and recommendation systems, some researchers have attempted to extend this paradigm to the graph field, aiming to construct graph foundation models. However, unlike CV and NLP, there are huge gaps among the semantics and properties of graphs in different domains, while current works still adopt traditional contrastive pre-training strategies designed in the single-domain scenario, which regard contrastive samples from different domains as equivalent. From experimental investigations, we discovered that inherent domain-specific differences prevent these strategies from effectively absorbing knowledge from different domains to generate informative representations. In this paper, we propose a novel multi-domain pre-training and cross-domain transfer framework, namely MDGCL.In the pre-training stage, we design a contrastive learning strategy to substantially recognize and capture domain differences, and introduce domain tokens to encode domain-level global information. In the downstream stage, we introduce a domain attention mechanism to enable fine-grained domain knowledge transfer. Extensive experiments on five benchmark datasets have demonstrated that our method outperforms state-of-the-art significantly, with the maximum improvement of 19.33\% on accuracy and 19.13\% on Macro-F1 score.

Related papers

• What's in a Latent? Leveraging Diffusion Latent Space for Domain Generalization [10.079844840768054]
  Domain Generalization aims to develop models that can generalize to novel and unseen data distributions.<n>We study how model architectures and pre-training objectives impact feature richness.<n>Our framework improves generalization to unseen domains by a maximum test accuracy improvement of over 4%.
  arXiv  Detail & Related papers  (2025-03-09T17:29:01Z)
• Multi-Domain Graph Foundation Models: Robust Knowledge Transfer via Topology Alignment [9.215549756572976]
  Real-world graphs are often sparse and prone to noisy connections and adversarial attacks.<n>We propose the Multi-Domain Graph Foundation Model (MDGFM), a unified framework that aligns and leverages cross-domain topological information.<n>By aligning topologies, MDGFM not only improves multi-domain pre-training but also enables robust knowledge transfer to unseen domains.
  arXiv  Detail & Related papers  (2025-02-04T05:09:23Z)
• Benchmarking Multi-Domain Active Learning on Image Classification [16.690755621494215]
  We introduce a multi-domain active learning benchmark to bridge the gap between research on single-source data and real-world data. Our benchmark demonstrates that traditional single-domain active learning strategies are often less effective than random selection in multi-domain scenarios. Analysis on our benchmark shows that all multi-domain strategies exhibit significant tradeoffs, with no strategy outperforming across all datasets or all metrics.
  arXiv  Detail & Related papers  (2023-12-01T06:11:14Z)
• Curriculum Graph Co-Teaching for Multi-Target Domain Adaptation [78.28390172958643]
  We identify two key aspects that can help to alleviate multiple domain-shifts in the multi-target domain adaptation (MTDA) We propose Curriculum Graph Co-Teaching (CGCT) that uses a dual classifier head, with one of them being a graph convolutional network (GCN) which aggregates features from similar samples across the domains. When the domain labels are available, we propose Domain-aware Curriculum Learning (DCL), a sequential adaptation strategy that first adapts on the easier target domains, followed by the harder ones.
  arXiv  Detail & Related papers  (2021-04-01T23:41:41Z)
• Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning [85.6386289476598]
  We develop a novel adversarial graph representation adaptation (AGRA) framework for cross-domain holistic-local feature co-adaptation. We conduct extensive and fair evaluations on several popular benchmarks and show that the proposed AGRA framework outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-03T15:00:31Z)
• Adversarial Graph Representation Adaptation for Cross-Domain Facial Expression Recognition [86.25926461936412]
  We propose a novel Adrialversa Graph Representation Adaptation (AGRA) framework that unifies graph representation propagation with adversarial learning for cross-domain holistic-local feature co-adaptation. We conduct extensive and fair experiments on several popular benchmarks and show that the proposed AGRA framework achieves superior performance over previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-03T13:27:24Z)
• Adversarial Bipartite Graph Learning for Video Domain Adaptation [50.68420708387015]
  Domain adaptation techniques, which focus on adapting models between distributionally different domains, are rarely explored in the video recognition area. Recent works on visual domain adaptation which leverage adversarial learning to unify the source and target video representations are not highly effective on the videos. This paper proposes an Adversarial Bipartite Graph (ABG) learning framework which directly models the source-target interactions.
  arXiv  Detail & Related papers  (2020-07-31T03:48:41Z)
• Domain Adaptation for Semantic Parsing [68.81787666086554]
  We propose a novel semantic for domain adaptation, where we have much fewer annotated data in the target domain compared to the source domain. Our semantic benefits from a two-stage coarse-to-fine framework, thus can provide different and accurate treatments for the two stages. Experiments on a benchmark dataset show that our method consistently outperforms several popular domain adaptation strategies.
  arXiv  Detail & Related papers  (2020-06-23T14:47:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.