SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptation

• URL: http://arxiv.org/abs/2502.05424v1
• Date: Sat, 08 Feb 2025 03:24:25 GMT
• Title: SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptation
• Authors: Xingtong Yu, Zechuan Gong, Chang Zhou, Yuan Fang, Hui Zhang,
• Abstract summary: We propose a novel Structure Alignment framework for text-free Multi-domain Graph Pre-Training and cross-domain adaptation. It is designed to learn multi-domain knowledge from graphs originating in multiple source domains, which can then be adapted to address applications in an unseen target domain.
• Score: 29.885969929608216
• License:
• Abstract: Graphs are able to model interconnected entities in many online services, supporting a wide range of applications on the Web. This raises an important question: How can we train a graph foundational model on multiple source domains and adapt to an unseen target domain? A major obstacle is that graphs from different domains often exhibit divergent characteristics. Some studies leverage large language models to align multiple domains based on textual descriptions associated with the graphs, limiting their applicability to text-attributed graphs. For text-free graphs, a few recent works attempt to align different feature distributions across domains, while generally neglecting structural differences. In this work, we propose a novel Structure Alignment framework for text-free Multi-domain Graph Pre-Training and cross-domain adaptation (SAMGPT). It is designed to learn multi-domain knowledge from graphs originating in multiple source domains, which can then be adapted to address applications in an unseen target domain. Specifically, we introduce a set of structure tokens to harmonize structure-based aggregation across source domains during the pre-training phase. Next, for cross-domain adaptation, we design dual prompts, namely, holistic prompts and specific prompts, which adapt unified multi-domain structural knowledge and fine-grained, domain-specific information, respectively, to a target domain. Finally, we conduct comprehensive experiments on seven public datasets to evaluate and analyze the effectiveness of SAMGPT.

Related papers

• SA-GDA: Spectral Augmentation for Graph Domain Adaptation [38.71041292000361]
  Graph neural networks (GNNs) have achieved impressive impressions for graph-related tasks. This paper presents the textitSpectral Augmentation for Graph Domain Adaptation (method) for graph node classification. We develop a dual graph convolutional network to jointly exploits local and global consistency for feature aggregation.
  arXiv  Detail & Related papers  (2024-08-17T13:01:45Z)
• A Unified Data Augmentation Framework for Low-Resource Multi-Domain Dialogue Generation [52.0964459842176]
  Current state-of-the-art dialogue systems heavily rely on extensive training datasets. We propose a novel data textbfAugmentation framework for textbfMulti-textbfDomain textbfDialogue textbfGeneration, referred to as textbfAMD$2$G. The AMD$2$G framework consists of a data augmentation process and a two-stage training approach: domain-agnostic training and domain adaptation training.
  arXiv  Detail & Related papers  (2024-06-14T09:52:27Z)
• Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Models [33.2696184519275]
  We propose MDGPT, a text free Multi-Domain Graph Pre-Training and adaptation framework. First, we propose a set of domain tokens to align features across source domains for synergistic pre-training. Second, we propose a dual prompts, consisting of a unifying prompt and a mixing prompt, to further adapt the target domain with unified multi-domain knowledge.
  arXiv  Detail & Related papers  (2024-05-22T19:06:39Z)
• Large-Scale Multi-Domain Recommendation: an Automatic Domain Feature Extraction and Personalized Integration Framework [30.46152832695426]
  We propose an Automatic Domain Feature Extraction and Personalized Integration (DFEI) framework for the large-scale multi-domain recommendation. The framework automatically transforms the behavior of each individual user into an aggregation of all user behaviors within the domain, which serves as the domain features. Experimental results on both public and industrial datasets, consisting of over 20 domains, clearly demonstrate that the proposed framework achieves significantly better performance compared with SOTA baselines.
  arXiv  Detail & Related papers  (2024-04-12T09:57:17Z)
• Bidirectional Generative Framework for Cross-domain Aspect-based Sentiment Analysis [68.742820522137]
  Cross-domain aspect-based sentiment analysis (ABSA) aims to perform various fine-grained sentiment analysis tasks on a target domain by transferring knowledge from a source domain. We propose a unified bidirectional generative framework to tackle various cross-domain ABSA tasks. Our framework trains a generative model in both text-to-label and label-to-text directions.
  arXiv  Detail & Related papers  (2023-05-16T15:02:23Z)
• Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval [55.122020263319634]
  Video moment retrieval (VMR) aims to localize the target moment from an untrimmed video according to a given language query. In this paper, we focus on a novel task: cross-domain VMR, where fully-annotated datasets are available in one domain but the domain of interest only contains unannotated datasets. We propose a novel Multi-Modal Cross-Domain Alignment network to transfer the annotation knowledge from the source domain to the target domain.
  arXiv  Detail & Related papers  (2022-09-23T12:58:20Z)
• Finding Diverse and Predictable Subgraphs for Graph Domain Generalization [88.32356432272356]
  This paper focuses on out-of-distribution generalization on graphs where performance drops due to the unseen distribution shift. We propose a new graph domain generalization framework, dubbed as DPS, by constructing multiple populations from the source domains. Experiments on both node-level and graph-level benchmarks shows that the proposed DPS achieves impressive performance for various graph domain generalization tasks.
  arXiv  Detail & Related papers  (2022-06-19T07:57:56Z)
• Dynamic Instance Domain Adaptation [109.53575039217094]
  Most studies on unsupervised domain adaptation assume that each domain's training samples come with domain labels. We develop a dynamic neural network with adaptive convolutional kernels to generate instance-adaptive residuals to adapt domain-agnostic deep features to each individual instance. Our model, dubbed DIDA-Net, achieves state-of-the-art performance on several commonly used single-source and multi-source UDA datasets.
  arXiv  Detail & Related papers  (2022-03-09T20:05:54Z)
• Multi-Domain Incremental Learning for Semantic Segmentation [42.30646442211311]
  We propose a dynamic architecture that assigns universally shared, domain-invariant parameters to capture homogeneous semantic features. We demonstrate the effectiveness of our proposed solution on domain incremental settings pertaining to real-world driving scenes from roads of Germany (Cityscapes), the United States (BDD100k), and India (IDD)
  arXiv  Detail & Related papers  (2021-10-23T12:21:42Z)
• Curriculum Graph Co-Teaching for Multi-Target Domain Adaptation [78.28390172958643]
  We identify two key aspects that can help to alleviate multiple domain-shifts in the multi-target domain adaptation (MTDA) We propose Curriculum Graph Co-Teaching (CGCT) that uses a dual classifier head, with one of them being a graph convolutional network (GCN) which aggregates features from similar samples across the domains. When the domain labels are available, we propose Domain-aware Curriculum Learning (DCL), a sequential adaptation strategy that first adapts on the easier target domains, followed by the harder ones.
  arXiv  Detail & Related papers  (2021-04-01T23:41:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.