Learning Robust Heterogeneous Graph Representations via Contrastive-Reconstruction under Sparse Semantics

• URL: http://arxiv.org/abs/2506.06682v1
• Date: Sat, 07 Jun 2025 06:35:42 GMT
• Title: Learning Robust Heterogeneous Graph Representations via Contrastive-Reconstruction under Sparse Semantics
• Authors: Di Lin, Wanjing Ren, Xuanbin Li, Rui Zhang,
• Abstract summary: Masked autoencoders (MAE) and contrastive learning (CL) are two prominent paradigms in graph self-supervised learning.<n>This paper introduces HetCRF, a novel dual-channel self-supervised learning framework for heterogeneous graphs.<n>HetCRF uses a two-stage aggregation strategy to adapt embedding semantics, making it compatible with both MAE and CL.
• Score: 13.555683316315683
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In graph self-supervised learning, masked autoencoders (MAE) and contrastive learning (CL) are two prominent paradigms. MAE focuses on reconstructing masked elements, while CL maximizes similarity between augmented graph views. Recent studies highlight their complementarity: MAE excels at local feature capture, and CL at global information extraction. Hybrid frameworks for homogeneous graphs have been proposed, but face challenges in designing shared encoders to meet the semantic requirements of both tasks. In semantically sparse scenarios, CL struggles with view construction, and gradient imbalance between positive and negative samples persists. This paper introduces HetCRF, a novel dual-channel self-supervised learning framework for heterogeneous graphs. HetCRF uses a two-stage aggregation strategy to adapt embedding semantics, making it compatible with both MAE and CL. To address semantic sparsity, it enhances encoder output for view construction instead of relying on raw features, improving efficiency. Two positive sample augmentation strategies are also proposed to balance gradient contributions. Node classification experiments on four real-world heterogeneous graph datasets demonstrate that HetCRF outperforms state-of-the-art baselines. On datasets with missing node features, such as Aminer and Freebase, at a 40% label rate in node classification, HetCRF improves the Macro-F1 score by 2.75% and 2.2% respectively compared to the second-best baseline, validating its effectiveness and superiority.

Related papers

• CIMAGE: Exploiting the Conditional Independence in Masked Graph Auto-encoders [3.700463358780727]
  Conditional Independence (CI) inherently satisfies the minimum redundancy and maximum relevance criteria, but its application typically requires access to downstream labels.<n>We introduce CIMAGE, a novel approach that leverages Conditional Independence to guide an effective masking strategy within the latent space.<n>Our theoretical analysis further supports the superiority of CIMAGE's novel CI-aware masking method by demonstrating that the learned embedding exhibits approximate linear separability.
  arXiv  Detail & Related papers  (2025-03-10T20:59:27Z)
• Graph Structure Refinement with Energy-based Contrastive Learning [56.957793274727514]
  We introduce an unsupervised method based on a joint of generative training and discriminative training to learn graph structure and representation.<n>We propose an Energy-based Contrastive Learning (ECL) guided Graph Structure Refinement (GSR) framework, denoted as ECL-GSR.<n>ECL-GSR achieves faster training with fewer samples and memories against the leading baseline, highlighting its simplicity and efficiency in downstream tasks.
  arXiv  Detail & Related papers  (2024-12-20T04:05:09Z)
• GRE^2-MDCL: Graph Representation Embedding Enhanced via Multidimensional Contrastive Learning [0.0]
  Graph representation learning has emerged as a powerful tool for preserving graph topology when mapping nodes to vector representations.<n>Current graph neural network models face the challenge of requiring extensive labeled data.<n>We propose Graph Representation Embedding Enhanced via Multidimensional Contrastive Learning.
  arXiv  Detail & Related papers  (2024-09-12T03:09:05Z)
• UGMAE: A Unified Framework for Graph Masked Autoencoders [67.75493040186859]
  We propose UGMAE, a unified framework for graph masked autoencoders. We first develop an adaptive feature mask generator to account for the unique significance of nodes. We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information.
  arXiv  Detail & Related papers  (2024-02-12T19:39:26Z)
• Rethinking Graph Masked Autoencoders through Alignment and Uniformity [26.86368034133612]
  Self-supervised learning on graphs can be bifurcated into contrastive and generative methods. Recent advent of graph masked autoencoder (GraphMAE) rekindles momentum behind generative methods.
  arXiv  Detail & Related papers  (2024-02-11T15:21:08Z)
• Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning [56.45977379288308]
  Masked autoencoder (MAE) learns to reconstruct masked graph edges or node features. Contrastive Learning (CL) maximizes the similarity between augmented views of the same graph. We propose graph contrastive masked autoencoder (GCMAE) framework to unify MAE and CL.
  arXiv  Detail & Related papers  (2023-10-24T05:06:06Z)
• GraphLearner: Graph Node Clustering with Fully Learnable Augmentation [76.63963385662426]
  Contrastive deep graph clustering (CDGC) leverages the power of contrastive learning to group nodes into different clusters. We propose a Graph Node Clustering with Fully Learnable Augmentation, termed GraphLearner. It introduces learnable augmentors to generate high-quality and task-specific augmented samples for CDGC.
  arXiv  Detail & Related papers  (2022-12-07T10:19:39Z)
• Single-Pass Contrastive Learning Can Work for Both Homophilic and Heterophilic Graph [60.28340453547902]
  Graph contrastive learning (GCL) techniques typically require two forward passes for a single instance to construct the contrastive loss. Existing GCL approaches fail to provide strong performance guarantees. We implement the Single-Pass Graph Contrastive Learning method (SP-GCL) Empirically, the features learned by the SP-GCL can match or outperform existing strong baselines with significantly less computational overhead.
  arXiv  Detail & Related papers  (2022-11-20T07:18:56Z)
• Mixed Graph Contrastive Network for Semi-Supervised Node Classification [63.924129159538076]
  We propose a novel graph contrastive learning method, termed Mixed Graph Contrastive Network (MGCN)<n>In our method, we improve the discriminative capability of the latent embeddings by an unperturbed augmentation strategy and a correlation reduction mechanism.<n>By combining the two settings, we extract rich supervision information from both the abundant nodes and the rare yet valuable labeled nodes for discriminative representation learning.
  arXiv  Detail & Related papers  (2022-06-06T14:26:34Z)
• Graph Contrastive Learning with Adaptive Augmentation [23.37786673825192]
  We propose a novel graph contrastive representation learning method with adaptive augmentation. Specifically, we design augmentation schemes based on node centrality measures to highlight important connective structures. Our proposed method consistently outperforms existing state-of-the-art baselines and even surpasses some supervised counterparts.
  arXiv  Detail & Related papers  (2020-10-27T15:12:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.