LATEX-GCL: Large Language Models (LLMs)-Based Data Augmentation for Text-Attributed Graph Contrastive Learning

• URL: http://arxiv.org/abs/2409.01145v1
• Date: Mon, 2 Sep 2024 10:30:55 GMT
• Title: LATEX-GCL: Large Language Models (LLMs)-Based Data Augmentation for Text-Attributed Graph Contrastive Learning
• Authors: Haoran Yang, Xiangyu Zhao, Sirui Huang, Qing Li, Guandong Xu,
• Abstract summary: GCL for learning on Text-Attributed Graphs (TAGs) has yet to be explored. A naive strategy for applying GCL to TAGs is to encode the textual attributes into feature embeddings via a language model. We propose a novel GCL framework named LATEX-GCL to utilize Large Language Models (LLMs) to produce textual augmentations.
• Score: 35.69403361648343
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph Contrastive Learning (GCL) is a potent paradigm for self-supervised graph learning that has attracted attention across various application scenarios. However, GCL for learning on Text-Attributed Graphs (TAGs) has yet to be explored. Because conventional augmentation techniques like feature embedding masking cannot directly process textual attributes on TAGs. A naive strategy for applying GCL to TAGs is to encode the textual attributes into feature embeddings via a language model and then feed the embeddings into the following GCL module for processing. Such a strategy faces three key challenges: I) failure to avoid information loss, II) semantic loss during the text encoding phase, and III) implicit augmentation constraints that lead to uncontrollable and incomprehensible results. In this paper, we propose a novel GCL framework named LATEX-GCL to utilize Large Language Models (LLMs) to produce textual augmentations and LLMs' powerful natural language processing (NLP) abilities to address the three limitations aforementioned to pave the way for applying GCL to TAG tasks. Extensive experiments on four high-quality TAG datasets illustrate the superiority of the proposed LATEX-GCL method. The source codes and datasets are released to ease the reproducibility, which can be accessed via this link: https://anonymous.4open.science/r/LATEX-GCL-0712.

Related papers

• Large Language Model-based Augmentation for Imbalanced Node Classification on Text-Attributed Graphs [13.42259312243504]
  We propose a novel approach called LA-TAG (LLM-based Augmentation on Text-Attributed Graphs) It prompts Large Language Models to generate synthetic texts based on existing node texts in the graph. To integrate these synthetic text-attributed nodes into the graph, we introduce a text-based link predictor.
  arXiv  Detail & Related papers  (2024-10-22T10:36:15Z)
• GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphs [27.169892145194638]
  GraphCLIP is a framework to learn graph foundation models with strong cross-domain zero/few-shot transferability. We generate and curate large-scale graph-summary pair data with the assistance of LLMs. For few-shot learning, we propose a novel graph prompt tuning technique aligned with our pretraining objective.
  arXiv  Detail & Related papers  (2024-10-14T09:40:52Z)
• Hierarchical Compression of Text-Rich Graphs via Large Language Models [63.75293588479027]
  Text-rich graphs are prevalent in data mining contexts like e-commerce and academic graphs. This paper introduces Hierarchical Compression'' (HiCom), a novel method to align the capabilities of LLMs with the structure of text-rich graphs. HiCom can outperform both GNNs and LLM backbones for node classification on e-commerce and citation graphs.
  arXiv  Detail & Related papers  (2024-06-13T07:24:46Z)
• LLaGA: Large Language and Graph Assistant [73.71990472543027]
  Large Language and Graph Assistant (LLaGA) is an innovative model to handle the complexities of graph-structured data. LLaGA excels in versatility, generalizability and interpretability, allowing it to perform consistently well across different datasets and tasks. Our experiments show that LLaGA delivers outstanding performance across four datasets and three tasks using one single model.
  arXiv  Detail & Related papers  (2024-02-13T02:03:26Z)
• Large Language Models on Graphs: A Comprehensive Survey [77.16803297418201]
  We provide a systematic review of scenarios and techniques related to large language models on graphs. We first summarize potential scenarios of adopting LLMs on graphs into three categories, namely pure graphs, text-attributed graphs, and text-paired graphs. We discuss the real-world applications of such methods and summarize open-source codes and benchmark datasets.
  arXiv  Detail & Related papers  (2023-12-05T14:14:27Z)
• Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning [51.90524745663737]
  A key innovation is our use of explanations as features, which can be used to boost GNN performance on downstream tasks. Our method achieves state-of-the-art results on well-established TAG datasets. Our method significantly speeds up training, achieving a 2.88 times improvement over the closest baseline on ogbn-arxiv.
  arXiv  Detail & Related papers  (2023-05-31T03:18:03Z)
• MA-GCL: Model Augmentation Tricks for Graph Contrastive Learning [41.963242524220654]
  We present three easy-to-implement model augmentation tricks for graph contrastive learning (GCL) Specifically, we present three easy-to-implement model augmentation tricks for GCL, namely asymmetric, random and shuffling. Experimental results show that MA-GCL can achieve state-of-the-art performance on node classification benchmarks.
  arXiv  Detail & Related papers  (2022-12-14T05:04:10Z)
• Unifying Graph Contrastive Learning with Flexible Contextual Scopes [57.86762576319638]
  We present a self-supervised learning method termed Unifying Graph Contrastive Learning with Flexible Contextual Scopes (UGCL for short) Our algorithm builds flexible contextual representations with contextual scopes by controlling the power of an adjacency matrix. Based on representations from both local and contextual scopes, distL optimises a very simple contrastive loss function for graph representation learning.
  arXiv  Detail & Related papers  (2022-10-17T07:16:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.