Can Large Language Models Act as Ensembler for Multi-GNNs?

• URL: http://arxiv.org/abs/2410.16822v1
• Date: Tue, 22 Oct 2024 08:48:52 GMT
• Title: Can Large Language Models Act as Ensembler for Multi-GNNs?
• Authors: Hanqi Duan, Yao Cheng, Jianxiang Yu, Xiang Li,
• Abstract summary: Graph Neural Networks (GNNs) have emerged as powerful models for learning from graph-structured data. GNNs lack the inherent semantic understanding capability of rich textual nodesattributes, limiting their effectiveness in applications. This research advances text-attributed graph ensemble learning by providing a robust, superior solution for integrating semantic and structural information.
• Score: 6.387816922598151
• License:
• Abstract: Graph Neural Networks (GNNs) have emerged as powerful models for learning from graph-structured data. However, GNNs lack the inherent semantic understanding capability of rich textual nodesattributes, limiting their effectiveness in applications. On the other hand, we empirically observe that for existing GNN models, no one can consistently outperforms others across diverse datasets. In this paper, we study whether LLMs can act as an ensembler for multi-GNNs and propose the LensGNN model. The model first aligns multiple GNNs, mapping the representations of different GNNs into the same space. Then, through LoRA fine-tuning, it aligns the space between the GNN and the LLM, injecting graph tokens and textual information into LLMs. This allows LensGNN to integrate multiple GNNs and leverage LLM's strengths, resulting in better performance. Experimental results show that LensGNN outperforms existing models. This research advances text-attributed graph ensemble learning by providing a robust, superior solution for integrating semantic and structural information. We provide our code and data here: https://anonymous.4open.science/r/EnsemGNN-E267/.

Related papers

• All Against Some: Efficient Integration of Large Language Models for Message Passing in Graph Neural Networks [51.19110891434727]
  Large Language Models (LLMs) with pretrained knowledge and powerful semantic comprehension abilities have recently shown a remarkable ability to benefit applications using vision and text data. E-LLaGNN is a framework with an on-demand LLM service that enriches message passing procedure of graph learning by enhancing a limited fraction of nodes from the graph.
  arXiv  Detail & Related papers  (2024-07-20T22:09:42Z)
• LOGIN: A Large Language Model Consulted Graph Neural Network Training Framework [30.54068909225463]
  We aim to streamline the GNN design process and leverage the advantages of Large Language Models (LLMs) to improve the performance of GNNs on downstream tasks. We formulate a new paradigm, coined "LLMs-as-Consultants," which integrates LLMs with GNNs in an interactive manner. We empirically evaluate the effectiveness of LOGIN on node classification tasks across both homophilic and heterophilic graphs.
  arXiv  Detail & Related papers  (2024-05-22T18:17:20Z)
• E2GNN: Efficient Graph Neural Network Ensembles for Semi-Supervised Classification [30.55931541782854]
  This work studies ensemble learning for graph neural networks (GNNs) under the popular semi-supervised setting. We propose an efficient ensemble learner--E2GNN to assemble multiple GNNs in a learnable way by leveraging both labeled and unlabeled nodes. Comprehensive experiments over both transductive and inductive settings, across different GNN backbones and 8 benchmark datasets, demonstrate the superiority of E2GNN.
  arXiv  Detail & Related papers  (2024-05-06T12:11:46Z)
• Label-free Node Classification on Graphs with Large Language Models (LLMS) [46.937442239949256]
  This work introduces a label-free node classification on graphs with Large Language Models pipeline, LLM-GNN. Itates the strengths of both GNNs and LLMs while mitigating their limitations. In particular, LLM-GNN can achieve an accuracy of 74.9% on a vast-scale dataset with a cost less than 1 dollar.
  arXiv  Detail & Related papers  (2023-10-07T03:14:11Z)
• Graph Ladling: Shockingly Simple Parallel GNN Training without Intermediate Communication [100.51884192970499]
  GNNs are a powerful family of neural networks for learning over graphs. scaling GNNs either by deepening or widening suffers from prevalent issues of unhealthy gradients, over-smoothening, information squashing. We propose not to deepen or widen current GNNs, but instead present a data-centric perspective of model soups tailored for GNNs.
  arXiv  Detail & Related papers  (2023-06-18T03:33:46Z)
• Graph Neural Networks are Inherently Good Generalizers: Insights by Bridging GNNs and MLPs [71.93227401463199]
  This paper pinpoints the major source of GNNs' performance gain to their intrinsic capability, by introducing an intermediate model class dubbed as P(ropagational)MLP. We observe that PMLPs consistently perform on par with (or even exceed) their GNN counterparts, while being much more efficient in training.
  arXiv  Detail & Related papers  (2022-12-18T08:17:32Z)
• AKE-GNN: Effective Graph Learning with Adaptive Knowledge Exchange [14.919474099848816]
  Graph Neural Networks (GNNs) have already been widely used in various graph mining tasks. Recent works reveal that the learned weights (channels) in well-trained GNNs are highly redundant, which limits the performance of GNNs. We introduce a novel GNN learning framework named AKE-GNN, which performs the Adaptive Knowledge Exchange strategy.
  arXiv  Detail & Related papers  (2021-06-10T02:00:26Z)
• FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks [68.64678614325193]
  Graph Neural Network (GNN) research is rapidly growing thanks to the capacity of GNNs to learn representations from graph-structured data. Centralizing a massive amount of real-world graph data for GNN training is prohibitive due to user-side privacy concerns. We introduce FedGraphNN, an open research federated learning system and a benchmark to facilitate GNN-based FL research.
  arXiv  Detail & Related papers  (2021-04-14T22:11:35Z)
• XGNN: Towards Model-Level Explanations of Graph Neural Networks [113.51160387804484]
  Graphs neural networks (GNNs) learn node features by aggregating and combining neighbor information. GNNs are mostly treated as black-boxes and lack human intelligible explanations. We propose a novel approach, known as XGNN, to interpret GNNs at the model-level.
  arXiv  Detail & Related papers  (2020-06-03T23:52:43Z)
• Efficient Probabilistic Logic Reasoning with Graph Neural Networks [63.099999467118245]
  Markov Logic Networks (MLNs) can be used to address many knowledge graph problems. Inference in MLN is computationally intensive, making the industrial-scale application of MLN very difficult. We propose a graph neural network (GNN) variant, named ExpressGNN, which strikes a nice balance between the representation power and the simplicity of the model.
  arXiv  Detail & Related papers  (2020-01-29T23:34:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.