Parametric Graph Representations in the Era of Foundation Models: A Survey and Position

• URL: http://arxiv.org/abs/2410.12126v1
• Date: Wed, 16 Oct 2024 00:01:31 GMT
• Title: Parametric Graph Representations in the Era of Foundation Models: A Survey and Position
• Authors: Dongqi Fu, Liri Fang, Zihao Li, Hanghang Tong, Vetle I. Torvik, Jingrui He,
• Abstract summary: Graphs have been widely used in the past decades of big data and AI to model comprehensive relational data. Identifying meaningful graph laws can significantly enhance the effectiveness of various applications.
• Score: 69.48708136448694
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Graphs have been widely used in the past decades of big data and AI to model comprehensive relational data. When analyzing a graph's statistical properties, graph laws serve as essential tools for parameterizing its structure. Identifying meaningful graph laws can significantly enhance the effectiveness of various applications, such as graph generation and link prediction. Facing the large-scale foundation model developments nowadays, the study of graph laws reveals new research potential, e.g., providing multi-modal information for graph neural representation learning and breaking the domain inconsistency of different graph data. In this survey, we first review the previous study of graph laws from multiple perspectives, i.e., macroscope and microscope of graphs, low-order and high-order graphs, static and dynamic graphs, different observation spaces, and newly proposed graph parameters. After we review various real-world applications benefiting from the guidance of graph laws, we conclude the paper with current challenges and future research directions.

Related papers

• GraphSOS: Graph Sampling and Order Selection to Help LLMs Understand Graphs Better [13.742220809751627]
  GraphSOS is a novel framework for converting graph data into natural language text. It features an Order Selector Module to ensure proper serialization order of the graph and a Subgraph Sampling Module to sample subgraphs with better structure for better reasoning. Experiments on multiple datasets for node classification and graph question-answering demonstrate that GraphSOS improves LLMs' performance and ability on graph tasks.
  arXiv  Detail & Related papers  (2025-01-24T11:55:57Z)
• How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehension [53.6373473053431]
  This work introduces a benchmark to assess large language models' capabilities in graph pattern tasks. We have developed a benchmark that evaluates whether LLMs can understand graph patterns based on either terminological or topological descriptions. Our benchmark encompasses both synthetic and real datasets, and a variety of models, with a total of 11 tasks and 7 models.
  arXiv  Detail & Related papers  (2024-10-04T04:48:33Z)
• Does Graph Prompt Work? A Data Operation Perspective with Theoretical Analysis [7.309233340654514]
  This paper introduces a theoretical framework that rigorously analyzes graph prompting from a data operation perspective. We provide a formal guarantee theorem, demonstrating graph prompts capacity to approximate graph transformation operators. We derive upper bounds on the error of these data operations by graph prompts for a single graph and extend this discussion to batches of graphs.
  arXiv  Detail & Related papers  (2024-10-02T15:07:13Z)
• Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models [88.4320775961431]
  We introduce ProGraph, a benchmark for large language models (LLMs) to process graphs. Our findings reveal that the performance of current LLMs is unsatisfactory, with the best model achieving only 36% accuracy. We propose LLM4Graph datasets, which include crawled documents and auto-generated codes based on 6 widely used graph libraries.
  arXiv  Detail & Related papers  (2024-09-29T11:38:45Z)
• GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability [28.713449421717193]
  We evaluate and enhance the graph understanding abilities of large language models (LLMs) In this paper, we propose a benchmark named GraphInstruct, which includes 21 classical graph reasoning tasks. We construct GraphLM through efficient instruction-tuning, which shows prominent graph understanding capability.
  arXiv  Detail & Related papers  (2024-03-07T13:36:08Z)
• Neural Scaling Laws on Graphs [54.435688297561015]
  We study neural scaling laws on graphs from both model and data perspectives. For model scaling, we investigate the phenomenon of scaling law collapse and identify overfitting as the potential reason. For data scaling, we suggest that the number of graphs can not effectively metric the graph data volume in scaling law since the sizes of different graphs are highly irregular.
  arXiv  Detail & Related papers  (2024-02-03T06:17:21Z)
• Graph Domain Adaptation: Challenges, Progress and Prospects [61.9048172631524]
  We propose graph domain adaptation as an effective knowledge-transfer paradigm across graphs. GDA introduces a bunch of task-related graphs as source graphs and adapts the knowledge learnt from source graphs to the target graphs. We outline the research status and challenges, propose a taxonomy, introduce the details of representative works, and discuss the prospects.
  arXiv  Detail & Related papers  (2024-02-01T02:44:32Z)
• Large Language Models on Graphs: A Comprehensive Survey [77.16803297418201]
  We provide a systematic review of scenarios and techniques related to large language models on graphs. We first summarize potential scenarios of adopting LLMs on graphs into three categories, namely pure graphs, text-attributed graphs, and text-paired graphs. We discuss the real-world applications of such methods and summarize open-source codes and benchmark datasets.
  arXiv  Detail & Related papers  (2023-12-05T14:14:27Z)
• Integrating Graphs with Large Language Models: Methods and Prospects [68.37584693537555]
  Large language models (LLMs) have emerged as frontrunners, showcasing unparalleled prowess in diverse applications. Merging the capabilities of LLMs with graph-structured data has been a topic of keen interest. This paper bifurcates such integrations into two predominant categories.
  arXiv  Detail & Related papers  (2023-10-09T07:59:34Z)
• Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications [38.83545631999851]
  We propose a framework of graph-aware language model pre-training on a large graph corpus. We conduct experiments on Amazon's real internal datasets and large public datasets.
  arXiv  Detail & Related papers  (2023-06-05T04:46:44Z)
• Can Language Models Solve Graph Problems in Natural Language? [51.28850846990929]
  Large language models (LLMs) are increasingly adopted for a variety of tasks with implicit graphical structures. We propose NLGraph, a benchmark of graph-based problem solving simulating in natural language.
  arXiv  Detail & Related papers  (2023-05-17T08:29:21Z)
• Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT [10.879701971582502]
  We aim to develop a large language model (LLM) with the reasoning ability on complex graph data. Inspired by the latest ChatGPT and Toolformer models, we propose the Graph-ToolFormer framework to teach LLMs themselves with prompts augmented by ChatGPT to use external graph reasoning API tools.
  arXiv  Detail & Related papers  (2023-04-10T05:25:54Z)
• State of the Art and Potentialities of Graph-level Learning [54.68482109186052]
  Graph-level learning has been applied to many tasks including comparison, regression, classification, and more. Traditional approaches to learning a set of graphs rely on hand-crafted features, such as substructures. Deep learning has helped graph-level learning adapt to the growing scale of graphs by extracting features automatically and encoding graphs into low-dimensional representations.
  arXiv  Detail & Related papers  (2023-01-14T09:15:49Z)
• Graph Learning and Its Advancements on Large Language Models: A Holistic Survey [37.01696685233113]
  This survey focuses on the most recent advancements in integrating graph learning with pre-trained language models. We provide a holistic review that analyzes current works from the perspective of graph structure, and discusses the latest applications, trends, and challenges in graph learning.
  arXiv  Detail & Related papers  (2022-12-17T22:05:07Z)
• Data Augmentation for Deep Graph Learning: A Survey [66.04015540536027]
  We first propose a taxonomy for graph data augmentation and then provide a structured review by categorizing the related work based on the augmented information modalities. Focusing on the two challenging problems in DGL (i.e., optimal graph learning and low-resource graph learning), we also discuss and review the existing learning paradigms which are based on graph data augmentation.
  arXiv  Detail & Related papers  (2022-02-16T18:30:33Z)
• Deep Learning for Learning Graph Representations [58.649784596090385]
  Mining graph data has become a popular research topic in computer science. The huge amount of network data has posed great challenges for efficient analysis. This motivates the advent of graph representation which maps the graph into a low-dimension vector space.
  arXiv  Detail & Related papers  (2020-01-02T02:13:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.