GraphGhost: Tracing Structures Behind Large Language Models

• URL: http://arxiv.org/abs/2510.08613v1
• Date: Tue, 07 Oct 2025 20:28:19 GMT
• Title: GraphGhost: Tracing Structures Behind Large Language Models
• Authors: Xinnan Dai, Kai Guo, Chung-Hsiang Lo, Shenglai Zeng, Jiayuan Ding, Dongsheng Luo, Subhabrata Mukherjee, Jiliang Tang,
• Abstract summary: We introduce GraphGhost, a unified framework that represents neuron activations and their signal propagation as graphs.<n>This graph-based perspective enables us to employ graph algorithms such as PageRank to characterize the properties of Large Language Models.<n>We show that edits to key neuron nodes can trigger reasoning collapse, altering both logical flow and semantic understanding.
• Score: 48.8586898059844
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) demonstrate remarkable reasoning capabilities, yet the structural mechanisms underlying these abilities remain under explored. In this work, we introduce GraphGhost, a unified framework that represents neuron activations and their signal propagation as graphs, explaining how LLMs capture structural semantics from sequential inputs and generate outputs through structurally consistent mechanisms. This graph-based perspective enables us to employ graph algorithms such as PageRank to characterize the properties of LLMs, revealing both shared and model-specific reasoning behaviors across diverse datasets. We further identify the activated neurons within GraphGhost and evaluate them through structural interventions, showing that edits to key neuron nodes can trigger reasoning collapse, altering both logical flow and semantic understanding. Together, these contributions position GraphGhost as a powerful tool for analyzing, intervening in, and ultimately understanding the structural foundations of reasoning in LLMs.

Related papers

• When Structure Doesn't Help: LLMs Do Not Read Text-Attributed Graphs as Effectively as We Expected [10.031229573133709]
  Large language models (LLMs) have excelled at understanding natural language and integrating cross-modal signals.<n>Recent work has explored how different strategies for encoding graph structure affect LLM performance on text-attributed graphs.<n>We show that explicit structural priors are often unnecessary and, in some cases, counterproductive when powerful language models are involved.
  arXiv  Detail & Related papers  (2025-11-20T19:34:58Z)
• Do We Really Need GNNs with Explicit Structural Modeling? MLPs Suffice for Language Model Representations [50.45261187796993]
  Graph Neural Networks (GNNs) fail to fully utilize structural information, whereas Multi-Layer Perceptrons (MLPs) exhibit a surprising ability in structure-aware tasks.<n>This paper introduces a comprehensive probing framework from an information-theoretic perspective.
  arXiv  Detail & Related papers  (2025-06-26T18:10:28Z)
• A Graph Perspective to Probe Structural Patterns of Knowledge in Large Language Models [52.52824699861226]
  Large language models have been extensively studied as neural knowledge bases for their knowledge access, editability, reasoning, and explainability.<n>We quantify the knowledge of LLMs at both the triplet and entity levels, and analyze how it relates to graph structural properties such as node degree.
  arXiv  Detail & Related papers  (2025-05-25T19:34:15Z)
• Attention Mechanisms Perspective: Exploring LLM Processing of Graph-Structured Data [10.907949155931474]
  We study how large language models (LLMs) process graph-structured data.<n>We uncover unique phenomena regarding how LLMs apply attention to graph-structured data.<n>We analyze these findings to improve the modeling of such data by LLMs.
  arXiv  Detail & Related papers  (2025-05-04T14:40:31Z)
• Scalability Matters: Overcoming Challenges in InstructGLM with Similarity-Degree-Based Sampling [1.2805157669888096]
  We propose SDM-InstructGLM, a novel instruction-tuned Graph Language Model (InstructGLM) framework that enhances scalability and efficiency without relying on GNNs.<n>Our method introduces a similarity-degree-based biased random walk mechanism, which selectively samples and encodes graph information based on node-feature similarity and degree centrality.<n>Our results demonstrate the feasibility of LLM-only graph processing, enabling scalable and interpretable Graph Language Models (GLMs) optimized through instruction-based fine-tuning.
  arXiv  Detail & Related papers  (2025-05-02T06:08:21Z)
• Graph Self-Supervised Learning with Learnable Structural and Positional Encodings [39.20899720477907]
  We introduce emphGenHopNet, a GNN framework that integrates a $k$-hop message-passing scheme.<n>We also propose a structural- and positional-aware GSSL framework that incorporates topological information throughout the learning process.<n>Our work significantly advances GSSL's capability in distinguishing graphs with similar local structures but different global topologies.
  arXiv  Detail & Related papers  (2025-02-22T14:10:06Z)
• Learning to Model Graph Structural Information on MLPs via Graph Structure Self-Contrasting [50.181824673039436]
  We propose a Graph Structure Self-Contrasting (GSSC) framework that learns graph structural information without message passing. The proposed framework is based purely on Multi-Layer Perceptrons (MLPs), where the structural information is only implicitly incorporated as prior knowledge. It first applies structural sparsification to remove potentially uninformative or noisy edges in the neighborhood, and then performs structural self-contrasting in the sparsified neighborhood to learn robust node representations.
  arXiv  Detail & Related papers  (2024-09-09T12:56:02Z)
• LangTopo: Aligning Language Descriptions of Graphs with Tokenized Topological Modeling [10.907949155931474]
  We introduce LangTopo, which aligns graph structure modeling with natural language understanding at the token level. We demonstrate the effectiveness of our proposed method on multiple datasets.
  arXiv  Detail & Related papers  (2024-06-19T06:20:22Z)
• Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs [57.052160123387104]
  We present the Disentangled Graph-Text Learner (DGTL) model, which is able to enhance the reasoning and predicting capabilities of LLMs for TAGs. Our proposed DGTL model incorporates graph structure information through tailored disentangled graph neural network (GNN) layers. Experimental evaluations demonstrate the effectiveness of the proposed DGTL model on achieving superior or comparable performance over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-10-27T14:00:04Z)
• Motif-based Graph Representation Learning with Application to Chemical Molecules [11.257235936629689]
  Existing graph neural networks offer limited ability to capture complex interactions within local structural contexts. We propose a new motif-based graph representation learning technique to better utilize local structural information. MCM builds a motif vocabulary in an unsupervised way and deploys a novel motif convolution operation to extract the local structural context.
  arXiv  Detail & Related papers  (2022-08-09T03:37:37Z)
• GraphOpt: Learning Optimization Models of Graph Formation [72.75384705298303]
  We propose an end-to-end framework that learns an implicit model of graph structure formation and discovers an underlying optimization mechanism. The learned objective can serve as an explanation for the observed graph properties, thereby lending itself to transfer across different graphs within a domain. GraphOpt poses link formation in graphs as a sequential decision-making process and solves it using maximum entropy inverse reinforcement learning algorithm.
  arXiv  Detail & Related papers  (2020-07-07T16:51:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.