Generalization Boundaries of Fine-Tuned Small Language Models for Graph Structural Inference

• URL: http://arxiv.org/abs/2604.18092v1
• Date: Mon, 20 Apr 2026 11:09:12 GMT
• Title: Generalization Boundaries of Fine-Tuned Small Language Models for Graph Structural Inference
• Abstract summary: We investigate the boundaries of structural inference in fine-tuned small language models.<n>We assess domain-learning capability on real-world graph benchmarks.
• Score: 0.48302896549293584
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Small language models fine-tuned for graph property estimation have demonstrated strong in-distribution performance, yet their generalization capabilities beyond training conditions remain poorly understood. In this work, we systematically investigate the boundaries of structural inference in fine-tuned small language models along two generalization axes - graph size and graph family distribution - and assess domain-learning capability on real-world graph benchmarks. Using a controlled experimental setup with three instruction-tuned models in the 3-4B parameter class and two graph serialization formats, we evaluate performance on graphs substantially larger than the training range and across held-out random graph families. Our results show that fine-tuned models maintain strong ordinal consistency across structurally distinct graph families and continue to rank graphs by structural properties on inputs substantially larger than those seen during training, with distinct architecture-specific degradation profiles. These findings delineate where fine-tuned small language models generalize reliably, providing empirical grounding for their use in graph-based reasoning tasks.

Related papers

• Model Graph Inductive Learning for Knowledge Graph Completion [0.0]
  Link prediction in knowledge graphs depends on the quality of learned embeddings for entities and relations.<n>We introduce Model Graph Inductive Learning (bftextMGIL), a framework that constructs a model graph by clustering entities.<n>A GNN is then applied to this model graph to produce embeddings that capture the global view of the knowledge graph.
  arXiv  Detail & Related papers  (2026-06-15T10:13:27Z)
• TinyGraphEstimator: Adapting Lightweight Language Models for Graph Structure Inference [0.48302896549293584]
  This paper investigates whether compact transformer-based language models can infer graph-theoretic parameters directly from graph representations.<n>We introduce the TinyGraphEstimator dataset - a collection of connected graphs generated from multiple random graph models.<n>We evaluate several small open models on their ability to predict key graph parameters such as density, clustering, and chromatic number.
  arXiv  Detail & Related papers  (2025-10-09T20:47:07Z)
• GraphUniverse: Enabling Systematic Evaluation of Inductive Generalization [8.95975918626961]
  GraphUniverse is a framework for generating entire families of graphs.<n>It enables the first systematic evaluation of inductive generalization at scale.<n>Our core innovation is the generation of graphs with persistent semantic communities.
  arXiv  Detail & Related papers  (2025-09-25T12:46:01Z)
• Revisiting Graph Neural Networks on Graph-level Tasks: Comprehensive Experiments, Analysis, and Improvements [54.006506479865344]
  We propose a unified evaluation framework for graph-level Graph Neural Networks (GNNs)<n>This framework provides a standardized setting to evaluate GNNs across diverse datasets.<n>We also propose a novel GNN model with enhanced expressivity and generalization capabilities.
  arXiv  Detail & Related papers  (2025-01-01T08:48:53Z)
• Generative Modelling of Structurally Constrained Graphs [44.28728853270471]
  We present ConStruct, a novel framework that enables graph diffusion models to incorporate hard constraints on specific properties, such as planarity or acyclicity.<n>Our approach ensures that the sampled graphs remain within the domain of graphs that satisfy the specified property throughout the entire trajectory in both the forward and reverse processes.
  arXiv  Detail & Related papers  (2024-06-25T07:54:32Z)
• GraphGLOW: Universal and Generalizable Structure Learning for Graph Neural Networks [72.01829954658889]
  This paper introduces the mathematical definition of this novel problem setting. We devise a general framework that coordinates a single graph-shared structure learner and multiple graph-specific GNNs. The well-trained structure learner can directly produce adaptive structures for unseen target graphs without any fine-tuning.
  arXiv  Detail & Related papers  (2023-06-20T03:33:22Z)
• GraphDCA -- a Framework for Node Distribution Comparison in Real and Synthetic Graphs [72.51835626235368]
  We argue that when comparing two graphs, the distribution of node structural features is more informative than global graph statistics. We present GraphDCA - a framework for evaluating similarity between graphs based on the alignment of their respective node representation sets.
  arXiv  Detail & Related papers  (2022-02-08T14:19:19Z)
• Generating the Graph Gestalt: Kernel-Regularized Graph Representation Learning [47.506013386710954]
  A complete scientific understanding of graph data should address both global and local structure. We propose a joint model for both as complementary objectives in a graph VAE framework. Our experiments demonstrate a significant improvement in the realism of the generated graph structures, typically by 1-2 orders of magnitude of graph structure metrics.
  arXiv  Detail & Related papers  (2021-06-29T10:48:28Z)
• Model-Agnostic Graph Regularization for Few-Shot Learning [60.64531995451357]
  We present a comprehensive study on graph embedded few-shot learning. We introduce a graph regularization approach that allows a deeper understanding of the impact of incorporating graph information between labels. Our approach improves the performance of strong base learners by up to 2% on Mini-ImageNet and 6.7% on ImageNet-FS.
  arXiv  Detail & Related papers  (2021-02-14T05:28:13Z)
• Graph Information Bottleneck [77.21967740646784]
  Graph Neural Networks (GNNs) provide an expressive way to fuse information from network structure and node features. Inheriting from the general Information Bottleneck (IB), GIB aims to learn the minimal sufficient representation for a given task. We show that our proposed models are more robust than state-of-the-art graph defense models.
  arXiv  Detail & Related papers  (2020-10-24T07:13:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.