Graph Neural Multilevel Preconditioners for Iterative Solvers

• URL: http://arxiv.org/abs/2607.28456v1
• Date: Thu, 30 Jul 2026 16:19:41 GMT
• Title: Graph Neural Multilevel Preconditioners for Iterative Solvers
• Abstract summary: We propose a Graph Neural Multilevel Preconditioner (GMP) that adopts an AMG hierarchy as a structural prior and learns smoothing, restriction, and operators in a unified framework.<n>Our method targets general sparse systems and is instantiated as a drop-in preconditioner for standard Krylov solvers.
• Score: 4.805777654040642
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Solving large, sparse linear systems is a core task in scientific computing, and efficient iterative solvers rely critically on effective and robust preconditioning. While classical methods such as algebraic multigrid (AMG) are highly scalable, their robustness can degrade on indefinite or nonsymmetric systems where heuristics originally developed for elliptic PDEs are less reliable. Recently, Graph Neural Networks (GNNs) have emerged as data-driven preconditioners; yet, the practical impact of imposing an AMG-style hierarchy remains underexplored for general sparse matrices. In this work, we propose a Graph Neural Multilevel Preconditioner (GMP) that adopts an AMG hierarchy as a structural prior and learns smoothing, restriction, and interpolation operators in a unified framework. Our method targets general sparse systems and is instantiated as a drop-in preconditioner for standard Krylov solvers. On a benchmark of over 800 sparse matrices, we compare against classical AMG, single-level ILUT, and state-of-the-art GNN preconditioners, and characterize the regimes where multilevel graph neural preconditioning improves convergence or, conversely, introduces overhead relative to strong single-level baselines. These results highlight both the promise and the limitations of enforcing AMG-style multilevel structure in learned preconditioners for large-scale scientific simulations.

Related papers

• The Role of Gradient Modification in Heavy-Tailed Nonconvex Stochastic Min-Max Optimization [50.59470683896735]
  min-max optimization has attracted increasing attention due to its applications in modern machine learning.<n>We provide a comprehensive theoretical of gradient descent (SGDA)<n>We develop new clipping-free algorithms, i.e., Stoc-GDAM and Stoc-TRmax, and they both can achieve the optimal dependence on the target accuracy without using gradient clipping.
  arXiv  Detail & Related papers  (2026-09-05T12:49:28Z)
• Beyond the Aggregation Dilemma: Prior-Retaining Decoupled Learning for Multimodal Graphs [69.56407708794845]
  MultimodalAttributed Graph Learning (MAGL) integrates node attributes with structural topology via graph aggregation.<n>As pretrained encoders evolve into Large Foundation Models (LFMs), the landscape of MAGL fundamentally shifts.<n>Under high-confidence LFM priors, mandatory aggregation introduces topological intrinsic noise that overwhelms discriminative signals.<n>We propose SUPRA (Shared-Unique Prior-Retaining Architecture), a decoupled dual-pathway paradigm.<n> SUPRA processes modality-specific features through topology-agnostics while capturing structural synergy via a lightweight shared GNN, with auxiliary deep supervision counteracting gradient starvation.
  arXiv  Detail & Related papers  (2026-05-23T17:42:25Z)
• Position: Weight Space Should Be a First-Class Generative AI Modality [67.10702320971893]
  This paper argues that model checkpoints should be treated as a first-class data modality.<n>It argues that generative modeling in weight space should be standardized as a core machine learning primitive.<n>Our goal is to shift the community's default mindset from optimizing models per task to sampling models from learned weight distributions.
  arXiv  Detail & Related papers  (2026-05-18T16:38:26Z)
• Towards The Implicit Bias on Multiclass Separable Data Under Norm Constraints [3.421126849736858]
  We introduce NucGD, a geometry-aware geometry designed to enforce low rank structures through nuclear norm constraints.<n>To enable scalable training, we derive an efficient SVD-free update rule via asynchronous power iteration.<n>We empirically dissect the impact of gradient optimization dynamics, characterizing how varying levels of gradient noise induced by mini-batch sampling and momentum modulate the convergence toward the expected maximum margin solutions.
  arXiv  Detail & Related papers  (2026-03-24T05:51:50Z)
• HIPO: Instruction Hierarchy via Constrained Reinforcement Learning [57.40686733111483]
  textscHIPO is a novel alignment framework that formulates HIF as a Constrained Markov Decision Process.<n>textscHIPO elevates system prompts from mere input context to strict algorithmic boundaries.
  arXiv  Detail & Related papers  (2026-03-17T06:12:41Z)
• Deep Hierarchical Learning with Nested Subspace Networks [53.71337604556311]
  We propose Nested Subspace Networks (NSNs) for large neural networks.<n>NSNs enable a single model to be dynamically and granularly adjusted across a continuous spectrum of compute budgets.<n>We show that NSNs can be surgically applied to pre-trained LLMs and unlock a smooth and predictable compute-performance frontier.
  arXiv  Detail & Related papers  (2025-09-22T15:13:14Z)
• Matrix-free Neural Preconditioner for the Dirac Operator in Lattice Gauge Theory [13.32375374102012]
  We propose a framework, leveraging operator learning techniques, to construct linear maps as effective preconditioners.<n>In the context of the Schwinger model U(1) gauge theory in 1+1 spacetime dimensions, this preconditioning scheme effectively decreases the condition number of the linear systems.
  arXiv  Detail & Related papers  (2025-09-12T16:10:18Z)
• Towards Efficient General Feature Prediction in Masked Skeleton Modeling [59.46799426434277]
  We propose a novel General Feature Prediction framework (GFP) for efficient mask skeleton modeling.<n>Our key innovation is replacing conventional low-level reconstruction with high-level feature prediction that spans from local motion patterns to global semantic representations.
  arXiv  Detail & Related papers  (2025-09-03T18:05:02Z)
• Message-Passing GNNs Fail to Approximate Sparse Triangular Factorizations [3.868222899558346]
  Graph Neural Networks (GNNs) have been proposed as a tool for learning sparse matrix preconditioners.<n>This paper argues that message-passing GNNs are incapable of approximating sparse triangular factorizations.
  arXiv  Detail & Related papers  (2025-02-03T14:28:20Z)
• Harnessing Scale and Physics: A Multi-Graph Neural Operator Framework for PDEs on Arbitrary Geometries [9.954826518381468]
  This paper introduces the AMG method, a Multi-Graph neural operator approach designed for efficiently solving PDEs on Arbitrary geometries.<n> AMG leverages advanced graph-based techniques and dynamic attention mechanisms within a novel GraphFormer architecture.<n>We present a comprehensive evaluation of AMG across six benchmarks, demonstrating its consistent superiority over existing state-of-the-art models.
  arXiv  Detail & Related papers  (2024-11-18T12:35:03Z)
• Language Models as Zero-shot Lossless Gradient Compressors: Towards General Neural Parameter Prior Models [56.00251589760559]
  Large language models (LLMs) can act as gradient priors in a zero-shot setting.<n>We introduce LM-GC, a novel method that integrates LLMs with arithmetic coding.<n>Experiments indicate that LM-GC surpasses existing state-of-the-art lossless compression methods.
  arXiv  Detail & Related papers  (2024-09-26T13:38:33Z)
• Graph Neural Preconditioners for Iterative Solutions of Sparse Linear Systems [5.083469153675402]
  We propose using graph neural networks as a general-purpose preconditioner.<n>They show attractive performance for many problems and can be used when the mainstream preconditioners perform poorly.
  arXiv  Detail & Related papers  (2024-06-02T17:18:41Z)
• Learning from Linear Algebra: A Graph Neural Network Approach to Preconditioner Design for Conjugate Gradient Solvers [40.6591136324878]
  We train GNNs to obtain preconditioners that reduce the condition number of the system more significantly than classical preconditioners.<n>Our approach outperforms both classical and neural network-based methods for an important class of parametric partial differential equations.
  arXiv  Detail & Related papers  (2024-05-24T13:44:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.