Rethinking Graph Generalization through the Lens of Sharpness-Aware Minimization

• URL: http://arxiv.org/abs/2602.08855v1
• Date: Mon, 09 Feb 2026 16:20:07 GMT
• Title: Rethinking Graph Generalization through the Lens of Sharpness-Aware Minimization
• Abstract summary: We focus on a prevalent yet under-explored phenomenon in graph generalization, Minimal Shift Flip (MSF)<n>MSF occurs when test samples that slightly deviate from the training distribution are abruptly misclassified.<n>We propose an energy-driven generative augmentation framework (E2A) that leverages energy-guided latent perturbations to generate pseudo-OOD samples.
• Score: 18.577399823354806
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph Neural Networks (GNNs) have achieved remarkable success across various graph-based tasks but remain highly sensitive to distribution shifts. In this work, we focus on a prevalent yet under-explored phenomenon in graph generalization, Minimal Shift Flip (MSF),where test samples that slightly deviate from the training distribution are abruptly misclassified. To interpret this phenomenon, we revisit MSF through the lens of Sharpness-Aware Minimization (SAM), which characterizes the local stability and sharpness of the loss landscape while providing a theoretical foundation for modeling generalization error. To quantify loss sharpness, we introduce the concept of Local Robust Radius, measuring the smallest perturbation required to flip a prediction and establishing a theoretical link between local stability and generalization. Building on this perspective, we further observe a continual decrease in the robust radius during training, indicating weakened local stability and an increasingly sharp loss landscape that gives rise to MSF. To jointly solve the MSF phenomenon and the intractability of radius, we develop an energy-based formulation that is theoretically proven to be monotonically correlated with the robust radius, offering a tractable and principled objective for modeling flatness and stability. Building on these insights, we propose an energy-driven generative augmentation framework (E2A) that leverages energy-guided latent perturbations to generate pseudo-OOD samples and enhance model generalization. Extensive experiments across multiple benchmarks demonstrate that E2A consistently improves graph OOD generalization, outperforming state-of-the-art baselines.

Related papers

• NICE: Scale-Stable Perturbations for Graph Neural Network Explanations via Noise Corruption [67.57304101271096]
  Element-wise Masking (EM) suppresses edge-induced messages toward zero.<n>Noise Corruption (NC) perturbs each message through matched-norm random-direction corruption.<n>NICE, a Noise Corruption-based explanation framework, learns a Restoration Boundary (SRB) under NC-induced uncertainty.
  arXiv  Detail & Related papers  (2026-08-17T03:08:14Z)
• Physically-Constrained Mamba-SDE for Remaining Useful Life Prediction under Irregular Observations [26.807602340696047]
  We propose PC-MambaSDE, a unified continuous-time framework for robust RUL prediction under irregular observations.<n>Specifically, we design a Physics-Guided Latent SDE with parametrically rectified hybrid drift.<n>We also develop a Hybrid Irregularity Generation Scheme that simulates realistic industrial imperfections.
  arXiv  Detail & Related papers  (2026-06-01T08:37:38Z)
• Where to Refine, When to Stop: Rethinking Redundancy via Latent Discrepancy for Efficient Visual Autoregressive Generation [50.139984798361375]
  We propose a training-free framework that removes redundancy via latent discrepancy.<n> LD-Pruning substantially reduces inference while maintaining high generation quality, achieving up to 2.35x speedup on Infinity-8B.
  arXiv  Detail & Related papers  (2026-05-29T19:34:39Z)
• A Distributional View for Visual Mechanistic Interpretability: KL-Minimal Soft-Constraint Principle [77.72852487795772]
  We propose a theoretical distributional view for visual mechanistic interpretability.<n>This view models the influence of a feature activation on the natural image distribution.<n>We propose a model with a KL-minimal soft-constraint principle for visual MI that theoretically balances interpretability and faithfulness.
  arXiv  Detail & Related papers  (2026-05-17T15:31:42Z)
• Mitigating Error Amplification in Fast Adversarial Training [58.74042726356826]
  Fast Adversarial Training (FAT) has proven effective in enhancing model robustness by encouraging networks to learn perturbation-invariant representations.<n>FAT often suffers from catastrophic overfitting (CO), where the model overfits to the training attack and fails to generalize to unseen ones.<n>We propose a Distribution-aware Dynamic Guidance (DDG) strategy that dynamically adjusts both the perturbation budget and supervision signal.
  arXiv  Detail & Related papers  (2026-04-27T11:23:18Z)
• DOC-GS: Dual-Domain Observation and Calibration for Reliable Sparse-View Gaussian Splatting [80.43237927269575]
  In this paper, we revisit sparse-view 3DGS reconstruction from a new perspective.<n>We identify the core challenge as the unobservability of Gaussian primitive reliability.<n>Motivated by this observation, we propose a unified Dual-domain Observation and geometrics in rendered images framework.
  arXiv  Detail & Related papers  (2026-04-08T07:01:24Z)
• The Inductive Bias of Convolutional Neural Networks: Locality and Weight Sharing Reshape Implicit Regularization [57.37943479039033]
  We study how architectural inductive bias reshapes the implicit regularization induced by the edge-of-stability phenomenon in gradient descent.<n>We show that locality and weight sharing fundamentally change this picture.
  arXiv  Detail & Related papers  (2026-03-05T04:50:51Z)
• Gradients Must Earn Their Influence: Unifying SFT with Generalized Entropic Objectives [22.29000001610794]
  Standard negative log-likelihood for Supervised Fine-Tuning (SFT) applies uniform token-level weighting.<n>This rigidity creates a two-fold failure mode: (i) overemphasizing low-probability targets can amplify gradients on noisy supervision and disrupt robust priors, and (ii) uniform weighting provides weak sharpening when the model is already confident.<n>Existing methods fail to resolve the resulting plasticity--stability dilemma, often suppressing necessary learning signals alongside harmful ones.<n>We introduce Dynamic Entropy Fine-Tuning (DEFT), a parameter-free objective that modulates the
  arXiv  Detail & Related papers  (2026-02-11T22:56:43Z)
• Mitigating Reward Hacking in RLHF via Bayesian Non-negative Reward Modeling [49.41422138354821]
  We propose a principled reward modeling framework that integrates non-negative factor analysis into the Bradley-Terry preference model.<n>BNRM represents rewards through a sparse, non-negative latent factor generative process.<n>We show that BNRM substantially mitigates reward over-optimization, improves robustness under distribution shifts, and yields more interpretable reward decompositions than strong baselines.
  arXiv  Detail & Related papers  (2026-02-11T08:14:11Z)
• Understanding Degradation with Vision Language Model [56.09241449206817]
  Understanding visual degradations is a critical yet challenging problem in computer vision.<n>We introduce DU-VLM, a multimodal chain-of-thought model trained with supervised fine-tuning and reinforcement learning.<n>We also introduce textbfDU-110k, a large-scale dataset comprising 110,000 clean-degraded pairs with grounded physical annotations.
  arXiv  Detail & Related papers  (2026-02-04T13:51:15Z)
• Training Instabilities Induce Flatness Bias in Gradient Descent [6.628332915214955]
  Modern deep networks often achieve their best performance beyond a stability threshold.<n>We show that training instabilities induce an implicit bias in GD, driving parameters toward flatter regions of the loss landscape.<n>We also show that restoring instabilities in Adam further improves generalization.
  arXiv  Detail & Related papers  (2025-11-16T11:26:25Z)
• On the Stability of Graph Convolutional Neural Networks: A Probabilistic Perspective [21.0483487978294]
  We study how perturbations in the graph topology affect GCNN outputs and propose a novel formulation for analyzing model stability.<n>Unlike prior studies that focus only on worst-case perturbations, our distribution-aware formulation characterizes output perturbations across a broad range of input data.
  arXiv  Detail & Related papers  (2025-06-01T23:17:19Z)
• ACMamba: Fast Unsupervised Anomaly Detection via An Asymmetrical Consensus State Space Model [51.83639270669481]
  Unsupervised anomaly detection in hyperspectral images (HSI) aims to detect unknown targets from backgrounds.<n>HSI studies are hindered by steep computational costs due to the high-dimensional property of HSI and dense sampling-based training paradigm.<n>We propose an Asymmetrical Consensus State Space Model (ACMamba) to significantly reduce computational costs without compromising accuracy.
  arXiv  Detail & Related papers  (2025-04-16T05:33:42Z)
• Understanding Flatness in Generative Models: Its Role and Benefits [9.775257597631244]
  Flat minima, known to enhance robustness in supervised learning, remain largely unexplored in generative models.<n>We establish a theoretical claim that flatter minima improve robustness against perturbations in target prior distributions.<n>We demonstrate that flat minima in diffusion models indeed improve not only generative performance but also robustness.
  arXiv  Detail & Related papers  (2025-03-14T04:38:53Z)
• Interpreting Global Perturbation Robustness of Image Models using Axiomatic Spectral Importance Decomposition [0.0]
  Perturbation robustness evaluates the vulnerabilities of models, arising from a variety of perturbations, such as data corruptions and adversarial attacks.<n>We present a model-agnostic, global mechanistic interpretability method to interpret the perturbation robustness of image models.
  arXiv  Detail & Related papers  (2024-08-02T09:35:06Z)
• Unmasking Bias in Diffusion Model Training [40.90066994983719]
  Denoising diffusion models have emerged as a dominant approach for image generation. They still suffer from slow convergence in training and color shift issues in sampling. In this paper, we identify that these obstacles can be largely attributed to bias and suboptimality inherent in the default training paradigm.
  arXiv  Detail & Related papers  (2023-10-12T16:04:41Z)
• Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reduction [49.66486092259376]
  The mean-field Langevin dynamics (MFLD) is a nonlinear generalization of the Langevin dynamics that incorporates a distribution-dependent drift. Recent works have shown that MFLD globally minimizes an entropy-regularized convex functional in the space of measures. We provide a framework to prove a uniform-in-time propagation of chaos for MFLD that takes into account the errors due to finite-particle approximation, time-discretization, and gradient approximation.
  arXiv  Detail & Related papers  (2023-06-12T16:28:11Z)
• Handling Distribution Shifts on Graphs: An Invariance Perspective [78.31180235269035]
  We formulate the OOD problem on graphs and develop a new invariant learning approach, Explore-to-Extrapolate Risk Minimization (EERM) EERM resorts to multiple context explorers that are adversarially trained to maximize the variance of risks from multiple virtual environments. We prove the validity of our method by theoretically showing its guarantee of a valid OOD solution.
  arXiv  Detail & Related papers  (2022-02-05T02:31:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.