PRISM: A Geometric Risk Bound that Decomposes Drift into Scale, Shape, and Head

• URL: http://arxiv.org/abs/2605.11608v1
• Date: Tue, 12 May 2026 06:40:34 GMT
• Title: PRISM: A Geometric Risk Bound that Decomposes Drift into Scale, Shape, and Head
• Abstract summary: We propose PRISM, which exploits the linear output head of LLMs and the empirically near-isometric structure of their backbones.<n>The bound is calibrated for variant ranking and decomposes drift into three independently measurable axes.<n> PRISM ranks variants with mean Spearman correlations of 0.820 for post-training quantization and 0.831 for LoRA forgetting.
• Score: 14.880821907124451
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Comparing post-training LLM variants, such as quantized, LoRA-adapted, and distilled models, requires a diagnostic that identifies how a variant has drifted, not only whether it has degraded. Existing similarity scores such as CKA and SVCCA can flag degradation, but they do not directly link representation drift to risk or mechanism. We propose PRISM, Proxy Risk Inference via Structural Mapping, which exploits the linear output head of LLMs and the empirically near-isometric structure of their backbones to derive a closed-form upper bound on the cross-entropy risk gap between a target model and a post-training variant. The bound is calibrated for variant ranking and decomposes drift into three independently measurable axes: scale mismatch, shape mismatch, and head divergence. Each axis corresponds to a distinct failure mode, including shape distortion under low-bit quantization, scale separability under LoRA forgetting, and head divergence under GGUF k-quantization. As a result, the dominant axis suggests a remediation direction rather than merely raising a degradation flag. Because the shape term is differentiable, the same geometry can also serve as a training-time regularizer against catastrophic forgetting. Across two model families and five benchmarks, PRISM ranks variants with mean Spearman correlations of 0.820 for post-training quantization and 0.831 for LoRA forgetting, and its axis-guided shape regularizer outperforms experience replay in aggregate at mitigating downstream forgetting.

Related papers

• One Step, One Lead: Mitigating Higher-Order Interference in Multi-Domain Reinforcement Learning via Cross-Step Control [62.82372407840088]
  We show that same-point domain gradients may remain nearly even when consecutive realized updates partially reverse one another in output space.<n>We propose OSOL, which designates a focus domain at each iteration, to rank token-level rebound risk.<n>OSOL reaches a domain-macro average of 0.4822, improving by 5.7% over the strongest compared baseline, without explicit higher-order differentiation.
  arXiv  Detail & Related papers  (2026-09-06T08:32:59Z)
• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• When Does Consensus Mean Correctness? Measuring the Agreement-Accuracy Coupling with Semantics-Preserving Re-Rendering [0.8594140167290097]
  A model's agreement across perturbed inputs is used as a label-free reliability signal.<n>Rerendering beats resampling on both accuracy and reliability.<n>Fine-tuning on the model's own cross-render consensus inverts.
  arXiv  Detail & Related papers  (2026-08-06T07:08:05Z)
• Convex-Hull-Neighborhood Smooth Dual Generalization: Controlling Local Correction Propagation in Offline RL [7.803284875836737]
  bootstrapping can amplify estimation errors in offline reinforcement learning.<n>We propose Convex Hull Neighborhood Smooth Dual Generalization, which expresses the Bellman backup as an in-sample value target plus a CHN-local correction.<n> Experiments on Gym-MuJoCo and AntMaze show strong aggregate performance and stable value estimation.
  arXiv  Detail & Related papers  (2026-08-04T04:30:02Z)
• Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems [47.027290803102666]
  Normal behaviour is the union of many imbalanced, curved, thin-fringed operating regimes.<n>We model the normal law with a jointly learned latent representation plus explicit Gaussian-mixture mode clustering.<n>We evaluate under a deliberately fair protocol: raw point-wise metrics with no point adjustment, a trivial-detector difficulty split, prevalence-matched F1, and train-normal-only calibration.
  arXiv  Detail & Related papers  (2026-07-07T10:10:00Z)
• CouCE: A Unified Causal Framework for Debiased Deep Metric Learning [40.50126673028885]
  Counterfactual Causal Embedding (CouCE) is a unified causal framework that explicitly models and neutralizes both confounders.<n>CouCE consistently achieves state-of-the-art performance, providing a principled and robust solution for debiased DML.
  arXiv  Detail & Related papers  (2026-06-29T14:28:22Z)
• GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs [5.018957704056544]
  Activation steering controls model behavior by modifying intermediate hidden states at inference time without retraining.<n>We show that when multiple semantic directions are superposed without constraints, the model collapses.<n>We propose GEMS, a training-free method that maps each source to a corresponding geometric constraint.
  arXiv  Detail & Related papers  (2026-06-18T08:43:33Z)
• How Useful is Causal Invariance for Domain Adaptation in Finite-Sample Settings? [58.740078141879984]
  Machine learning models often degrade when they are deployed on a target distribution that differs from the source distributions they were trained on.<n>Recent work in causality-based domain generalization has shown how shared causal structure can induce invariant predictors.<n>In this paper, we study when full or partial causal knowledge can provably improve supervised domain adaptation.
  arXiv  Detail & Related papers  (2026-06-10T21:07:49Z)
• DISA: Offline Importance Sampling for Distribution-Matching LLM-RL [56.9445657766829]
  We introduce DISA, which moves this calibration problem outside the RL loop.<n> DISA draws proposal trajectories offline, estimates the partition function via importance sampling, and freezes the resulting partition-function estimate.<n>On two open-weight backbones across six math and three code benchmarks, DISA matches or exceeds the online-coupled distribution-matching baseline Flow.
  arXiv  Detail & Related papers  (2026-05-17T07:14:44Z)
• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• ATLAS: Constitution-Conditioned Latent Geometry and Redistribution Across Language Models and Neural Perturbation Data [0.0]
  Constitution-conditioned post-training can be analysed as a structured perturbation of a model's learned representational geometry.<n>We introduce ATLAS, a geometry-first program that traces constitution-induced hidden-state structure across charts, models, and substrates.
  arXiv  Detail & Related papers  (2026-04-19T23:26:02Z)
• Information-Geometric Decomposition of Generalization Error in Unsupervised Learning [0.0]
  We decompose the Kullback--Leibler generalization error (GE) of unsupervised learning into three non-negative components: model error, data bias, and variance.<n>The decomposition is exact for any e-flat model class.
  arXiv  Detail & Related papers  (2026-04-14T06:23:18Z)
• Convergence of Byzantine-Resilient Gradient Tracking via Probabilistic Edge Dropout [1.3902537392439644]
  We study distributed optimization over networks with Byzantine agents that may send arbitrary adversarial messages.<n>We propose emphGradient Tracking with Probabilistic Edge Dropout and Leaky Integration (GT-PD-L)<n>GT-PD-L outperforms coordinate-wise mean by up to 4.3 percentage points under stealth attacks.
  arXiv  Detail & Related papers  (2026-04-01T03:55:42Z)
• Le Cam Distortion: A Decision-Theoretic Framework for Robust Transfer Learning [0.0]
  We introduce Le Cam Distortion as a rigorous upper bound for transfer risk conditional on simulability.<n>Our framework enables transfer without source degradation by learning a kernel that simulates the target from the source.<n>Le Cam Distortion provides the first principled framework for risk-controlled transfer learning in domains where negative transfer is unacceptable.
  arXiv  Detail & Related papers  (2025-12-29T17:21:44Z)
• Latent Sculpting for Zero-Shot Generalization: A Manifold Learning Approach to Out-of-Distribution Anomaly Detection [2.8547732086436306]
  A fundamental limitation of supervised deep learning is "Generalization Collapse"<n>We propose Latent Sculpting, a hierarchical two-stage representation learning framework.<n>We report an 88.89% detection rate on "Infiltration" scenarios.
  arXiv  Detail & Related papers  (2025-12-19T11:37:02Z)
• Self-Supervised Contrastive Learning is Approximately Supervised Contrastive Learning [48.11265601808718]
  We show that standard self-supervised contrastive learning objectives implicitly approximate a supervised variant we call the negatives-only supervised contrastive loss (NSCL)<n>We prove that the gap between the CL and NSCL losses vanishes as the number of semantic classes increases, under a bound that is both label-agnostic and architecture-independent.
  arXiv  Detail & Related papers  (2025-06-04T19:43:36Z)
• Distributionally Robust Models with Parametric Likelihood Ratios [123.05074253513935]
  Three simple ideas allow us to train models with DRO using a broader class of parametric likelihood ratios. We find that models trained with the resulting parametric adversaries are consistently more robust to subpopulation shifts when compared to other DRO approaches.
  arXiv  Detail & Related papers  (2022-04-13T12:43:12Z)
• Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivity [55.29408396918968]
  We study a family of loss functions named label-distributionally robust (LDR) losses for multi-class classification. Our contributions include both consistency and robustness by establishing top-$k$ consistency of LDR losses for multi-class classification. We propose a new adaptive LDR loss that automatically adapts the individualized temperature parameter to the noise degree of class label of each instance.
  arXiv  Detail & Related papers  (2021-12-30T00:27:30Z)
• Shaping Deep Feature Space towards Gaussian Mixture for Visual Classification [74.48695037007306]
  We propose a Gaussian mixture (GM) loss function for deep neural networks for visual classification. With a classification margin and a likelihood regularization, the GM loss facilitates both high classification performance and accurate modeling of the feature distribution. The proposed model can be implemented easily and efficiently without using extra trainable parameters.
  arXiv  Detail & Related papers  (2020-11-18T03:32:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.