An Information-Geometric Framework for Stability Analysis of Large Language Models under Entropic Stress

• URL: http://arxiv.org/abs/2604.24076v1
• Date: Mon, 27 Apr 2026 06:00:29 GMT
• Title: An Information-Geometric Framework for Stability Analysis of Large Language Models under Entropic Stress
• Abstract summary: This study proposes a thermodynamic inspired modeling framework for analyzing the stability of large language models (LLMs)<n>Rather than interpreting quantities as physical variables, the formulation is intended as an interpretable abstraction that captures how internal structure may modulate the impact of disorder on model behavior.<n>The contribution of this work is a compact and interpretable modeling perspective that connects uncertainty, performance, and internal structure within a unied evaluation lens.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: As large language models (LLMs) are increasingly deployed in high-stakes and operational settings, evaluation strategies based solely on aggregate accuracy are often insucient to characterize system reliability. This study proposes a thermodynamic inspired modeling framework for analyzing the stability of LLM outputs under conditions of uncertainty and perturbation. The framework introduces a composite stability score that integrates task utility, entropy as a measure of external uncertainty, and two internal structural proxies: internal integration and aligned reective capacity. Rather than interpreting these quantities as physical variables, the formulation is intended as an interpretable abstraction that captures how internal structure may modulate the impact of disorder on model behavior. Using the IST-20 benchmarking protocol and associated metadata, we analyze 80 modelscenario observations across four contemporary LLMs. The proposed formulation consistently yields higher stability scores than a reduced utilityentropy baseline, with a mean improvement of 0.0299 (95% CI: 0.02470.0351). The observed gain is more pronounced under higher entropy conditions, suggesting that the framework captures a form of nonlinear attenuation of uncertainty. We do not claim a fundamental physical law or a complete theory of machine ethics. Instead, the contribution of this work is a compact and interpretable modeling perspective that connects uncertainty, performance, and internal structure within a unied evaluation lens. The framework is intended to complement existing benchmarking approaches and to support ongoing discussions in AI safety, reliability, and governance.

Related papers

• Structured Neural Chaos: An Adaptive Surrogate Modeling Framework for Functional Uncertainty Quantification and Global Sensitivity Analysis [3.4686904619645866]
  We introduce the Structured Neural Chaos (sNC) expansion as a surrogate modeling framework for variance-based global sensitivity analysis.<n>The proposed framework retains the interpretability of structured decompositions while leveraging the power of neural networks.
  arXiv  Detail & Related papers  (2026-07-31T00:03:07Z)
• Latent Performance Profiling of Large Language Models [47.009623327601226]
  We introduce Latent Performance Profiling (LPP) -- a framework that derives task-agnostic diagnostics from hidden activations and output distributions.<n>Unlike static accuracy scores, LPP provides stable, architecture-sensitive signatures across models of similar size.<n>We show that models with similar benchmark scores can exhibit contrasting latent profiles, such as differences in entropy or adaptability.
  arXiv  Detail & Related papers  (2026-05-28T14:41:26Z)
• Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representations [36.24681446361556]
  We propose a conformal framework for question answering that uses internal representations rather than output-facing statistics.<n>We show that our method achieves a better validity--efficiency trade-off than strong text-level baselines.<n>These results suggest that internal representations can provide more informative conformal scores when surface-level uncertainty is unstable under distribution shift.
  arXiv  Detail & Related papers  (2026-04-17T16:28:31Z)
• Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetry [19.669339224904277]
  A vital risk is intrinsic deception, wherein models strategically mislead users to achieve their own objectives.<n>Existing alignment approaches based on chain-of-thought (CoT) monitoring supervise explicit reasoning traces.<n>We propose Stability Asymmetry Regularization (SAR), a novel alignment objective that penalizes this distributional asymmetry during reinforcement learning.
  arXiv  Detail & Related papers  (2026-03-27T09:47:57Z)
• Equivariant Evidential Deep Learning for Interatomic Potentials [55.6997213490859]
  Uncertainty quantification is critical for assessing the reliability of machine learning interatomic potentials in molecular dynamics simulations.<n>Existing UQ approaches for MLIPs are often limited by high computational cost or suboptimal performance.<n>We propose textitEquivariant Evidential Deep Learning for Interatomic Potentials ($texte2$IP), a backbone-agnostic framework that models atomic forces and their uncertainty jointly.
  arXiv  Detail & Related papers  (2026-02-11T02:00:25Z)
• Not All Preferences Are Created Equal: Stability-Aware and Gradient-Efficient Alignment for Reasoning Models [52.48582333951919]
  We propose a dynamic framework designed to enhance alignment reliability by maximizing the Signal-to-Noise Ratio of policy updates.<n>SAGE (Stability-Aware Gradient Efficiency) integrates a coarse-grained curriculum mechanism that refreshes candidate pools based on model competence.<n> Experiments on multiple mathematical reasoning benchmarks demonstrate that SAGE significantly accelerates convergence and outperforms static baselines.
  arXiv  Detail & Related papers  (2026-02-01T12:56:10Z)
• Trust in One Round: Confidence Estimation for Large Language Models via Structural Signals [13.89434979851652]
  Large language models (LLMs) are increasingly deployed in domains where errors carry high social, scientific, or safety costs.<n>We present Structural Confidence, a single-pass, model-agnostic framework that enhances output correctness prediction.
  arXiv  Detail & Related papers  (2026-02-01T02:35:59Z)
• Stability as a Liability:Systematic Breakdown of Linguistic Structure in LLMs [5.96875296117642]
  We show that stable parameter trajectories lead stationary solutions to minimize the forward KL divergence to the empirical distribution.<n>We empirically validate this effect using a controlled feedback-based training framework.<n>It indicates that optimization stability and generative expressivity are not inherently aligned, and that stability alone is an insufficient indicator of generative quality.
  arXiv  Detail & Related papers  (2026-01-26T15:34:50Z)
• MaP: A Unified Framework for Reliable Evaluation of Pre-training Dynamics [72.00014675808228]
  Instability in Large Language Models evaluation process obscures true learning dynamics.<n>We introduce textbfMaP, a framework that integrates underlineMerging underlineand the underlinePass@k metric.<n>Experiments show that MaP yields significantly smoother performance curves, reduces inter-run variance, and ensures more consistent rankings.
  arXiv  Detail & Related papers  (2025-10-10T11:40:27Z)
• Mechanistic Interpretability as Statistical Estimation: A Variance Analysis of EAP-IG [10.620784202716404]
  We argue that interpretability methods, such as circuit discovery, should be viewed as statistical estimators.<n>We present a systematic stability analysis of a state-of-the-art circuit discovery method: EAP-IG.
  arXiv  Detail & Related papers  (2025-10-01T12:55:34Z)
• Tuning for Trustworthiness -- Balancing Performance and Explanation Consistency in Neural Network Optimization [49.567092222782435]
  We introduce the novel concept of XAI consistency, defined as the agreement among different feature attribution methods.<n>We create a multi-objective optimization framework that balances predictive performance with explanation.<n>Our research provides a foundation for future investigations into whether models from the trade-off zone-balancing performance loss and XAI consistency-exhibit greater robustness.
  arXiv  Detail & Related papers  (2025-05-12T13:19:14Z)
• Quantifying Robustness: A Benchmarking Framework for Deep Learning Forecasting in Cyber-Physical Systems [44.61435605872856]
  We introduce a practical robustness definition grounded in distributional robustness, explicitly tailored to industrial CPS.<n>Our framework simulates realistic disturbances, such as sensor drift, noise and irregular sampling, enabling thorough robustness analyses of forecasting models.
  arXiv  Detail & Related papers  (2025-04-04T14:50:48Z)
• Enhancing LLM Reliability via Explicit Knowledge Boundary Modeling [41.19330514054401]
  Large language models (LLMs) are prone to hallucination stemming from misaligned self-awareness.<n>We propose the Explicit Knowledge Boundary Modeling framework to integrate fast and slow reasoning systems to harmonize reliability and usability.
  arXiv  Detail & Related papers  (2025-03-04T03:16:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.