Improved Predictive Performance and Interpretability for Mesomorphic Neural Networks Using Local Fidelity Regularization

• URL: http://arxiv.org/abs/2606.29951v1
• Date: Mon, 29 Jun 2026 08:26:20 GMT
• Title: Improved Predictive Performance and Interpretability for Mesomorphic Neural Networks Using Local Fidelity Regularization
• Abstract summary: Interpretable Mesomorphic Neural Networks (IMNs) offer a promising framework that combines the predictive power of deep neural networks with the interpretability of linear models.<n>We introduce Local Fidelity Regularization (LFR), a novel penalty term that prevents degenerate weight collapse.<n>This structural constraint guarantees faithful explanations and substantially improves the reliability of model interpretations.
• Score: 1.5647246115099227
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Interpretable Mesomorphic Neural Networks (IMNs) offer a promising framework that combines the predictive power of deep neural networks with the interpretability of linear models. However, the original formulation lacks safeguards to ensure that the learned interpretations are in fact reliable. In particular, the network is free to concentrate all explanatory variance into a single weight of the linear output layer, achieving strong predictive performance while producing interpretations that are largely meaningless. Paradoxically, the L1 penalty proposed to encourage sparse solutions exacerbates this problem by further incentivizing such degenerate configurations. To address this vulnerability, we introduce Local Fidelity Regularization (LFR), a novel penalty term that prevents degenerate weight collapse by aligning the linear output weights with local data variations. This structural constraint guarantees faithful explanations and substantially improves the reliability of model interpretations. Furthermore, empirical evaluations across the OpenML benchmark suite demonstrate that LFR does not compromise accuracy for explainability; rather, it achieved improved AUROC over the unregularized IMN. By yielding results highly competitive with state-of-the-art black-box models, LFR provides the dual benefit of reliable interpretability and superior predictive performance. Source code and usage instructions are available at https://github.com/hugohammer/LFR-IMN.git.

Related papers

• LiD-GLM: Lipschitz-constrained Deep Generalized Linear Models [1.389948527681755]
  Combination of traditional statistical models and neural network (NN) components is an intriguing approach to construct hybrid models.<n>We propose invertible residual neural networks (i-ResNets) to equip models with a flexible correction of distributional assumptions.<n>We develop specific inherent interpretation techniques for our model and enforce model identifiability through an adapted post-hocization.
  arXiv  Detail & Related papers  (2026-08-17T09:45:35Z)
• Interpretability Without Tradeoffs: Disentangling Polysemanticity At Equal Predictive Performance [58.49623467782041]
  ELUDe is a method for improving the interpretability of deep neural networks.<n>It breaks latent representations into clear, inspectable sub-units that behave like interpretable features.<n>It requires no explicit training, no labels, and can be applied to pretrained models.
  arXiv  Detail & Related papers  (2026-05-29T13:39:09Z)
• Learning a Generative Meta-Model of LLM Activations [75.30161960337892]
  We create "meta-models" that learn the distribution of a network's internal states.<n>Applying the meta-model's learned prior to steering interventions improves fluency, with larger gains as loss decreases.<n>These results suggest generative meta-models offer a scalable path toward interpretability without restrictive structural assumptions.
  arXiv  Detail & Related papers  (2026-02-06T18:59:56Z)
• FIRE: Multi-fidelity Regression with Distribution-conditioned In-context Learning using Tabular Foundation Models [3.8824066002669855]
  Multi-fidelity (MF) regression often operates in regimes of extreme data imbalance.<n>We introduce FIRE, a training-free MF framework.<n>Fire delivers a stronger performance-time trade-off than seven state-of-the-art GP-based or deep learning MF regression methods.
  arXiv  Detail & Related papers  (2026-01-29T22:29:58Z)
• Bayesian Neural Networks vs. Mixture Density Networks: Theoretical and Empirical Insights for Uncertainty-Aware Nonlinear Modeling [2.0797819204842036]
  We compare the approaches of Bayesian Neural Networks (BNNs) and Mixture Density Networks (MDNs) for uncertainty-aware nonlinear regression.<n>On the theoretical side, we derive convergence rates and error bounds under H"older smoothness conditions, showing that MDNs achieve faster Kullback-Leibler (KL) divergence convergence.<n>Our findings clarify the complementary strengths of posterior-based and likelihood-based probabilistic learning, offering guidance for uncertainty-aware modeling in nonlinear systems.
  arXiv  Detail & Related papers  (2025-10-28T22:00:30Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• An NMF-Based Building Block for Interpretable Neural Networks With Continual Learning [0.8158530638728501]
  Existing learning methods often struggle to balance interpretability and predictive performance. Our approach aims to strike a better balance between these two aspects through the use of a building block based on NMF.
  arXiv  Detail & Related papers  (2023-11-20T02:00:33Z)
• Achieving Constraints in Neural Networks: A Stochastic Augmented Lagrangian Approach [49.1574468325115]
  Regularizing Deep Neural Networks (DNNs) is essential for improving generalizability and preventing overfitting. We propose a novel approach to DNN regularization by framing the training process as a constrained optimization problem. We employ the Augmented Lagrangian (SAL) method to achieve a more flexible and efficient regularization mechanism.
  arXiv  Detail & Related papers  (2023-10-25T13:55:35Z)
• Outlier-robust neural network training: variation regularization meets trimmed loss to prevent functional breakdown [2.5628953713168685]
  We tackle the challenge of outlier-robust predictive modeling using highly expressive neural networks.<n>Our approach integrates two key components: (1) a transformed trimmed loss (TTL), and (2) higher-order variation regularization (HOVR), which imposes smoothness constraints on the prediction function.
  arXiv  Detail & Related papers  (2023-08-04T12:57:13Z)
• Lipschitz Continuity Retained Binary Neural Network [52.17734681659175]
  We introduce the Lipschitz continuity as the rigorous criteria to define the model robustness for BNN. We then propose to retain the Lipschitz continuity as a regularization term to improve the model robustness. Our experiments prove that our BNN-specific regularization method can effectively strengthen the robustness of BNN.
  arXiv  Detail & Related papers  (2022-07-13T22:55:04Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.