Evaluating the stability of model explanations in instance-dependent cost-sensitive credit scoring

• URL: http://arxiv.org/abs/2509.01409v1
• Date: Mon, 01 Sep 2025 12:05:59 GMT
• Title: Evaluating the stability of model explanations in instance-dependent cost-sensitive credit scoring
• Authors: Matteo Ballegeer, Matthias Bogaert, Dries F. Benoit,
• Abstract summary: Instance-dependent cost-sensitive (IDCS) classifiers offer a promising approach to improving cost-efficiency in credit scoring.<n>Impact of such loss functions on the stability of model explanations remains unexplored in literature.
• Score: 0.45880283710344055
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Instance-dependent cost-sensitive (IDCS) classifiers offer a promising approach to improving cost-efficiency in credit scoring by tailoring loss functions to instance-specific costs. However, the impact of such loss functions on the stability of model explanations remains unexplored in literature, despite increasing regulatory demands for transparency. This study addresses this gap by evaluating the stability of Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP) when applied to IDCS models. Using four publicly available credit scoring datasets, we first assess the discriminatory power and cost-efficiency of IDCS classifiers, introducing a novel metric to enhance cross-dataset comparability. We then investigate the stability of SHAP and LIME feature importance rankings under varying degrees of class imbalance through controlled resampling. Our results reveal that while IDCS classifiers improve cost-efficiency, they produce significantly less stable explanations compared to traditional models, particularly as class imbalance increases, highlighting a critical trade-off between cost optimization and interpretability in credit scoring. Amid increasing regulatory scrutiny on explainability, this research underscores the pressing need to address stability issues in IDCS classifiers to ensure that their cost advantages are not undermined by unstable or untrustworthy explanations.

Related papers

• Measuring the Fragility of Trust: Devising Credibility Index via Explanation Stability (CIES) for Business Decision Support Systems [3.8615905456206256]
  This paper introduces the Credibility Index via Explanation Stability (CIES), a metric that measures how robust a model's explanations are when subject to realistic business noise.<n>CIES captures whether the reasons behind a prediction remain consistent, not just the prediction itself.<n>Results demonstrate that model complexity impacts explanation credibility, class imbalance treatment via SMOTE affects not only predictive performance but also explanation stability.
  arXiv  Detail & Related papers  (2026-03-05T10:11:55Z)
• VI-CuRL: Stabilizing Verifier-Independent RL Reasoning via Confidence-Guided Variance Reduction [55.04308051033549]
  Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a dominant paradigm for enhancing Large Language Models (LLMs) reasoning.<n>We introduceVerifier-Independent Curriculum Reinforcement Learning (VI-CuRL), a framework that leverages the model's intrinsic confidence to construct a curriculum independent from external verifiers.
  arXiv  Detail & Related papers  (2026-02-13T03:40:52Z)
• Not All Preferences Are Created Equal: Stability-Aware and Gradient-Efficient Alignment for Reasoning Models [52.48582333951919]
  We propose a dynamic framework designed to enhance alignment reliability by maximizing the Signal-to-Noise Ratio of policy updates.<n>SAGE (Stability-Aware Gradient Efficiency) integrates a coarse-grained curriculum mechanism that refreshes candidate pools based on model competence.<n> Experiments on multiple mathematical reasoning benchmarks demonstrate that SAGE significantly accelerates convergence and outperforms static baselines.
  arXiv  Detail & Related papers  (2026-02-01T12:56:10Z)
• ReasonBENCH: Benchmarking the (In)Stability of LLM Reasoning [2.1461777157838724]
  We introduce ReasonBENCH, the first benchmark designed to quantify the underlying instability in large language models (LLMs) reasoning.<n>Across tasks from different domains, we find that the vast majority of reasoning strategies and models exhibit high instability.<n>We further analyze the impact of prompts, model families, and scale on the trade-off between solve rate and stability.
  arXiv  Detail & Related papers  (2025-12-08T18:26:58Z)
• Learning-to-Context Slope: Evaluating In-Context Learning Effectiveness Beyond Performance Illusions [42.80928434779115]
  In-context learning (ICL) has emerged as an effective approach to enhance the performance of large language models.<n>Current evaluation approaches suffer from low reliability, poor attribution, and impracticality in data-insufficient scenarios.<n>We propose the Learning-to-Context Slope (LCS), a novel metric that quantifies ICL effectiveness by modeling the slope between learning gain and contextual relevance.
  arXiv  Detail & Related papers  (2025-06-29T08:55:37Z)
• Cost-Optimal Active AI Model Evaluation [71.2069549142394]
  Development of generative AI systems requires continual evaluation, data acquisition, and annotation.<n>We develop novel, cost-aware methods for actively balancing the use of a cheap, but often inaccurate, weak rater.<n>We derive a family of cost-optimal policies for allocating a given annotation budget between weak and strong raters.
  arXiv  Detail & Related papers  (2025-06-09T17:14:41Z)
• Aurora: Are Android Malware Classifiers Reliable and Stable under Distribution Shift? [51.12297424766236]
  AURORA is a framework to evaluate malware classifiers based on their confidence quality and operational resilience.<n>AURORA is complemented by a set of metrics designed to go beyond point-in-time performance.<n>The fragility in SOTA frameworks across datasets of varying drift suggests the need for a return to the whiteboard.
  arXiv  Detail & Related papers  (2025-05-28T20:22:43Z)
• An Identifiable Cost-Aware Causal Decision-Making Framework Using Counterfactual Reasoning [18.324601057882386]
  We propose a minimum-cost causal decision (MiCCD) framework via counterfactual reasoning to solve the necessary cause.<n> Emphasis is placed on making counterfactual reasoning processes identifiable in the presence of mixed anomaly data.<n>MiCCD outperforms conventional methods across multiple metrics, including F1-score, cost efficiency, and ranking quality(nDCG@k values)
  arXiv  Detail & Related papers  (2025-05-13T08:41:45Z)
• Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization [58.390885294401066]
  Retrieval-augmented generation (RAG) has become a widely adopted paradigm for enabling knowledge-grounded large language models (LLMs)<n>RAG pipelines often fail to ensure that model reasoning remains consistent with the evidence retrieved, leading to factual inconsistencies or unsupported conclusions.<n>We propose AlignRAG, a novel iterative framework grounded in Critique-Driven Alignment (CDA)<n>We introduce AlignRAG-auto, an autonomous variant that dynamically terminates refinement, removing the need to pre-specify the number of critique iterations.
  arXiv  Detail & Related papers  (2025-04-21T04:56:47Z)
• Implementation of an Asymmetric Adjusted Activation Function for Class Imbalance Credit Scoring [4.032608240950855]
  Credit scoring is a systematic approach to evaluate a borrower's probability of default (PD) on a bank loan.<n>The data associated with such scenarios are characteristically imbalanced, complicating binary classification.<n>We introduce an innovative yet straightforward optimized activation function by incorporating an IR-dependent asymmetric adjusted factor embedded Sigmoid activation function.
  arXiv  Detail & Related papers  (2025-01-21T16:54:39Z)
• Self-Evolving Critique Abilities in Large Language Models [59.861013614500024]
  This paper explores enhancing critique abilities of Large Language Models (LLMs)<n>We introduce SCRIT, a framework that trains LLMs with self-generated data to evolve their critique abilities.<n>Our analysis reveals that SCRIT's performance scales positively with data and model size.
  arXiv  Detail & Related papers  (2025-01-10T05:51:52Z)
• On the Stability-Plasticity Dilemma of Class-Incremental Learning [50.863180812727244]
  A primary goal of class-incremental learning is to strike a balance between stability and plasticity. This paper aims to shed light on how effectively recent class-incremental learning algorithms address the stability-plasticity trade-off.
  arXiv  Detail & Related papers  (2023-04-04T09:34:14Z)
• Stabilizing and Improving Federated Learning with Non-IID Data and Client Dropout [15.569507252445144]
  Label distribution skew induced data heterogeniety has been shown to be a significant obstacle that limits the model performance in federated learning. We propose a simple yet effective framework by introducing a prior-calibrated softmax function for computing the cross-entropy loss. The improved model performance over existing baselines in the presence of non-IID data and client dropout is demonstrated.
  arXiv  Detail & Related papers  (2023-03-11T05:17:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.