Beyond Aggregate Calibration: Decomposing Income-Conditional Recall Disparities in Automated Credit Default Prediction

• URL: http://arxiv.org/abs/2608.08202v1
• Date: Sat, 08 Aug 2026 15:57:15 GMT
• Title: Beyond Aggregate Calibration: Decomposing Income-Conditional Recall Disparities in Automated Credit Default Prediction
• Abstract summary: High-income defaulters are disproportionately classified as label noise relative to low-income defaulters.<n>A 16.86 percentage point gap in true positive rate (recall) between high- and low-income borrowers who ultimately defaulted.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data-centric curation pipelines frequently rely on model confidence scores to flag and filter noisy or mislabeled training instances. Evaluating this filtering convention on a large-scale consumer lending sample (LendingClub, N = 1,344,936) uncovers an underlying demographic asymmetry: high-income defaulters are disproportionately classified as label noise relative to low-income defaulters (Cramer's V approximately 0.03-0.07). Re-examining this behavior through the lens of equal opportunity [Hardt et al., 2016] reveals a far more severe discrepancy: a 16.86 percentage point gap in true positive rate (recall) between high- and low-income borrowers who ultimately defaulted. Implementing a sequential feature-blinding methodology allows us to isolate the drivers of this disparity across three distinct mechanisms: (1) direct reliance on self-reported applicant income; (2) algorithmic absorption of upstream institutional bias encoded within origination interest rates; and (3) a residual disparity (3.55 percentage points in cross-validation; 2.56 percentage points on a held-out test partition, Z = -4.04, p < 0.0001) that remains even after purging both income and interest rates from the model. Out-of-sample signed SHAP valuations demonstrate that this residual gap is maintained by structural proxies, most notably loan amount and home ownership status. These empirical findings show that simply blinding an algorithm to sensitive attributes fails to ensure fairness when institutional pricing decisions and behavioral proxy variables collectively reconstruct the omitted signals. We outline the practical implications of these findings for auditing data-centric AI workflows within regulated financial institutions.

Related papers

• The Magnitude Mirage: Rethinking Confidence for Reasoning-Intensive Retrieval [22.82938587928396]
  Many production RAG systems implement retrieval abstention by thresholding raw similarity scores, implicitly treating score magnitude as a confidence signal.<n>We demonstrate that this practice degrades systematically as queries require reasoning beyond semantic matching.<n>To address this without computationally expensive alternatives, we conduct a large-scale empirical study of six zero-cost Query Performance Prediction (QPP) metrics across three cognitive tiers.
  arXiv  Detail & Related papers  (2026-09-14T13:49:24Z)
• Credit Without Ground Truth: Auditing Step-Level Credit Assignment in LLM Agents Against Executed Replay [0.6768558752130311]
  We audit against policy-conditional ground truth from executed replay in a single-agent tool environment.<n>No step-level credit signals we audit show reliable incremental fidelity beyond marginal-matched shuffled control.<n>A confidence-only router recovers pivotal steps at chance level, but cuts judge cost by 13.1% per turn.
  arXiv  Detail & Related papers  (2026-08-20T08:04:00Z)
• PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception [75.70273182182397]
  PerceptionRubrics is a rubric-based evaluation framework.<n>It addresses the gap between saturated benchmark scores and real-world brittleness.<n>It pairs 1,038 information-dense images with over 10,000 instance-specific rubrics.
  arXiv  Detail & Related papers  (2026-06-26T17:59:15Z)
• From Observation to Intervention: A Causal Audit of Expert Importance in Mixture-of-Experts Models [0.2230291569252836]
  We investigate the use of routing statistics in Mixture-of-Experts (MoE) pruning.<n>We find no observational metric predicts causal expert importance in any model.<n>Our results provide an explicit counterexample to the common inferential step from population-level observational summaries to token-level interventional claims about expert importance.
  arXiv  Detail & Related papers  (2026-06-09T11:04:19Z)
• Do Fair Models Reason Fairly? Counterfactual Explanation Consistency for Procedural Fairness in Credit Decisions [0.0]
  We show that existing outcome-fair models can still apply fundamentally different reasoning to individuals.<n>We propose Counterfactual Explanation Consistency ( CEC), a framework that detects and mitigates this bias.<n>Key contributions include a nearest-neighbor counterfactual generation method, a modified baseline for integrated gradient comparisons, and a corresponding training loss.
  arXiv  Detail & Related papers  (2026-05-12T19:54:25Z)
• Towards Anytime-Valid Statistical Watermarking [63.02116925616554]
  We develop the first e-value-based watermarking framework, Anchored E-Watermarking, that unifies optimal sampling with anytime-valid inference.<n>Our framework can significantly enhance sample efficiency, reducing the average token budget required for detection by 13-15% relative to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-02-19T18:32:26Z)
• PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering [71.15346406323827]
  We introduce PRIME, a benchmark for evaluating verifiers on Process-Outcome Alignment verification.<n>We find that current verifiers frequently fail to detect derivation flaws.<n>We propose a process-aware RLVR training paradigm utilizing verifiers selected via PRIME.
  arXiv  Detail & Related papers  (2026-02-12T04:45:01Z)
• Dependence-Aware Label Aggregation for LLM-as-a-Judge via Ising Models [55.94503936470247]
  Large-scale AI evaluation increasingly relies on aggregating binary judgments from $K$ annotators, including judges.<n>Most classical methods assume annotators are conditionally independent given the true label $Yin0,1$, an assumption often violated by LLM judges.<n>We study label aggregation through a hierarchy of dependence-aware models based on Ising graphical models and latent factors.
  arXiv  Detail & Related papers  (2026-01-29T21:26:50Z)
• Empirical Likelihood-Based Fairness Auditing: Distribution-Free Certification and Flagging [18.71249153088185]
  Machine learning models in high-stakes applications, such as recidivism prediction and automated personnel selection, often exhibit systematic performance disparities.<n>We propose a novel empirical likelihood-based (EL) framework that constructs robust statistical measures for model performance disparities.
  arXiv  Detail & Related papers  (2026-01-28T05:36:19Z)
• Matchings Under Biased and Correlated Evaluations [18.002268181853527]
  We study a two-institution stable matching model in which candidates from two distinct groups are evaluated.<n>We study the representation ratio, i.e., the ratio of disadvantaged to advantaged candidates selected by the matching process.
  arXiv  Detail & Related papers  (2025-10-24T00:33:52Z)
• Reference-Free Rating of LLM Responses via Latent Information [53.463883683503106]
  We study the common practice of asking a judge model to assign Likert-scale scores to free-text responses.<n>We then propose and evaluate Latent Judges, which derive scalar ratings from internal model signals.<n>Across a broad suite of pairwise and single-rating benchmarks, latent methods match or surpass standard prompting.
  arXiv  Detail & Related papers  (2025-09-29T12:15:52Z)
• TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them [58.04324690859212]
  Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks.<n>We identify two fundamental types of inconsistencies: Score-Comparison Inconsistency and Pairwise Transitivity Inconsistency.<n>We propose TrustJudge, a probabilistic framework that addresses these limitations through two key innovations.
  arXiv  Detail & Related papers  (2025-09-25T13:04:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.