Foundation Models for Credit Risk Prediction: A Game Changer?

• URL: http://arxiv.org/abs/2605.18147v1
• Date: Mon, 18 May 2026 09:52:48 GMT
• Title: Foundation Models for Credit Risk Prediction: A Game Changer?
• Abstract summary: Predictive models play a pivotal role in credit risk management, guiding critical decisions through accurate estimation of default probabilities and losses.<n>Foundation models, pretrained on extensive datasets from diverse domains, have demonstrated remarkable performance by leveraging prior knowledge.
• Score: 12.43753732401141
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Predictive models play a pivotal role in credit risk management, guiding critical decisions through accurate estimation of default probabilities and losses. Extensive research has introduced new modeling techniques, complemented by large-scale benchmarking studies consolidating the state-of-the-art. Today, quasi-standards such as gradient-boosting models paired with SHAP explainers have emerged, yet continuous improvement of risk models remains a top priority. Concurrently, rapid advancements in AI, most notably large language models, have disrupted predictive modeling paradigms. Foundation models, pretrained on extensive datasets from diverse domains, have demonstrated remarkable performance by leveraging prior knowledge. While prevalent in natural language processing and computer vision, foundation models for tabular data have only recently emerged. We conjecture that pretraining on out-of-domain data is particularly beneficial in small-data settings, such as SME lending or specialized corporate portfolios, and may help address longstanding challenges including low default portfolios and class imbalance. This paper benchmarks recently proposed tabular foundation models against a broad set of competitors, including established and advanced machine learning techniques, across two core tasks: PD and LGD modeling. Our evaluation encompasses various datasets, performance indicators, and experimental conditions. We find that tabular foundation models generally perform best across datasets and tasks. Moreover, they offer significant improvement in predictive performance as dataset size shrinks. These results are remarkable given that the models are tested out-of-the-box, without hyperparameter tuning, ensuring ease of use and mitigating computational costs.

Related papers

• Beyond IID: How General Are Tabular Foundation Models, Really? [76.03669797389378]
  Research communities are increasingly evaluating foundation models on diverse datasets and tasks.<n>Standard benchmarks are mostly defined for tasks where tabular foundation models already excel.<n>We introduce BeyondArena, the first unified holistic benchmark for tabular data.
  arXiv  Detail & Related papers  (2026-06-29T14:55:52Z)
• On Predicting the Post-training Potential of Pre-trained LLMs [60.07459271263409]
  We introduce a new task of predicting post-training potential - forecasting a base model's performance before post-training.<n>We propose RuDE, a unified framework that bypasses the generation gap of base models by leveraging response discrimination.<n>Experiments demonstrate a correlation greater than 90% with post-training performance.
  arXiv  Detail & Related papers  (2026-05-12T11:33:49Z)
• Re(Visiting) Time Series Foundation Models in Finance [3.295157175236371]
  Financial time series forecasting is central to trading, portfolio optimization, and risk management.<n>Recent advances in time series foundation models (TSFMs) offer a new paradigm for learning generalizable temporal representations from large and diverse datasets.<n>This paper presents the first comprehensive empirical study of TSFMs in global financial markets.
  arXiv  Detail & Related papers  (2025-11-23T18:44:19Z)
• On Identifying Why and When Foundation Models Perform Well on Time-Series Forecasting Using Automated Explanations and Rating [7.375605655806626]
  Time-series forecasting models (TSFM) have evolved from classical statistical methods to sophisticated foundation models.<n>This work addresses concerns by combining traditional explainable AI (XAI) methods with Rating Driven Explanations (RDE)<n>We evaluate four distinct model architectures: ARIMA, Gradient Boosting, Chronos (time-series specific foundation model), Llama (general-purpose; both fine-tuned and base models)
  arXiv  Detail & Related papers  (2025-08-28T05:27:45Z)
• Towards Robust Universal Information Extraction: Benchmark, Evaluation, and Solution [66.11004226578771]
  Existing robust benchmark datasets have two key limitations.<n>They generate only a limited range of perturbations for a single Information Extraction (IE) task.<n>Considering the powerful generation capabilities of Large Language Models (LLMs), we introduce a new benchmark dataset for Robust UIE, called RUIE-Bench.<n>We show that training with only textbf15% of the data leads to an average textbf7.5% relative performance improvement across three IE tasks.
  arXiv  Detail & Related papers  (2025-03-05T05:39:29Z)
• Class-Imbalanced-Aware Adaptive Dataset Distillation for Scalable Pretrained Model on Credit Scoring [10.737033782376905]
  We present a novel framework for scaling up the application of large pretrained models on financial datasets.<n>We integrate the imbalance-aware techniques during dataset distillation, resulting in improved performance in financial datasets.
  arXiv  Detail & Related papers  (2025-01-18T06:59:36Z)
• Low-Rank Adaptation for Foundation Models: A Comprehensive Review [56.341827242332194]
  Low-Rank Adaptation (LoRA) has emerged as a highly promising approach for mitigating these challenges.<n>This survey provides the first comprehensive review of LoRA techniques beyond large Language Models to general foundation models.
  arXiv  Detail & Related papers  (2024-12-31T09:38:55Z)
• Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy [104.48511402784763]
  Performance Law for SR models aims to theoretically investigate and model the relationship between model performance and data quality.<n>We propose Approximate Entropy (ApEn) to assess data quality, presenting a more nuanced approach compared to traditional data quantity metrics.
  arXiv  Detail & Related papers  (2024-11-30T10:56:30Z)
• KACDP: A Highly Interpretable Credit Default Prediction Model [2.776411854233918]
  This paper introduces a method based on Kolmogorov-Arnold Networks (KANs)<n>KANs is a new type of neural network architecture with learnable activation functions and no linear weights.<n>Experiments show that the KACDP model outperforms mainstream credit default prediction models in performance metrics.
  arXiv  Detail & Related papers  (2024-11-26T12:58:03Z)
• A Collaborative Ensemble Framework for CTR Prediction [73.59868761656317]
  We propose a novel framework, Collaborative Ensemble Training Network (CETNet), to leverage multiple distinct models. Unlike naive model scaling, our approach emphasizes diversity and collaboration through collaborative learning. We validate our framework on three public datasets and a large-scale industrial dataset from Meta.
  arXiv  Detail & Related papers  (2024-11-20T20:38:56Z)
• Multi-document Summarization: A Comparative Evaluation [0.0]
  This paper is aimed at evaluating state-of-the-art models for Multi-document Summarization (MDS) on different types of datasets in various domains. We analyzed the performance of PRIMERA and PEG models on Big-Survey and MS$2$ datasets.
  arXiv  Detail & Related papers  (2023-09-10T07:43:42Z)
• Universal Domain Adaptation from Foundation Models: A Baseline Study [58.51162198585434]
  We make empirical studies of state-of-the-art UniDA methods using foundation models. We introduce textitCLIP distillation, a parameter-free method specifically designed to distill target knowledge from CLIP models. Although simple, our method outperforms previous approaches in most benchmark tasks.
  arXiv  Detail & Related papers  (2023-05-18T16:28:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.