A transformer-based model for default prediction in mid-cap corporate markets

• URL: http://arxiv.org/abs/2111.09902v4
• Date: Thu, 20 Apr 2023 10:13:35 GMT
• Title: A transformer-based model for default prediction in mid-cap corporate markets
• Authors: Kamesh Korangi, Christophe Mues, Cristi\'an Bravo
• Abstract summary: We study mid-cap companies with less than US $10 billion in market capitalisation. We look to predict the default probability term structure over the medium term. We understand which data sources contribute most to the default risk.
• Score: 13.535770763481905
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this paper, we study mid-cap companies, i.e. publicly traded companies with less than US $10 billion in market capitalisation. Using a large dataset of US mid-cap companies observed over 30 years, we look to predict the default probability term structure over the medium term and understand which data sources (i.e. fundamental, market or pricing data) contribute most to the default risk. Whereas existing methods typically require that data from different time periods are first aggregated and turned into cross-sectional features, we frame the problem as a multi-label time-series classification problem. We adapt transformer models, a state-of-the-art deep learning model emanating from the natural language processing domain, to the credit risk modelling setting. We also interpret the predictions of these models using attention heat maps. To optimise the model further, we present a custom loss function for multi-label classification and a novel multi-channel architecture with differential training that gives the model the ability to use all input data efficiently. Our results show the proposed deep learning architecture's superior performance, resulting in a 13% improvement in AUC (Area Under the receiver operating characteristic Curve) over traditional models. We also demonstrate how to produce an importance ranking for the different data sources and the temporal relationships using a Shapley approach specific to these models.

Related papers

• Learning Augmentation Policies from A Model Zoo for Time Series Forecasting [58.66211334969299]
  We introduce AutoTSAug, a learnable data augmentation method based on reinforcement learning. By augmenting the marginal samples with a learnable policy, AutoTSAug substantially improves forecasting performance.
  arXiv  Detail & Related papers  (2024-09-10T07:34:19Z)
• F-FOMAML: GNN-Enhanced Meta-Learning for Peak Period Demand Forecasting with Proxy Data [65.6499834212641]
  We formulate the demand prediction as a meta-learning problem and develop the Feature-based First-Order Model-Agnostic Meta-Learning (F-FOMAML) algorithm. By considering domain similarities through task-specific metadata, our model improved generalization, where the excess risk decreases as the number of training tasks increases. Compared to existing state-of-the-art models, our method demonstrates a notable improvement in demand prediction accuracy, reducing the Mean Absolute Error by 26.24% on an internal vending machine dataset and by 1.04% on the publicly accessible JD.com dataset.
  arXiv  Detail & Related papers  (2024-06-23T21:28:50Z)
• A Scalable and Transferable Time Series Prediction Framework for Demand Forecasting [24.06534393565697]
  Time series forecasting is one of the most essential and ubiquitous tasks in many business problems. We propose Forecasting orchestra (Forchestra), a simple but powerful framework capable of accurately predicting future demand for a diverse range of items.
  arXiv  Detail & Related papers  (2024-02-29T18:01:07Z)
• Generative Pretrained Hierarchical Transformer for Time Series Forecasting [3.739587363053192]
  We propose a novel generative pretrained hierarchical transformer architecture for forecasting, named textbfGPHT. We conduct sufficient experiments on eight datasets with mainstream self-supervised pretraining models and supervised models. The results demonstrated that GPHT surpasses the baseline models across various fine-tuning and zero/few-shot learning settings in the traditional long-term forecasting task.
  arXiv  Detail & Related papers  (2024-02-26T11:54:54Z)
• Dirichlet-based Uncertainty Quantification for Personalized Federated Learning with Improved Posterior Networks [9.54563359677778]
  This paper presents a new approach to federated learning that allows selecting a model from global and personalized ones. It is achieved through a careful modeling of predictive uncertainties that helps to detect local and global in- and out-of-distribution data. The comprehensive experimental evaluation on the popular real-world image datasets shows the superior performance of the model in the presence of out-of-distribution data.
  arXiv  Detail & Related papers  (2023-12-18T14:30:05Z)
• Generative Machine Learning for Multivariate Equity Returns [0.0]
  We study the efficacy of conditional importance weighted autoencoders and conditional normalizing flows for the task of modeling the returns of equities. The main problem we work to address is modeling the joint distribution of all the members of the S&P 500, or, in other words, learning a 500-dimensional joint distribution. We show that this generative model has a broad range of applications in finance, including generating realistic synthetic data, volatility and correlation estimation, risk analysis, and portfolio optimization.
  arXiv  Detail & Related papers  (2023-11-21T18:41:48Z)
• Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model [74.62272538148245]
  We show that for arbitrary pairings of pretrained models, one model extracts significant data context unavailable in the other. We investigate if it is possible to transfer such "complementary" knowledge from one model to another without performance degradation.
  arXiv  Detail & Related papers  (2023-10-26T17:59:46Z)
• Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting [54.04430089029033]
  We present Lag-Llama, a general-purpose foundation model for time series forecasting based on a decoder-only transformer architecture. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities. When fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-12T12:29:32Z)
• Improving Neural Ranking Models with Traditional IR Methods [13.354623448774877]
  TF-IDF, a traditional keyword matching method, with a shallow embedding model provides a low cost path to compete well with the performance of complex neural ranking models on 3 datasets. Adding TF-IDF measures improves the performance of large-scale fine tuned models on these tasks.
  arXiv  Detail & Related papers  (2023-08-29T05:18:47Z)
• Dataless Knowledge Fusion by Merging Weights of Language Models [51.8162883997512]
  Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models. This creates a barrier to fusing knowledge across individual models to yield a better single model. We propose a dataless knowledge fusion method that merges models in their parameter space.
  arXiv  Detail & Related papers  (2022-12-19T20:46:43Z)
• Synthetic Model Combination: An Instance-wise Approach to Unsupervised Ensemble Learning [92.89846887298852]
  Consider making a prediction over new test data without any opportunity to learn from a training set of labelled data. Give access to a set of expert models and their predictions alongside some limited information about the dataset used to train them.
  arXiv  Detail & Related papers  (2022-10-11T10:20:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.