e-Profits: A Business-Aligned Evaluation Metric for Profit-Sensitive Customer Churn Prediction

• URL: http://arxiv.org/abs/2507.08860v1
• Date: Wed, 09 Jul 2025 11:22:24 GMT
• Title: e-Profits: A Business-Aligned Evaluation Metric for Profit-Sensitive Customer Churn Prediction
• Authors: Awais Manzoor, M. Atif Qureshi, Etain Kidney, Luca Longo,
• Abstract summary: Retention campaigns often rely on churn prediction models evaluated using traditional metrics such as AUC and F1-score.<n>We introduce e-Profits, a novel business-aligned evaluation metric that quantifies model performance based on customer-specific value, retention probability, and intervention costs.<n>e-Profits uses Kaplan-Meier survival analysis to estimate personalised retention rates and supports granular, per customer evaluation.
• Score: 1.5749416770494704
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retention campaigns in customer relationship management often rely on churn prediction models evaluated using traditional metrics such as AUC and F1-score. However, these metrics fail to reflect financial outcomes and may mislead strategic decisions. We introduce e-Profits, a novel business-aligned evaluation metric that quantifies model performance based on customer-specific value, retention probability, and intervention costs. Unlike existing profit-based metrics such as Expected Maximum Profit, which assume fixed population-level parameters, e-Profits uses Kaplan-Meier survival analysis to estimate personalised retention rates and supports granular, per customer evaluation. We benchmark six classifiers across two telecom datasets (IBM Telco and Maven Telecom) and demonstrate that e-Profits reshapes model rankings compared to traditional metrics, revealing financial advantages in models previously overlooked by AUC or F1-score. The metric also enables segment-level insight into which models maximise return on investment for high-value customers. e-Profits is designed as an understandable, post hoc tool to support model evaluation in business contexts, particularly for marketing and analytics teams prioritising profit-driven decisions. All source code is available at: https://github.com/matifq/eprofits.

Related papers

• Comparative Analysis of Modern Machine Learning Models for Retail Sales Forecasting [0.0]
  When forecasts underestimate the level of sales, firms experience lost sales, shortages, and impact on the reputation of the retailer in their relevant market.<n>This study provides an exhaustive assessment of the forecasting models applied to a high-resolution brick-and-mortar retail dataset.
  arXiv  Detail & Related papers  (2025-06-06T10:08:17Z)
• Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Training [51.41246396610475]
  This paper aims to predict performance in closed-book question answering (QA) without the help of external tools.<n>We conduct large-scale retrieval and semantic analysis across the pre-training corpora of 21 publicly available and 3 custom-trained large language models.<n>Building on these foundations, we propose Size-dependent Mutual Information (SMI), an information-theoretic metric that linearly correlates pre-training data characteristics.
  arXiv  Detail & Related papers  (2025-02-06T13:23:53Z)
• Customer Lifetime Value Prediction with Uncertainty Estimation Using Monte Carlo Dropout [3.187236205541292]
  We propose a novel approach that enhances the architecture of purely neural network models by incorporating the Monte Carlo Dropout (MCD) framework. We benchmarked the proposed method using data from one of the most downloaded mobile games in the world. Our approach provides confidence metric as an extra dimension for performance evaluation across various neural network models.
  arXiv  Detail & Related papers  (2024-11-24T18:14:44Z)
• A Probabilistic Perspective on Unlearning and Alignment for Large Language Models [48.96686419141881]
  We introduce the first formal probabilistic evaluation framework for Large Language Models (LLMs)<n> Namely, we propose novel metrics with high probability guarantees concerning the output distribution of a model.<n>Our metrics are application-independent and allow practitioners to make more reliable estimates about model capabilities before deployment.
  arXiv  Detail & Related papers  (2024-10-04T15:44:23Z)
• Harnessing Earnings Reports for Stock Predictions: A QLoRA-Enhanced LLM Approach [6.112119533910774]
  This paper introduces an advanced approach by employing Large Language Models (LLMs) instruction fine-tuned with a novel combination of instruction-based techniques and quantized low-rank adaptation (QLoRA) compression. Our methodology integrates 'base factors', such as financial metric growth and earnings transcripts, with 'external factors', including recent market indices performances and analyst grades, to create a rich, supervised dataset. This study not only demonstrates the power of integrating cutting-edge AI with fine-tuned financial data but also paves the way for future research in enhancing AI-driven financial analysis tools.
  arXiv  Detail & Related papers  (2024-08-13T04:53:31Z)
• F-FOMAML: GNN-Enhanced Meta-Learning for Peak Period Demand Forecasting with Proxy Data [65.6499834212641]
  We formulate the demand prediction as a meta-learning problem and develop the Feature-based First-Order Model-Agnostic Meta-Learning (F-FOMAML) algorithm. By considering domain similarities through task-specific metadata, our model improved generalization, where the excess risk decreases as the number of training tasks increases. Compared to existing state-of-the-art models, our method demonstrates a notable improvement in demand prediction accuracy, reducing the Mean Absolute Error by 26.24% on an internal vending machine dataset and by 1.04% on the publicly accessible JD.com dataset.
  arXiv  Detail & Related papers  (2024-06-23T21:28:50Z)
• Optimizing Sales Forecasts through Automated Integration of Market Indicators [0.0]
  This work investigates the potential of data-driven techniques to automatically select and integrate market indicators for improving customer demand predictions. By adopting an exploratory methodology, we integrate macroeconomic time series, such as national GDP growth, into textitNeural Prophet and textitSARIMAX forecasting models. It could be shown that forecasts can be significantly enhanced by incorporating external information.
  arXiv  Detail & Related papers  (2024-05-15T08:11:41Z)
• Churn Prediction via Multimodal Fusion Learning:Integrating Customer Financial Literacy, Voice, and Behavioral Data [14.948017876322597]
  This paper proposes a multimodal fusion learning model for identifying customer churn risk levels in financial service providers. Our approach integrates customer sentiments financial literacy (FL) level, and financial behavioral data. Our novel approach demonstrates a marked improvement in churn prediction, achieving a test accuracy of 91.2%, a Mean Average Precision (MAP) score of 66, and a Macro-Averaged F1 score of 54.
  arXiv  Detail & Related papers  (2023-12-03T06:28:55Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• Variable Importance Matching for Causal Inference [73.25504313552516]
  We describe a general framework called Model-to-Match that achieves these goals. Model-to-Match uses variable importance measurements to construct a distance metric. We operationalize the Model-to-Match framework with LASSO.
  arXiv  Detail & Related papers  (2023-02-23T00:43:03Z)
• Exploring validation metrics for offline model-based optimisation with diffusion models [50.404829846182764]
  In model-based optimisation (MBO) we are interested in using machine learning to design candidates that maximise some measure of reward with respect to a black box function called the (ground truth) oracle. While an approximation to the ground oracle can be trained and used in place of it during model validation to measure the mean reward over generated candidates, the evaluation is approximate and vulnerable to adversarial examples. This is encapsulated under our proposed evaluation framework which is also designed to measure extrapolation.
  arXiv  Detail & Related papers  (2022-11-19T16:57:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.