Return Prediction for Mean-Variance Portfolio Selection: How Decision-Focused Learning Shapes Forecasting Models

• URL: http://arxiv.org/abs/2409.09684v3
• Date: Wed, 13 Aug 2025 14:52:58 GMT
• Title: Return Prediction for Mean-Variance Portfolio Selection: How Decision-Focused Learning Shapes Forecasting Models
• Authors: Junhyeong Lee, Haeun Jeon, Hyunglip Bae, Yongjae Lee,
• Abstract summary: Decision-Focused Learning can integrate prediction and optimization to improve decision-making outcomes.<n>This study investigates how DFL adjusts stock return prediction models to optimize decisions in mean-variance optimization (MVO)<n>Our findings reveal why DFL achieves superior portfolio performance despite higher prediction errors.
• Score: 25.72157859795055
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Markowitz laid the foundation of portfolio theory through the mean-variance optimization (MVO) framework. However, the effectiveness of MVO is contingent on the precise estimation of expected returns, variances, and covariances of asset returns, which are typically uncertain. Machine learning models are becoming useful in estimating uncertain parameters, and such models are trained to minimize prediction errors, such as mean squared errors (MSE), which treat prediction errors uniformly across assets. Recent studies have pointed out that this approach would lead to suboptimal decisions and proposed Decision-Focused Learning (DFL) as a solution, integrating prediction and optimization to improve decision-making outcomes. While studies have shown DFL's potential to enhance portfolio performance, the detailed mechanisms of how DFL modifies prediction models for MVO remain unexplored. This study investigates how DFL adjusts stock return prediction models to optimize decisions in MVO. Theoretically, we show that DFL's gradient can be interpreted as tilting the MSE-based prediction errors by the inverse covariance matrix, effectively incorporating inter-asset correlations into the learning process, while MSE treats each asset's error independently. This tilting mechanism leads to systematic prediction biases where DFL overestimates returns for assets included in portfolios while underestimating excluded assets. Our findings reveal why DFL achieves superior portfolio performance despite higher prediction errors. The strategic biases are features, not flaws.

Related papers

• $φ$-DPO: Fairness Direct Preference Optimization Approach to Continual Learning in Large Multimodal Models [58.217707070069885]
  This paper presents a novel Fairness Direct Preference Optimization (FaiDPO or $$-DPO) framework for continual learning in LMMs.<n>We first propose a new continual learning paradigm based on Direct Preference Optimization (DPO) to mitigate catastrophic forgetting by aligning learning with pairwise preference signals.<n> Extensive experiments and ablation studies show the proposed $$-DPO achieves State-of-the-Art performance across multiple benchmarks.
  arXiv  Detail & Related papers  (2026-02-26T04:14:33Z)
• Integrated Prediction and Multi-period Portfolio Optimization [29.582959310549594]
  Multi-period portfolio optimization accounts for transaction costs, path-dependent risks, and the intertemporal structure of trading decisions.<n>This paper introduces IPMO, a model for multi-period mean-variance portfolio optimization with turnover penalties.<n>For scalability, we introduce a mirror-descent fixed-point (MDFP) differentiation scheme that avoids factorizing the Karush-Kuhn-Tucker (KKT) systems.
  arXiv  Detail & Related papers  (2025-12-12T04:31:22Z)
• Estimating Covariance for Global Minimum Variance Portfolio: A Decision-Focused Learning Approach [27.791742749950203]
  Decision-focused learning (DFL) is an approach that directly optimize decision quality rather than prediction error.<n>We show that DFL-based methods consistently deliver superior decision performance.
  arXiv  Detail & Related papers  (2025-08-14T16:00:52Z)
• Gen-DFL: Decision-Focused Generative Learning for Robust Decision Making [48.62706690668867]
  Decision-focused generative learning (Gen-DFL) is a novel framework that leverages generative models to adaptively model uncertainty and improve decision quality. The paper shows, theoretically, that Gen-DFL achieves improved worst-case performance bounds compared to traditional DFL.
  arXiv  Detail & Related papers  (2025-02-08T06:52:11Z)
• Decision-informed Neural Networks with Large Language Model Integration for Portfolio Optimization [29.30269598267018]
  This paper addresses the critical disconnect between prediction and decision quality in portfolio optimization. We exploit the representational power of Large Language Models (LLMs) for investment decisions. Experiments on S&P100 and DOW30 datasets show that our model consistently outperforms state-of-the-art deep learning models.
  arXiv  Detail & Related papers  (2025-02-02T15:45:21Z)
• DFF: Decision-Focused Fine-tuning for Smarter Predict-then-Optimize with Limited Data [7.70699448711673]
  Decision-focused learning (DFL) offers an end-to-end approach to the predict-then-optimize (PO) framework by training predictive models directly on decision loss (DL) Some predictive models are non-differentiable or black-box, which cannot be adjusted using gradient-based methods. We propose a novel framework, Decision-Focused Fine-tuning (DFF), which embeds the DFL module into the PO pipeline via a novel bias correction module.
  arXiv  Detail & Related papers  (2025-01-03T15:46:25Z)
• Uncertainty-Penalized Direct Preference Optimization [52.387088396044206]
  We develop a pessimistic framework for DPO by introducing preference uncertainty penalization schemes. The penalization serves as a correction to the loss which attenuates the loss gradient for uncertain samples. We show improved overall performance compared to vanilla DPO, as well as better completions on prompts from high-uncertainty chosen/rejected responses.
  arXiv  Detail & Related papers  (2024-10-26T14:24:37Z)
• Scaling Laws for Predicting Downstream Performance in LLMs [75.28559015477137]
  This work focuses on the pre-training loss as a more computation-efficient metric for performance estimation.<n>We present FLP-M, a fundamental approach for performance prediction that addresses the practical need to integrate datasets from multiple sources during pre-training.
  arXiv  Detail & Related papers  (2024-10-11T04:57:48Z)
• Self-Evolutionary Large Language Models through Uncertainty-Enhanced Preference Optimization [9.618391485742968]
  Iterative preference optimization has recently become one of the de-facto training paradigms for large language models (LLMs) We present an uncertainty-enhanced textbfPreference textbfOptimization framework to make the LLM self-evolve with reliable feedback. Our framework substantially alleviates the noisy problem and improves the performance of iterative preference optimization.
  arXiv  Detail & Related papers  (2024-09-17T14:05:58Z)
• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions. We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance. Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimization [49.396692286192206]
  We study the use of deep reinforcement learning for responsible portfolio optimization by incorporating ESG states and objectives. Our results show that deep reinforcement learning policies can provide competitive performance against mean-variance approaches for responsible portfolio allocation.
  arXiv  Detail & Related papers  (2024-03-25T12:04:03Z)
• End-to-End Learning for Fair Multiobjective Optimization Under Uncertainty [55.04219793298687]
  The Predict-Then-Forecast (PtO) paradigm in machine learning aims to maximize downstream decision quality. This paper extends the PtO methodology to optimization problems with nondifferentiable Ordered Weighted Averaging (OWA) objectives. It shows how optimization of OWA functions can be effectively integrated with parametric prediction for fair and robust optimization under uncertainty.
  arXiv  Detail & Related papers  (2024-02-12T16:33:35Z)
• On the Robustness of Decision-Focused Learning [0.0]
  Decision-Focused Learning (DFL) is an emerging learning paradigm that tackles the task of training a machine learning (ML) model to predict missing parameters of an incomplete optimization problem, where the missing parameters are predicted. DFL trains an ML model in an end-to-end system, by integrating the prediction and optimization tasks, providing better alignment of the training and testing objectives.
  arXiv  Detail & Related papers  (2023-11-28T04:34:04Z)
• Prediction-Oriented Bayesian Active Learning [51.426960808684655]
  Expected predictive information gain (EPIG) is an acquisition function that measures information gain in the space of predictions rather than parameters. EPIG leads to stronger predictive performance compared with BALD across a range of datasets and models.
  arXiv  Detail & Related papers  (2023-04-17T10:59:57Z)
• Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models [51.3422222472898]
  We document the capability of large language models (LLMs) like ChatGPT to predict stock price movements using news headlines. We develop a theoretical model incorporating information capacity constraints, underreaction, limits-to-arbitrage, and LLMs.
  arXiv  Detail & Related papers  (2023-04-15T19:22:37Z)
• Optimizing Stock Option Forecasting with the Assembly of Machine Learning Models and Improved Trading Strategies [9.553857741758742]
  This paper introduced key aspects of applying Machine Learning (ML) models, improved trading strategies, and the Quasi-Reversibility Method (QRM) to optimize stock option forecasting and trading results.
  arXiv  Detail & Related papers  (2022-11-29T04:01:16Z)
• The Sharpe predictor for fairness in machine learning [0.0]
  In machine learning applications, unfair predictions may discriminate against a minority group. Most existing approaches for fair machine learning (FML) treat fairness as a constraint or a penalization term in the optimization of a ML model. We introduce a new paradigm for FML based on Multi-Objective Optimization (SMOO), where accuracy and fairness metrics stand as conflicting objectives to be optimized simultaneously. The Sharpe predictor for FML provides the highest prediction return (accuracy) per unit of prediction risk (unfairness).
  arXiv  Detail & Related papers  (2021-08-13T22:22:34Z)
• When Does Uncertainty Matter?: Understanding the Impact of Predictive Uncertainty in ML Assisted Decision Making [68.19284302320146]
  We carry out user studies to assess how people with differing levels of expertise respond to different types of predictive uncertainty. We found that showing posterior predictive distributions led to smaller disagreements with the ML model's predictions. This suggests that posterior predictive distributions can potentially serve as useful decision aids which should be used with caution and take into account the type of distribution and the expertise of the human.
  arXiv  Detail & Related papers  (2020-11-12T02:23:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.