Scalable Decision-Focused Learning through Cost-Sensitive Regression

• URL: http://arxiv.org/abs/2605.18005v1
• Date: Mon, 18 May 2026 07:58:26 GMT
• Title: Scalable Decision-Focused Learning through Cost-Sensitive Regression
• Abstract summary: Decision-focused learning (DFL) approaches often rely on repeated solving of the underlying optimization problem during training, making them computationally expensive and difficult to scale.<n>We achieve the learning problem as a cost-sensitive multi-output regression problem: multi-output due to the problem having multiple uncertain parameters, and cost-sensitive due to the downstream task cost being the real target.<n>We show that the combination of loss components comparable downstream task quality to the state of the art, while being significantly more efficient, enabling scaling to problem sizes that have not been tackled before with DFL.
• Score: 14.812233565076399
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Many real-world combinatorial problems involve uncertain parameters, which can be predicted given contextual features and historical data. These `predict-then-optimize' or `contextual optimization' problems have gained significant attention: end-to-end training methods can now minimize the downstream task cost rather than the predictive error. However, despite their effectiveness, these decision-focused learning (DFL) approaches often rely on repeated solving of the underlying combinatorial optimization problem during training, making them computationally expensive and difficult to scale. We reframe the learning problem as a cost-sensitive multi-output regression problem: multi-output due to the combinatorial problem having multiple uncertain parameters, and cost-sensitive due to the downstream task cost being the real target. Our technical contribution is the formalization of multiple loss function components that follow from this reframing: cost-insensitive normalization, decision-aware asymmetric penalization of over- and underpredictions, and instance-based costs that mimic the true downstream task-based loss locally. These components require zero or one solve per training data instance, while requiring no further solves during training. Experiments show that the combination of loss components achieves comparable downstream task quality to the state of the art, while being significantly more efficient, enabling scaling to problem sizes that have not been tackled before with DFL.

Related papers

• Breakeven complexity: A new perspective on neural partial differential equation solvers [13.91054053357187]
  We evaluate the breakeven complexity of neural PDE solvers on three PDEs on 2D periodic domains from APEBench.<n>Our results suggest that neural PDE solvers become more effective as problems get harder in terms of cost, dimension, rollout, physics regime.
  arXiv  Detail & Related papers  (2026-05-14T20:33:24Z)
• Cost-Aware Learning [72.31444819326795]
  We consider the problem of Cost-Aware Learning, where sampling different component functions incur different costs.<n>We propose the Cost-Aware Descent algorithm for convex functions, and derive its cost complexity to attain an error of $$.<n>We introduce Cost-Aware GRPO, an algorithm designed to reduce the cost of policy optimization while preserving performance.
  arXiv  Detail & Related papers  (2026-04-30T15:39:09Z)
• Scalable Decision Focused Learning via Online Trainable Surrogates [9.624413875440233]
  We propose an acceleration method based on replacing costly loss function evaluations with an efficient surrogate.<n>Unlike previously defined surrogates, our approach relies on unbiased estimators reducing the risk of spurious local optima.<n>In our results, the method reduces costly inner solver calls, with a solution quality comparable to other state-of-the-art techniques.
  arXiv  Detail & Related papers  (2025-12-03T15:09:21Z)
• Online Decision-Focused Learning [74.3205104323777]
  Decision-focused learning (DFL) is an increasingly popular paradigm for training models whose predictive outputs are used in decision-making tasks.<n>In this paper, we regularize the objective function to make it different and investigate how to overcome nonoptimality function.<n>We also showcase the effectiveness of our algorithms on a knapsack experiment, where they outperform two standard benchmarks.
  arXiv  Detail & Related papers  (2025-05-19T10:40:30Z)
• Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum [78.27945336558987]
  Decentralized server (DFL) eliminates reliance on client-client architecture.<n>Non-smooth regularization is often incorporated into machine learning tasks.<n>We propose a novel novel DNCFL algorithm to solve these problems.
  arXiv  Detail & Related papers  (2025-04-17T08:32:25Z)
• When to Forget? Complexity Trade-offs in Machine Unlearning [23.507879460531264]
  Machine Unlearning (MU) aims at removing the influence of specific data points from a trained model.<n>We analyze the efficiency of unlearning methods and establish the first upper and lower bounds on minimax times for this problem.<n>We provide a phase diagram for the unlearning complexity ratio -- a novel metric that compares the computational cost of the best unlearning method to full model retraining.
  arXiv  Detail & Related papers  (2025-02-24T16:56:27Z)
• Towards Resource-Efficient Federated Learning in Industrial IoT for Multivariate Time Series Analysis [50.18156030818883]
  Anomaly and missing data constitute a thorny problem in industrial applications. Deep learning enabled anomaly detection has emerged as a critical direction. The data collected in edge devices contain user privacy.
  arXiv  Detail & Related papers  (2024-11-06T15:38:31Z)
• Multi-Resolution Active Learning of Fourier Neural Operators [33.63483360957646]
  We propose Multi-Resolution Active learning of FNO (MRA-FNO), which can dynamically select the input functions and resolutions to lower the data cost as much as possible. Specifically, we propose a probabilistic multi-resolution FNO and use ensemble Monte-Carlo to develop an effective posterior inference algorithm. We have shown the advantage of our method in several benchmark operator learning tasks.
  arXiv  Detail & Related papers  (2023-09-29T04:41:27Z)
• A Multi-Head Ensemble Multi-Task Learning Approach for Dynamical Computation Offloading [62.34538208323411]
  We propose a multi-head ensemble multi-task learning (MEMTL) approach with a shared backbone and multiple prediction heads (PHs) MEMTL outperforms benchmark methods in both the inference accuracy and mean square error without requiring additional training data.
  arXiv  Detail & Related papers  (2023-09-02T11:01:16Z)
• Score Function Gradient Estimation to Widen the Applicability of Decision-Focused Learning [17.962860438133312]
  Decision-focused learning (DFL) paradigm overcomes limitation by training to directly minimize a task loss, e.g. regret. We propose an alternative method that makes no such assumptions, it combines smoothing with score function estimation which works on any task loss. Experiments show that it typically requires more epochs, but that it is on par with specialized methods and performs especially well for the difficult case of problems with uncertainty in the constraints, in terms of solution quality, scalability, or both.
  arXiv  Detail & Related papers  (2023-07-11T12:32:13Z)
• FIRE: A Failure-Adaptive Reinforcement Learning Framework for Edge Computing Migrations [54.34189781923818]
  FIRE is a framework that adapts to rare events by training a RL policy in an edge computing digital twin environment.<n>We propose ImRE, an importance sampling-based Q-learning algorithm, which samples rare events proportionally to their impact on the value function.<n>We show that FIRE reduces costs compared to vanilla RL and the greedy baseline in the event of failures.
  arXiv  Detail & Related papers  (2022-09-28T19:49:39Z)
• Instance-optimality in optimal value estimation: Adaptivity via variance-reduced Q-learning [99.34907092347733]
  We analyze the problem of estimating optimal $Q$-value functions for a discounted Markov decision process with discrete states and actions. Using a local minimax framework, we show that this functional arises in lower bounds on the accuracy on any estimation procedure. In the other direction, we establish the sharpness of our lower bounds, up to factors logarithmic in the state and action spaces, by analyzing a variance-reduced version of $Q$-learning.
  arXiv  Detail & Related papers  (2021-06-28T00:38:54Z)
• Cost-Effective Federated Learning Design [37.16466118235272]
  Federated learning (FL) is a distributed learning paradigm that enables a large number of devices to collaboratively learn a model without sharing their raw data. Despite its efficiency and effectiveness, the iterative on-device learning process incurs a considerable cost in terms of learning time and energy consumption. We analyze how to design adaptive FL that optimally chooses essential control variables to minimize the total cost while ensuring convergence.
  arXiv  Detail & Related papers  (2020-12-15T14:45:11Z)
• Automatically Learning Compact Quality-aware Surrogates for Optimization Problems [55.94450542785096]
  Solving optimization problems with unknown parameters requires learning a predictive model to predict the values of the unknown parameters and then solving the problem using these values. Recent work has shown that including the optimization problem as a layer in a complex training model pipeline results in predictions of iteration of unobserved decision making. We show that we can improve solution quality by learning a low-dimensional surrogate model of a large optimization problem.
  arXiv  Detail & Related papers  (2020-06-18T19:11:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.