It's All in the Mix: Wasserstein Machine Learning with Mixed Features
- URL: http://arxiv.org/abs/2312.12230v1
- Date: Tue, 19 Dec 2023 15:15:52 GMT
- Title: It's All in the Mix: Wasserstein Machine Learning with Mixed Features
- Authors: Reza Belbasi and Aras Selvi and Wolfram Wiesemann
- Abstract summary: We present a practically efficient algorithm to solve mixed-feature problems.
We demonstrate that our approach can significantly outperform existing methods that are to the presence of discrete features.
- Score: 5.739657897440173
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Problem definition: The recent advent of data-driven and end-to-end
decision-making across different areas of operations management has led to an
ever closer integration of prediction models from machine learning and
optimization models from operations research. A key challenge in this context
is the presence of estimation errors in the prediction models, which tend to be
amplified by the subsequent optimization model -- a phenomenon that is often
referred to as the Optimizer's Curse or the Error-Maximization Effect of
Optimization.
Methodology/results: A contemporary approach to combat such estimation errors
is offered by distributionally robust problem formulations that consider all
data-generating distributions close to the empirical distribution derived from
historical samples, where `closeness' is determined by the Wasserstein
distance. While those techniques show significant promise in problems where all
input features are continuous, they scale exponentially when binary and/or
categorical features are present. This paper demonstrates that such
mixed-feature problems can indeed be solved in polynomial time. We present a
practically efficient algorithm to solve mixed-feature problems, and we compare
our method against alternative techniques both theoretically and empirically on
standard benchmark instances.
Managerial implications: Data-driven operations management problems often
involve prediction models with discrete features. We develop and analyze a
methodology that faithfully accounts for the presence of discrete features, and
we demonstrate that our approach can significantly outperform existing methods
that are agnostic to the presence of discrete features, both theoretically and
across standard benchmark instances.
Related papers
- Forecasting Outside the Box: Application-Driven Optimal Pointwise Forecasts for Stochastic Optimization [0.0]
We present an integrated learning and optimization procedure that yields the best approximation of an unknown situation.
Numerical results conducted with inventory problems from the literature as well as a bike-sharing problem with real data demonstrate that the proposed approach performs well.
arXiv Detail & Related papers (2024-11-05T21:54:50Z) - Comparative study of regression vs pairwise models for surrogate-based heuristic optimisation [1.2535250082638645]
This paper addresses the formulation of surrogate problems as both regression models that approximate fitness (surface surrogate models) and a novel way to connect classification models (pairwise surrogate models)
The performance of the overall search, when using online machine learning-based surrogate models, depends not only on the accuracy of the predictive model but also on the kind of bias towards positive or negative cases.
arXiv Detail & Related papers (2024-10-04T13:19:06Z) - Learning from Uncertain Data: From Possible Worlds to Possible Models [13.789554282826835]
We introduce an efficient method for learning linear models from uncertain data, where uncertainty is represented as a set of possible variations in the data.
We compactly represent these dataset variations, enabling the symbolic execution of gradient descent on all possible worlds simultaneously.
Our method provides sound over-approximations of all possible optimal models and viable prediction ranges.
arXiv Detail & Related papers (2024-05-28T19:36:55Z) - Structured Radial Basis Function Network: Modelling Diversity for
Multiple Hypotheses Prediction [51.82628081279621]
Multi-modal regression is important in forecasting nonstationary processes or with a complex mixture of distributions.
A Structured Radial Basis Function Network is presented as an ensemble of multiple hypotheses predictors for regression problems.
It is proved that this structured model can efficiently interpolate this tessellation and approximate the multiple hypotheses target distribution.
arXiv Detail & Related papers (2023-09-02T01:27:53Z) - HyperImpute: Generalized Iterative Imputation with Automatic Model
Selection [77.86861638371926]
We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models.
We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
arXiv Detail & Related papers (2022-06-15T19:10:35Z) - Robust, Accurate Stochastic Optimization for Variational Inference [68.83746081733464]
We show that common optimization methods lead to poor variational approximations if the problem is moderately large.
Motivated by these findings, we develop a more robust and accurate optimization framework by viewing the underlying algorithm as producing a Markov chain.
arXiv Detail & Related papers (2020-09-01T19:12:11Z) - Modal Uncertainty Estimation via Discrete Latent Representation [4.246061945756033]
We introduce a deep learning framework that learns the one-to-many mappings between the inputs and outputs, together with faithful uncertainty measures.
Our framework demonstrates significantly more accurate uncertainty estimation than the current state-of-the-art methods.
arXiv Detail & Related papers (2020-07-25T05:29:34Z) - Accounting for Unobserved Confounding in Domain Generalization [107.0464488046289]
This paper investigates the problem of learning robust, generalizable prediction models from a combination of datasets.
Part of the challenge of learning robust models lies in the influence of unobserved confounders.
We demonstrate the empirical performance of our approach on healthcare data from different modalities.
arXiv Detail & Related papers (2020-07-21T08:18:06Z) - Learning while Respecting Privacy and Robustness to Distributional
Uncertainties and Adversarial Data [66.78671826743884]
The distributionally robust optimization framework is considered for training a parametric model.
The objective is to endow the trained model with robustness against adversarially manipulated input data.
Proposed algorithms offer robustness with little overhead.
arXiv Detail & Related papers (2020-07-07T18:25:25Z) - Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers.
We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model.
Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
arXiv Detail & Related papers (2020-06-22T21:12:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.