Functional Output Regression with Infimal Convolution: Exploring the
Huber and $\epsilon$-insensitive Losses
- URL: http://arxiv.org/abs/2206.08220v1
- Date: Thu, 16 Jun 2022 14:45:53 GMT
- Title: Functional Output Regression with Infimal Convolution: Exploring the
Huber and $\epsilon$-insensitive Losses
- Authors: Alex Lambert, Dimitri Bouche, Zoltan Szabo, Florence d'Alch\'e-Buc
- Abstract summary: We propose a flexible framework capable of handling various forms of outliers and sparsity in the FOR family.
We derive computationally tractable algorithms relying on duality to tackle the resulting tasks.
The efficiency of the approach is demonstrated and contrasted with the classical squared loss setting on both synthetic and real-world benchmarks.
- Score: 1.7835960292396256
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The focus of the paper is functional output regression (FOR) with convoluted
losses. While most existing work consider the square loss setting, we leverage
extensions of the Huber and the $\epsilon$-insensitive loss (induced by infimal
convolution) and propose a flexible framework capable of handling various forms
of outliers and sparsity in the FOR family. We derive computationally tractable
algorithms relying on duality to tackle the resulting tasks in the context of
vector-valued reproducing kernel Hilbert spaces. The efficiency of the approach
is demonstrated and contrasted with the classical squared loss setting on both
synthetic and real-world benchmarks.
Related papers
- Byzantine-resilient Federated Learning With Adaptivity to Data Heterogeneity [54.145730036889496]
This paper deals with Gradient learning (FL) in the presence of malicious attacks Byzantine data.
A novel Average Algorithm (RAGA) is proposed, which leverages robustness aggregation and can select a dataset.
arXiv Detail & Related papers (2024-03-20T08:15:08Z) - A Robust Quantile Huber Loss With Interpretable Parameter Adjustment In
Distributional Reinforcement Learning [19.89141873890568]
This paper introduces a generalized quantile Huber loss function derived from Wasserstein distance (WD) calculation.
Compared to the classical quantile Huber loss, this innovative loss function enhances robustness against outliers.
Empirical tests on Atari games, a common application in distributional RL, and a recent hedging strategy using distributional RL, validate our proposed loss function.
arXiv Detail & Related papers (2024-01-04T15:51:49Z) - Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training.
We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
arXiv Detail & Related papers (2023-10-20T12:45:12Z) - Universal Online Learning with Gradient Variations: A Multi-layer Online Ensemble Approach [57.92727189589498]
We propose an online convex optimization approach with two different levels of adaptivity.
We obtain $mathcalO(log V_T)$, $mathcalO(d log V_T)$ and $hatmathcalO(sqrtV_T)$ regret bounds for strongly convex, exp-concave and convex loss functions.
arXiv Detail & Related papers (2023-07-17T09:55:35Z) - General Loss Functions Lead to (Approximate) Interpolation in High
Dimensions [6.738946307589741]
We provide a unified framework to approximately characterize the implicit bias of gradient descent in closed form.
Specifically, we show that the implicit bias is approximated (but not exactly equal to) the minimum-norm in high dimensions.
Our framework also recovers existing exact equivalence results for exponentially-tailed losses across binary and multiclass settings.
arXiv Detail & Related papers (2023-03-13T21:23:12Z) - Reinforcement Learning from Partial Observation: Linear Function Approximation with Provable Sample Efficiency [111.83670279016599]
We study reinforcement learning for partially observed decision processes (POMDPs) with infinite observation and state spaces.
We make the first attempt at partial observability and function approximation for a class of POMDPs with a linear structure.
arXiv Detail & Related papers (2022-04-20T21:15:38Z) - Nonconvex Extension of Generalized Huber Loss for Robust Learning and
Pseudo-Mode Statistics [0.0]
We show that using the log-exp together with the logistic function, we can create a loss combines.
We show a robust generalization that can be utilized to minimize the exponential convergence.
arXiv Detail & Related papers (2022-02-22T19:32:02Z) - Anti-Concentrated Confidence Bonuses for Scalable Exploration [57.91943847134011]
Intrinsic rewards play a central role in handling the exploration-exploitation trade-off.
We introduce emphanti-concentrated confidence bounds for efficiently approximating the elliptical bonus.
We develop a practical variant for deep reinforcement learning that is competitive with contemporary intrinsic rewards on Atari benchmarks.
arXiv Detail & Related papers (2021-10-21T15:25:15Z) - Online and Distribution-Free Robustness: Regression and Contextual
Bandits with Huber Contamination [29.85468294601847]
We revisit two classic high-dimensional online learning problems, namely linear regression and contextual bandits.
We show that our algorithms succeed where conventional methods fail.
arXiv Detail & Related papers (2020-10-08T17:59:05Z) - Approximation Schemes for ReLU Regression [80.33702497406632]
We consider the fundamental problem of ReLU regression.
The goal is to output the best fitting ReLU with respect to square loss given to draws from some unknown distribution.
arXiv Detail & Related papers (2020-05-26T16:26:17Z) - Nonlinear Functional Output Regression: a Dictionary Approach [1.8160945635344528]
We introduce projection learning (PL), a novel dictionary-based approach that learns to predict a function that is expanded on a dictionary.
PL minimizes an empirical risk based on a functional loss.
PL enjoys a particularily attractive trade-off between computational cost and performances.
arXiv Detail & Related papers (2020-03-03T10:31:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.