Signal to Noise Ratio Loss Function
- URL: http://arxiv.org/abs/2110.12275v1
- Date: Sat, 23 Oct 2021 18:44:32 GMT
- Title: Signal to Noise Ratio Loss Function
- Authors: Ali Ghobadzadeh and Amir Lashkari
- Abstract summary: This work proposes a new loss function targeting classification problems.
We derive a series of tightest upper and lower bounds for the probability of a random variable in a given interval.
A lower bound is proposed for the probability of a true positive for a parametric classification problem.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This work proposes a new loss function targeting classification problems,
utilizing a source of information overlooked by cross entropy loss. First, we
derive a series of the tightest upper and lower bounds for the probability of a
random variable in a given interval. Second, a lower bound is proposed for the
probability of a true positive for a parametric classification problem, where
the form of probability density function (pdf) of data is given. A closed form
for finding the optimal function of unknowns is derived to maximize the
probability of true positives. Finally, for the case that the pdf of data is
unknown, we apply the proposed boundaries to find the lower bound of the
probability of true positives and upper bound of the probability of false
positives and optimize them using a loss function which is given by combining
the boundaries. We demonstrate that the resultant loss function is a function
of the signal to noise ratio both within and across logits. We empirically
evaluate our proposals to show their benefit for classification problems.
Related papers
- Distribution-Free Inference for the Regression Function of Binary
Classification [0.0]
The paper presents a resampling framework to construct exact, distribution-free and non-asymptotically guaranteed confidence regions for the true regression function for any user-chosen confidence level.
It is proved that the constructed confidence regions are strongly consistent, that is, any false model is excluded in the long run with probability one.
arXiv Detail & Related papers (2023-08-03T15:52:27Z) - Kernel-based off-policy estimation without overlap: Instance optimality
beyond semiparametric efficiency [53.90687548731265]
We study optimal procedures for estimating a linear functional based on observational data.
For any convex and symmetric function class $mathcalF$, we derive a non-asymptotic local minimax bound on the mean-squared error.
arXiv Detail & Related papers (2023-01-16T02:57:37Z) - Data-Driven Influence Functions for Optimization-Based Causal Inference [105.5385525290466]
We study a constructive algorithm that approximates Gateaux derivatives for statistical functionals by finite differencing.
We study the case where probability distributions are not known a priori but need to be estimated from data.
arXiv Detail & Related papers (2022-08-29T16:16:22Z) - Inference on Strongly Identified Functionals of Weakly Identified
Functions [71.42652863687117]
We study a novel condition for the functional to be strongly identified even when the nuisance function is not.
We propose penalized minimax estimators for both the primary and debiasing nuisance functions.
arXiv Detail & Related papers (2022-08-17T13:38:31Z) - Statistical Properties of the log-cosh Loss Function Used in Machine Learning [0.0]
We present the distribution function from which the log-cosh loss arises.
We also examine the use of the log-cosh function for quantile regression.
arXiv Detail & Related papers (2022-08-09T07:03:58Z) - Variational Transport: A Convergent Particle-BasedAlgorithm for Distributional Optimization [106.70006655990176]
A distributional optimization problem arises widely in machine learning and statistics.
We propose a novel particle-based algorithm, dubbed as variational transport, which approximately performs Wasserstein gradient descent.
We prove that when the objective function satisfies a functional version of the Polyak-Lojasiewicz (PL) (Polyak, 1963) and smoothness conditions, variational transport converges linearly.
arXiv Detail & Related papers (2020-12-21T18:33:13Z) - Bayesian Optimization with Missing Inputs [53.476096769837724]
We develop a new acquisition function based on the well-known Upper Confidence Bound (UCB) acquisition function.
We conduct comprehensive experiments on both synthetic and real-world applications to show the usefulness of our method.
arXiv Detail & Related papers (2020-06-19T03:56:27Z) - All your loss are belong to Bayes [28.393499629583786]
Loss functions are a cornerstone of machine learning and the starting point of most algorithms.
We introduce a trick on squared Gaussian Processes to obtain a random process whose paths are compliant source functions.
Experimental results demonstrate substantial improvements over the state of the art.
arXiv Detail & Related papers (2020-06-08T14:31:21Z) - Approximation Schemes for ReLU Regression [80.33702497406632]
We consider the fundamental problem of ReLU regression.
The goal is to output the best fitting ReLU with respect to square loss given to draws from some unknown distribution.
arXiv Detail & Related papers (2020-05-26T16:26:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.