A Near-Optimal Algorithm for Debiasing Trained Machine Learning Models
- URL: http://arxiv.org/abs/2106.12887v1
- Date: Sun, 6 Jun 2021 09:45:37 GMT
- Title: A Near-Optimal Algorithm for Debiasing Trained Machine Learning Models
- Authors: Ibrahim Alabdulmohsin and Mario Lucic
- Abstract summary: We present a scalable post-processing algorithm for debiasing trained models, including deep neural networks (DNNs)
We prove to be near-optimal by bounding its excess Bayes risk.
We empirically validate its advantages on standard benchmark datasets.
- Score: 21.56208997475512
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present a scalable post-processing algorithm for debiasing trained models,
including deep neural networks (DNNs), which we prove to be near-optimal by
bounding its excess Bayes risk. We empirically validate its advantages on
standard benchmark datasets across both classical algorithms as well as modern
DNN architectures and demonstrate that it outperforms previous post-processing
methods while performing on par with in-processing. In addition, we show that
the proposed algorithm is particularly effective for models trained at scale
where post-processing is a natural and practical choice.
Related papers
- Discrete Neural Algorithmic Reasoning [21.852775399735005]
We propose to force neural reasoners to maintain the execution trajectory as a combination of finite predefined states.
We get perfect test scores on the SALSA-CLRS benchmark, where we get perfect test scores for all tasks.
arXiv Detail & Related papers (2024-02-18T16:03:04Z) - The Convex Landscape of Neural Networks: Characterizing Global Optima
and Stationary Points via Lasso Models [75.33431791218302]
Deep Neural Network Network (DNN) models are used for programming purposes.
In this paper we examine the use of convex neural recovery models.
We show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program.
We also show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program.
arXiv Detail & Related papers (2023-12-19T23:04:56Z) - Quantifying uncertainty for deep learning based forecasting and
flow-reconstruction using neural architecture search ensembles [0.8258451067861933]
We present an automated approach to deep neural network (DNN) discovery and demonstrate how this may also be utilized for ensemble-based uncertainty quantification.
We highlight how the proposed method not only discovers high-performing neural network ensembles for our tasks, but also quantifies uncertainty seamlessly.
We demonstrate the feasibility of this framework for two tasks - forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature.
arXiv Detail & Related papers (2023-02-20T03:57:06Z) - Improved Algorithms for Neural Active Learning [74.89097665112621]
We improve the theoretical and empirical performance of neural-network(NN)-based active learning algorithms for the non-parametric streaming setting.
We introduce two regret metrics by minimizing the population loss that are more suitable in active learning than the one used in state-of-the-art (SOTA) related work.
arXiv Detail & Related papers (2022-10-02T05:03:38Z) - DLCFT: Deep Linear Continual Fine-Tuning for General Incremental
Learning [29.80680408934347]
We propose an alternative framework to incremental learning where we continually fine-tune the model from a pre-trained representation.
Our method takes advantage of linearization technique of a pre-trained neural network for simple and effective continual learning.
We show that our method can be applied to general continual learning settings, we evaluate our method in data-incremental, task-incremental, and class-incremental learning problems.
arXiv Detail & Related papers (2022-08-17T06:58:14Z) - Direct Localization in Underwater Acoustics via Convolutional Neural
Networks: A Data-Driven Approach [31.399611901926583]
Direct localization (DLOC) methods generally outperform their indirect two-step counterparts.
Underwater acoustic DLOC methods require prior knowledge of the environment.
We propose what is to the best of our knowledge, the first data-driven DLOC method.
arXiv Detail & Related papers (2022-07-20T22:40:11Z) - Scalable computation of prediction intervals for neural networks via
matrix sketching [79.44177623781043]
Existing algorithms for uncertainty estimation require modifying the model architecture and training procedure.
This work proposes a new algorithm that can be applied to a given trained neural network and produces approximate prediction intervals.
arXiv Detail & Related papers (2022-05-06T13:18:31Z) - Neural Combinatorial Optimization: a New Player in the Field [69.23334811890919]
This paper presents a critical analysis on the incorporation of algorithms based on neural networks into the classical optimization framework.
A comprehensive study is carried out to analyse the fundamental aspects of such algorithms, including performance, transferability, computational cost and to larger-sized instances.
arXiv Detail & Related papers (2022-05-03T07:54:56Z) - Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
There is an increasing need for active learning algorithms that are compatible with deep neural networks.
This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
arXiv Detail & Related papers (2021-06-17T17:26:31Z) - An AI-Assisted Design Method for Topology Optimization Without
Pre-Optimized Training Data [68.8204255655161]
An AI-assisted design method based on topology optimization is presented, which is able to obtain optimized designs in a direct way.
Designs are provided by an artificial neural network, the predictor, on the basis of boundary conditions and degree of filling as input data.
arXiv Detail & Related papers (2020-12-11T14:33:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.