The information of attribute uncertainties: what convolutional neural networks can learn about errors in input data

• URL: http://arxiv.org/abs/2108.04742v1
• Date: Tue, 10 Aug 2021 15:10:46 GMT
• Title: The information of attribute uncertainties: what convolutional neural networks can learn about errors in input data
• Authors: Nat\'alia V. N. Rodrigues, L. Raul Abramo, Nina S. Hirata
• Abstract summary: We show how Convolutional Neural Networks (CNNs) are able to learn about the context and patterns of signal and noise. We show that, when each data point is subject to different levels of noise, that information can be learned by the CNNs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Errors in measurements are key to weighting the value of data, but are often neglected in Machine Learning (ML). We show how Convolutional Neural Networks (CNNs) are able to learn about the context and patterns of signal and noise, leading to improvements in the performance of classification methods. We construct a model whereby two classes of objects follow an underlying Gaussian distribution, and where the features (the input data) have varying, but known, levels of noise. This model mimics the nature of scientific data sets, where the noises arise as realizations of some random processes whose underlying distributions are known. The classification of these objects can then be performed using standard statistical techniques (e.g., least-squares minimization or Markov-Chain Monte Carlo), as well as ML techniques. This allows us to take advantage of a maximum likelihood approach to object classification, and to measure the amount by which the ML methods are incorporating the information in the input data uncertainties. We show that, when each data point is subject to different levels of noise (i.e., noises with different distribution functions), that information can be learned by the CNNs, raising the ML performance to at least the same level of the least-squares method -- and sometimes even surpassing it. Furthermore, we show that, with varying noise levels, the confidence of the ML classifiers serves as a proxy for the underlying cumulative distribution function, but only if the information about specific input data uncertainties is provided to the CNNs.

Related papers

• A Closer Look at Benchmarking Self-Supervised Pre-training with Image Classification [51.35500308126506]
  Self-supervised learning (SSL) is a machine learning approach where the data itself provides supervision, eliminating the need for external labels. We study how classification-based evaluation protocols for SSL correlate and how well they predict downstream performance on different dataset types.
  arXiv  Detail & Related papers  (2024-07-16T23:17:36Z)
• Learning with Noisy Foundation Models [95.50968225050012]
  This paper is the first work to comprehensively understand and analyze the nature of noise in pre-training datasets. We propose a tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise and improve generalization.
  arXiv  Detail & Related papers  (2024-03-11T16:22:41Z)
• SEMRes-DDPM: Residual Network Based Diffusion Modelling Applied to Imbalanced Data [9.969882349165745]
  In the field of data mining and machine learning, commonly used classification models cannot effectively learn in unbalanced data. Most of the classical oversampling methods are based on the SMOTE technique, which only focuses on the local information of the data. We propose a novel oversampling method SEMRes-DDPM.
  arXiv  Detail & Related papers  (2024-03-09T14:01:04Z)
• Learning from aggregated data with a maximum entropy model [73.63512438583375]
  We show how a new model, similar to a logistic regression, may be learned from aggregated data only by approximating the unobserved feature distribution with a maximum entropy hypothesis. We present empirical evidence on several public datasets that the model learned this way can achieve performances comparable to those of a logistic model trained with the full unaggregated data.
  arXiv  Detail & Related papers  (2022-10-05T09:17:27Z)
• Mutual Information Learned Classifiers: an Information-theoretic Viewpoint of Training Deep Learning Classification Systems [9.660129425150926]
  Cross entropy loss can easily lead us to find models which demonstrate severe overfitting behavior. In this paper, we prove that the existing cross entropy loss minimization for training DNN classifiers essentially learns the conditional entropy of the underlying data distribution. We propose a mutual information learning framework where we train DNN classifiers via learning the mutual information between the label and input.
  arXiv  Detail & Related papers  (2022-10-03T15:09:19Z)
• Few-Shot Non-Parametric Learning with Deep Latent Variable Model [50.746273235463754]
  We propose Non-Parametric learning by Compression with Latent Variables (NPC-LV) NPC-LV is a learning framework for any dataset with abundant unlabeled data but very few labeled ones. We show that NPC-LV outperforms supervised methods on all three datasets on image classification in low data regime.
  arXiv  Detail & Related papers  (2022-06-23T09:35:03Z)
• Transformers Can Do Bayesian Inference [56.99390658880008]
  We present Prior-Data Fitted Networks (PFNs) PFNs leverage in-context learning in large-scale machine learning techniques to approximate a large set of posteriors. We demonstrate that PFNs can near-perfectly mimic Gaussian processes and also enable efficient Bayesian inference for intractable problems.
  arXiv  Detail & Related papers  (2021-12-20T13:07:39Z)
• Estimating g-Leakage via Machine Learning [34.102705643128004]
  This paper considers the problem of estimating the information leakage of a system in the black-box scenario. It is assumed that the system's internals are unknown to the learner, or anyway too complicated to analyze. We propose a novel approach to perform black-box estimation of the g-vulnerability using Machine Learning (ML) algorithms.
  arXiv  Detail & Related papers  (2020-05-09T09:26:36Z)
• Learning with Out-of-Distribution Data for Audio Classification [60.48251022280506]
  We show that detecting and relabelling certain OOD instances, rather than discarding them, can have a positive effect on learning. The proposed method is shown to improve the performance of convolutional neural networks by a significant margin.
  arXiv  Detail & Related papers  (2020-02-11T21:08:06Z)
• Multi-class Gaussian Process Classification with Noisy Inputs [2.362412515574206]
  In some situations, the amount of noise can be known before-hand. We have evaluated the proposed methods by carrying out several experiments, involving synthetic and real data.
  arXiv  Detail & Related papers  (2020-01-28T18:55:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.