Adaptive Robust Learning using Latent Bernoulli Variables

• URL: http://arxiv.org/abs/2312.00585v2
• Date: Fri, 14 Jun 2024 12:19:30 GMT
• Title: Adaptive Robust Learning using Latent Bernoulli Variables
• Authors: Aleksandr Karakulev, Dave Zachariah, Prashant Singh,
• Abstract summary: We present an adaptive approach for learning from corrupted training sets. We identify corrupted non-corrupted samples with latent Bernoulli variables. The resulting problem is solved via variational inference.
• Score: 50.223140145910904
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present an adaptive approach for robust learning from corrupted training sets. We identify corrupted and non-corrupted samples with latent Bernoulli variables and thus formulate the learning problem as maximization of the likelihood where latent variables are marginalized. The resulting problem is solved via variational inference, using an efficient Expectation-Maximization based method. The proposed approach improves over the state-of-the-art by automatically inferring the corruption level, while adding minimal computational overhead. We demonstrate our robust learning method and its parameter-free nature on a wide variety of machine learning tasks including online learning and deep learning where it adapts to different levels of noise and maintains high prediction accuracy.

Related papers

• Stabilizing Linear Passive-Aggressive Online Learning with Weighted Reservoir Sampling [46.01254613933967]
  Online learning methods are still highly effective for high-dimensional streaming data, out-of-core processing, and other throughput-sensitive applications. Many such algorithms rely on fast adaptation to individual errors as a key to their convergence. While such algorithms enjoy low theoretical regret, in real-world deployment they can be sensitive to individual outliers that cause the algorithm to over-correct.
  arXiv  Detail & Related papers  (2024-10-31T03:35:48Z)
• Robust Deep Reinforcement Learning with Adaptive Adversarial Perturbations in Action Space [3.639580365066386]
  We propose an adaptive adversarial coefficient framework to adjust the effect of the adversarial perturbation during training. The appealing feature of our method is that it is simple to deploy in real-world applications and does not require accessing the simulator in advance. The experiments in MuJoCo show that our method can improve the training stability and learn a robust policy when migrated to different test environments.
  arXiv  Detail & Related papers  (2024-05-20T12:31:11Z)
• Primal Dual Continual Learning: Balancing Stability and Plasticity through Adaptive Memory Allocation [86.8475564814154]
  We show that it is both possible and beneficial to undertake the constrained optimization problem directly. We focus on memory-based methods, where a small subset of samples from previous tasks can be stored in a replay buffer. We show that dual variables indicate the sensitivity of the optimal value of the continual learning problem with respect to constraint perturbations.
  arXiv  Detail & Related papers  (2023-09-29T21:23:27Z)
• Variational Self-Supervised Contrastive Learning Using Beta Divergence [0.0]
  We present a contrastive self-supervised learning method which is robust to data noise, grounded in the domain of variational methods. We demonstrate the effectiveness of the proposed method through rigorous experiments including linear evaluation and fine-tuning scenarios with multi-label datasets in the face understanding domain.
  arXiv  Detail & Related papers  (2023-09-05T17:21:38Z)
• Robust Deep Reinforcement Learning Scheduling via Weight Anchoring [7.570246812206769]
  We use weight anchoring to cultivate and fixate desired behavior in Neural Networks. Weight anchoring may be used to find a solution to a learning problem that is nearby the solution of another learning problem. Results show that this method provides performance comparable to the state of the art of augmenting a simulation environment.
  arXiv  Detail & Related papers  (2023-04-20T09:30:23Z)
• A Boosting Approach to Reinforcement Learning [59.46285581748018]
  We study efficient algorithms for reinforcement learning in decision processes whose complexity is independent of the number of states. We give an efficient algorithm that is capable of improving the accuracy of such weak learning methods.
  arXiv  Detail & Related papers  (2021-08-22T16:00:45Z)
• Attribute-Guided Adversarial Training for Robustness to Natural Perturbations [64.35805267250682]
  We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space. Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations.
  arXiv  Detail & Related papers  (2020-12-03T10:17:30Z)
• Learning perturbation sets for robust machine learning [97.6757418136662]
  We use a conditional generator that defines the perturbation set over a constrained region of the latent space. We measure the quality of our learned perturbation sets both quantitatively and qualitatively. We leverage our learned perturbation sets to train models which are empirically and certifiably robust to adversarial image corruptions and adversarial lighting variations.
  arXiv  Detail & Related papers  (2020-07-16T16:39:54Z)
• Truncated Inference for Latent Variable Optimization Problems: Application to Robust Estimation and Learning [32.08441889054456]
  We propose two formally justified methods to remove the need to maintain the latent variables. These methods have applications in large scale robust estimation and in learning energy-based models from labeled data.
  arXiv  Detail & Related papers  (2020-03-12T16:32:06Z)
• Learning with Differentiable Perturbed Optimizers [54.351317101356614]
  We propose a systematic method to transform operations into operations that are differentiable and never locally constant. Our approach relies on perturbeds, and can be used readily together with existing solvers. We show how this framework can be connected to a family of losses developed in structured prediction, and give theoretical guarantees for their use in learning tasks.
  arXiv  Detail & Related papers  (2020-02-20T11:11:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.