Semantic Driven Energy based Out-of-Distribution Detection

• URL: http://arxiv.org/abs/2208.10787v1
• Date: Tue, 23 Aug 2022 07:40:34 GMT
• Title: Semantic Driven Energy based Out-of-Distribution Detection
• Authors: Abhishek Joshi, Sathish Chalasani, Kiran Nanjunda Iyer
• Abstract summary: Energy based OOD methods have proved to be promising and achieved impressive performance. We propose semantic driven energy based method, which is an end-to-end trainable system and easy to optimize. We find that, our novel approach enhances outlier detection and achieve state-of-the-art as an energy-based model on common benchmarks.
• Score: 0.4640835690336652
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Detecting Out-of-Distribution (OOD) samples in real world visual applications like classification or object detection has become a necessary precondition in today's deployment of Deep Learning systems. Many techniques have been proposed, of which Energy based OOD methods have proved to be promising and achieved impressive performance. We propose semantic driven energy based method, which is an end-to-end trainable system and easy to optimize. We distinguish in-distribution samples from out-distribution samples with an energy score coupled with a representation score. We achieve it by minimizing the energy for in-distribution samples and simultaneously learn respective class representations that are closer and maximizing energy for out-distribution samples and pushing their representation further out from known class representation. Moreover, we propose a novel loss function which we call Cluster Focal Loss(CFL) that proved to be simple yet very effective in learning better class wise cluster center representations. We find that, our novel approach enhances outlier detection and achieve state-of-the-art as an energy-based model on common benchmarks. On CIFAR-10 and CIFAR-100 trained WideResNet, our model significantly reduces the relative average False Positive Rate(at True Positive Rate of 95%) by 67.2% and 57.4% respectively, compared to the existing energy based approaches. Further, we extend our framework for object detection and achieve improved performance.

Related papers

• Shedding More Light on Robust Classifiers under the lens of Energy-based Models [3.953603590878949]
  We offer a new take on the dynamics of adversarial training (AT) Our analysis of the energy landscape during AT reveals that untargeted attacks generate adversarial images much more in-distribution (lower energy) than the original data from the point of view of the model. Motivated by rigorous evidence, we propose Weighted Energy Adversarial Training (WEAT)
  arXiv  Detail & Related papers  (2024-07-08T18:31:19Z)
• Iterated Denoising Energy Matching for Sampling from Boltzmann Densities [109.23137009609519]
  Iterated Denoising Energy Matching (iDEM) iDEM alternates between (I) sampling regions of high model density from a diffusion-based sampler and (II) using these samples in our matching objective. We show that the proposed approach achieves state-of-the-art performance on all metrics and trains $2-5times$ faster.
  arXiv  Detail & Related papers  (2024-02-09T01:11:23Z)
• Mean-AP Guided Reinforced Active Learning for Object Detection [31.304039641225504]
  This paper introduces Mean-AP Guided Reinforced Active Learning for Object Detection (MGRAL) MGRAL is a novel approach that leverages the concept of expected model output changes as informativeness for deep detection networks. Our approach demonstrates strong performance, establishing a new paradigm in reinforcement learning-based active learning for object detection.
  arXiv  Detail & Related papers  (2023-10-12T14:59:22Z)
• Glocal Energy-based Learning for Few-Shot Open-Set Recognition [57.84234213466372]
  Few-shot open-set recognition (FSOR) is a challenging task of great practical value. We propose a novel energy-based hybrid model for FSOR. Experiments on three standard FSOR datasets show the superior performance of our model.
  arXiv  Detail & Related papers  (2023-04-24T07:06:50Z)
• Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective [142.36200080384145]
  We propose a single objective which jointly optimize a latent-space model and policy to achieve high returns while remaining self-consistent. We demonstrate that the resulting algorithm matches or improves the sample-efficiency of the best prior model-based and model-free RL methods.
  arXiv  Detail & Related papers  (2022-09-18T03:51:58Z)
• Fake It Till You Make It: Near-Distribution Novelty Detection by Score-Based Generative Models [54.182955830194445]
  existing models either fail or face a dramatic drop under the so-called near-distribution" setting. We propose to exploit a score-based generative model to produce synthetic near-distribution anomalous data. Our method improves the near-distribution novelty detection by 6% and passes the state-of-the-art by 1% to 5% across nine novelty detection benchmarks.
  arXiv  Detail & Related papers  (2022-05-28T02:02:53Z)
• Energy-Efficient Adaptive Machine Learning on IoT End-Nodes With Class-Dependent Confidence [22.225875583595027]
  An effective way to obtain energy-efficiency with small accuracy drops is to sequentially execute a set of increasingly complex models. Current methods employ a single threshold on the output probabilities produced by each model. We show that our method can significantly reduce the energy consumption compared to the single-threshold approach.
  arXiv  Detail & Related papers  (2022-04-07T13:22:52Z)
• Energy-Efficient and Federated Meta-Learning via Projected Stochastic Gradient Ascent [79.58680275615752]
  We propose an energy-efficient federated meta-learning framework. We assume each task is owned by a separate agent, so a limited number of tasks is used to train a meta-model.
  arXiv  Detail & Related papers  (2021-05-31T08:15:44Z)
• DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation [57.358212277226315]
  In imitation learning from observation IfO, a learning agent seeks to imitate a demonstrating agent using only observations of the demonstrated behavior without access to the control signals generated by the demonstrator. Recent methods based on adversarial imitation learning have led to state-of-the-art performance on IfO problems, but they typically suffer from high sample complexity due to a reliance on data-inefficient, model-free reinforcement learning algorithms. This issue makes them impractical to deploy in real-world settings, where gathering samples can incur high costs in terms of time, energy, and risk. We propose a more data-efficient IfO algorithm
  arXiv  Detail & Related papers  (2021-03-31T23:46:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.