Learning with Mixture of Prototypes for Out-of-Distribution Detection
- URL: http://arxiv.org/abs/2402.02653v1
- Date: Mon, 5 Feb 2024 00:52:50 GMT
- Title: Learning with Mixture of Prototypes for Out-of-Distribution Detection
- Authors: Haodong Lu, Dong Gong, Shuo Wang, Jason Xue, Lina Yao, Kristen Moore
- Abstract summary: Out-of-distribution (OOD) detection aims to detect testing samples far away from the in-distribution (ID) training data.
We propose PrototypicAl Learning with a Mixture of prototypes (PALM) which models each class with multiple prototypes to capture the sample diversities.
Our method achieves state-of-the-art average AUROC performance of 93.82 on the challenging CIFAR-100 benchmark.
- Score: 25.67011646236146
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Out-of-distribution (OOD) detection aims to detect testing samples far away
from the in-distribution (ID) training data, which is crucial for the safe
deployment of machine learning models in the real world. Distance-based OOD
detection methods have emerged with enhanced deep representation learning. They
identify unseen OOD samples by measuring their distances from ID class
centroids or prototypes. However, existing approaches learn the representation
relying on oversimplified data assumptions, e.g, modeling ID data of each class
with one centroid class prototype or using loss functions not designed for OOD
detection, which overlook the natural diversities within the data. Naively
enforcing data samples of each class to be compact around only one prototype
leads to inadequate modeling of realistic data and limited performance. To
tackle these issues, we propose PrototypicAl Learning with a Mixture of
prototypes (PALM) which models each class with multiple prototypes to capture
the sample diversities, and learns more faithful and compact samples embeddings
to enhance OOD detection. Our method automatically identifies and dynamically
updates prototypes, assigning each sample to a subset of prototypes via
reciprocal neighbor soft assignment weights. PALM optimizes a maximum
likelihood estimation (MLE) loss to encourage the sample embeddings to be
compact around the associated prototypes, as well as a contrastive loss on all
prototypes to enhance intra-class compactness and inter-class discrimination at
the prototype level. Moreover, the automatic estimation of prototypes enables
our approach to be extended to the challenging OOD detection task with
unlabelled ID data. Extensive experiments demonstrate the superiority of PALM,
achieving state-of-the-art average AUROC performance of 93.82 on the
challenging CIFAR-100 benchmark. Code is available at
https://github.com/jeff024/PALM.
Related papers
- Proto-OOD: Enhancing OOD Object Detection with Prototype Feature Similarity [15.518552332992035]
We leverage feature similarity for out-of-distribution (OOD) object detection.
Proto-OOD enhances prototype representativeness through contrastive loss.
It employs a negative embedding generator to create negative embedding, which are then used to train the similarity module.
arXiv Detail & Related papers (2024-09-09T09:48:27Z) - Querying Easily Flip-flopped Samples for Deep Active Learning [63.62397322172216]
Active learning is a machine learning paradigm that aims to improve the performance of a model by strategically selecting and querying unlabeled data.
One effective selection strategy is to base it on the model's predictive uncertainty, which can be interpreted as a measure of how informative a sample is.
This paper proposes the it least disagree metric (LDM) as the smallest probability of disagreement of the predicted label.
arXiv Detail & Related papers (2024-01-18T08:12:23Z) - Decoupled Prototype Learning for Reliable Test-Time Adaptation [50.779896759106784]
Test-time adaptation (TTA) is a task that continually adapts a pre-trained source model to the target domain during inference.
One popular approach involves fine-tuning model with cross-entropy loss according to estimated pseudo-labels.
This study reveals that minimizing the classification error of each sample causes the cross-entropy loss's vulnerability to label noise.
We propose a novel Decoupled Prototype Learning (DPL) method that features prototype-centric loss computation.
arXiv Detail & Related papers (2024-01-15T03:33:39Z) - Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection
Capability [70.72426887518517]
Out-of-distribution (OOD) detection is an indispensable aspect of secure AI when deploying machine learning models in real-world applications.
We propose a novel method, Unleashing Mask, which aims to restore the OOD discriminative capabilities of the well-trained model with ID data.
Our method utilizes a mask to figure out the memorized atypical samples, and then finetune the model or prune it with the introduced mask to forget them.
arXiv Detail & Related papers (2023-06-06T14:23:34Z) - TTAPS: Test-Time Adaption by Aligning Prototypes using Self-Supervision [70.05605071885914]
We propose a novel modification of the self-supervised training algorithm SwAV that adds the ability to adapt to single test samples.
We show the success of our method on the common benchmark dataset CIFAR10-C.
arXiv Detail & Related papers (2022-05-18T05:43:06Z) - How to Exploit Hyperspherical Embeddings for Out-of-Distribution
Detection? [22.519572587827213]
CIDER is a representation learning framework that exploits hyperspherical embeddings for OOD detection.
CIDER establishes superior performance, outperforming the latest rival by 19.36% in FPR95.
arXiv Detail & Related papers (2022-03-08T23:44:01Z) - EARLIN: Early Out-of-Distribution Detection for Resource-efficient
Collaborative Inference [4.826988182025783]
Collaborative inference enables resource-constrained edge devices to make inferences by uploading inputs to a server.
While this setup works cost-effectively for successful inferences, it severely underperforms when the model faces input samples on which the model was not trained.
We propose a novel lightweight OOD detection approach that mines important features from the shallow layers of a pretrained CNN model.
arXiv Detail & Related papers (2021-06-25T18:43:23Z) - Lightweight Detection of Out-of-Distribution and Adversarial Samples via
Channel Mean Discrepancy [14.103271496247551]
We introduce Channel Mean Discrepancy (CMD), a model-agnostic distance metric for evaluating the statistics of features extracted by classification models.
We experimentally demonstrate that CMD magnitude is significantly smaller for legitimate samples than for OOD and adversarial samples.
Preliminary results show that our simple yet effective method outperforms several state-of-the-art approaches to detecting OOD and adversarial samples.
arXiv Detail & Related papers (2021-04-23T04:15:53Z) - Prototypical Contrastive Learning of Unsupervised Representations [171.3046900127166]
Prototypical Contrastive Learning (PCL) is an unsupervised representation learning method.
PCL implicitly encodes semantic structures of the data into the learned embedding space.
PCL outperforms state-of-the-art instance-wise contrastive learning methods on multiple benchmarks.
arXiv Detail & Related papers (2020-05-11T09:53:36Z) - Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders [51.691585766702744]
We propose a variant of Adversarial Autoencoder which uses a mirrored Wasserstein loss in the discriminator to enforce better semantic-level reconstruction.
We put forward an alternative measure of anomaly score to replace the reconstruction-based metric.
Our method outperforms the current state-of-the-art methods for anomaly detection on several OOD detection benchmarks.
arXiv Detail & Related papers (2020-03-24T08:26:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.