Towards Reliable AI Model Deployments: Multiple Input Mixup for Out-of-Distribution Detection

• URL: http://arxiv.org/abs/2312.15514v1
• Date: Sun, 24 Dec 2023 15:31:51 GMT
• Title: Towards Reliable AI Model Deployments: Multiple Input Mixup for Out-of-Distribution Detection
• Authors: Dasol Choi, Dongbin Na
• Abstract summary: We propose a novel and simple method to solve the Out-of-Distribution (OOD) detection problem. Our method can help improve the OOD detection performance with only single epoch fine-tuning. Our method does not require training the model from scratch and can be attached to the classifier simply.
• Score: 4.985768723667418
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent remarkable success in the deep-learning industries has unprecedentedly increased the need for reliable model deployment. For example, the model should alert the user if the produced model outputs might not be reliable. Previous studies have proposed various methods to solve the Out-of-Distribution (OOD) detection problem, however, they generally require a burden of resources. In this work, we propose a novel and simple method, Multiple Input Mixup (MIM). Our method can help improve the OOD detection performance with only single epoch fine-tuning. Our method does not require training the model from scratch and can be attached to the classifier simply. Despite its simplicity, our MIM shows competitive performance. Our method can be suitable for various environments because our method only utilizes the In-Distribution (ID) samples to generate the synthesized OOD data. With extensive experiments with CIFAR10 and CIFAR100 benchmarks that have been largely adopted in out-of-distribution detection fields, we have demonstrated our MIM shows comprehensively superior performance compared to the SOTA method. Especially, our method does not need additional computation on the feature vectors compared to the previous studies. All source codes are publicly available at https://github.com/ndb796/MultipleInputMixup.

Related papers

• Out-of-Distribution Detection with a Single Unconditional Diffusion Model [54.15132801131365]
  Out-of-distribution (OOD) detection is a critical task in machine learning that seeks to identify abnormal samples. Traditionally, unsupervised methods utilize a deep generative model for OOD detection. This paper explores whether a single model can perform OOD detection across diverse tasks.
  arXiv  Detail & Related papers  (2024-05-20T08:54:03Z)
• Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models [15.50128790503447]
  We propose a novel and theoretically motivated methodology for pre-training data detection, named Min-K%++. Specifically, we present a key insight that training samples tend to be local maxima of the modeled distribution along each input dimension through likelihood training.
  arXiv  Detail & Related papers  (2024-04-03T04:25:01Z)
• Enhancing Out-of-Distribution Detection with Multitesting-based Layer-wise Feature Fusion [11.689517005768046]
  Out-of-distribution samples may exhibit shifts in local or global features compared to the training distribution. We propose a novel framework, Multitesting-based Layer-wise Out-of-Distribution (OOD) Detection. Our scheme effectively enhances the performance of out-of-distribution detection when compared to baseline methods.
  arXiv  Detail & Related papers  (2024-03-16T04:35:04Z)
• Calibrated Out-of-Distribution Detection with a Generic Representation [28.658200157111505]
  Out-of-distribution detection is a common issue in deploying vision models in practice and solving it is an essential building block in safety critical applications. We propose a novel OOD method, called GROOD, that formulates the OOD detection as a Neyman-Pearson task with well calibrated scores and which achieves excellent performance. The method achieves state-of-the-art performance on a number of OOD benchmarks, reaching near perfect performance on several of them.
  arXiv  Detail & Related papers  (2023-03-23T10:03:12Z)
• Cross-Modal Fine-Tuning: Align then Refine [83.37294254884446]
  ORCA is a cross-modal fine-tuning framework that extends the applicability of a single large-scale pretrained model to diverse modalities. We show that ORCA obtains state-of-the-art results on 3 benchmarks containing over 60 datasets from 12 modalities.
  arXiv  Detail & Related papers  (2023-02-11T16:32:28Z)
• Boosting Out-of-Distribution Detection with Multiple Pre-trained Models [41.66566916581451]
  Post hoc detection utilizing pre-trained models has shown promising performance and can be scaled to large-scale problems. We propose a detection enhancement method by ensembling multiple detection decisions derived from a zoo of pre-trained models. Our method substantially improves the relative performance by 65.40% and 26.96% on the CIFAR10 and ImageNet benchmarks.
  arXiv  Detail & Related papers  (2022-12-24T12:11:38Z)
• Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuning [85.55727213502402]
  We focus on improving the few-shot performance of prompt tuning by transferring knowledge from soft prompts of source tasks. We propose Sample-specific Ensemble of Source Models (SESoM) SESoM learns to adjust the contribution of each source model for each target sample separately when ensembling source model outputs.
  arXiv  Detail & Related papers  (2022-10-23T01:33:16Z)
• MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation [68.30497162547768]
  We propose MoEBERT, which uses a Mixture-of-Experts structure to increase model capacity and inference speed. We validate the efficiency and effectiveness of MoEBERT on natural language understanding and question answering tasks.
  arXiv  Detail & Related papers  (2022-04-15T23:19:37Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• EARLIN: Early Out-of-Distribution Detection for Resource-efficient Collaborative Inference [4.826988182025783]
  Collaborative inference enables resource-constrained edge devices to make inferences by uploading inputs to a server. While this setup works cost-effectively for successful inferences, it severely underperforms when the model faces input samples on which the model was not trained. We propose a novel lightweight OOD detection approach that mines important features from the shallow layers of a pretrained CNN model.
  arXiv  Detail & Related papers  (2021-06-25T18:43:23Z)
• Unsupervised Multi-source Domain Adaptation Without Access to Source Data [58.551861130011886]
  Unsupervised Domain Adaptation (UDA) aims to learn a predictor model for an unlabeled domain by transferring knowledge from a separate labeled source domain. We propose a novel and efficient algorithm which automatically combines the source models with suitable weights in such a way that it performs at least as good as the best source model.
  arXiv  Detail & Related papers  (2021-04-05T10:45:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.