Towards Optimal Feature-Shaping Methods for Out-of-Distribution Detection

• URL: http://arxiv.org/abs/2402.00865v1
• Date: Thu, 1 Feb 2024 18:59:22 GMT
• Title: Towards Optimal Feature-Shaping Methods for Out-of-Distribution Detection
• Authors: Qinyu Zhao, Ming Xu, Kartik Gupta, Akshay Asthana, Liang Zheng, Stephen Gould
• Abstract summary: Feature shaping is a family of methods that exhibit state-of-the-art performance for out-of-distribution (OOD) detection. We propose a concrete reduction of the framework with a simple piecewise constant shaping function. We show that our method improves the generalization ability of OOD detection across a large variety of datasets.
• Score: 36.316424348556154
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Feature shaping refers to a family of methods that exhibit state-of-the-art performance for out-of-distribution (OOD) detection. These approaches manipulate the feature representation, typically from the penultimate layer of a pre-trained deep learning model, so as to better differentiate between in-distribution (ID) and OOD samples. However, existing feature-shaping methods usually employ rules manually designed for specific model architectures and OOD datasets, which consequently limit their generalization ability. To address this gap, we first formulate an abstract optimization framework for studying feature-shaping methods. We then propose a concrete reduction of the framework with a simple piecewise constant shaping function and show that existing feature-shaping methods approximate the optimal solution to the concrete optimization problem. Further, assuming that OOD data is inaccessible, we propose a formulation that yields a closed-form solution for the piecewise constant shaping function, utilizing solely the ID data. Through extensive experiments, we show that the feature-shaping function optimized by our method improves the generalization ability of OOD detection across a large variety of datasets and model architectures.

Related papers

• Sparse Modelling for Feature Learning in High Dimensional Data [0.0]
  This paper presents an innovative approach to dimensionality reduction and feature extraction in high-dimensional datasets. The proposed framework integrates sparse modeling techniques into a comprehensive pipeline for efficient and interpretable feature selection. We aim to advance the understanding and application of sparse modeling in machine learning, particularly in the context of wood surface defect detection.
  arXiv  Detail & Related papers  (2024-09-28T14:17:59Z)
• Efficient Fairness-Performance Pareto Front Computation [51.558848491038916]
  We show that optimal fair representations possess several useful structural properties. We then show that these approxing problems can be solved efficiently via concave programming methods.
  arXiv  Detail & Related papers  (2024-09-26T08:46:48Z)
• Learning Non-Linear Invariants for Unsupervised Out-of-Distribution Detection [5.019613806273252]
  We propose a framework consisting of a normalizing flow-like architecture capable of learning non-linear invariants. Our approach achieves state-of-the-art results on an extensive U-OOD benchmark.
  arXiv  Detail & Related papers  (2024-07-04T16:01:21Z)
• Provably Better Explanations with Optimized Aggregation of Feature Attributions [36.22433695108499]
  Using feature attributions for post-hoc explanations is a common practice to understand and verify the predictions of opaque machine learning models. We propose a novel approach to derive optimal convex combinations of feature attributions that yield provable improvements of desired quality criteria.
  arXiv  Detail & Related papers  (2024-06-07T17:03:43Z)
• Embedded feature selection in LSTM networks with multi-objective evolutionary ensemble learning for time series forecasting [49.1574468325115]
  We present a novel feature selection method embedded in Long Short-Term Memory networks. Our approach optimize the weights and biases of the LSTM in a partitioned manner. Experimental evaluations on air quality time series data from Italy and southeast Spain demonstrate that our method substantially improves the ability generalization of conventional LSTMs.
  arXiv  Detail & Related papers  (2023-12-29T08:42:10Z)
• A General Framework for Sample-Efficient Function Approximation in Reinforcement Learning [132.45959478064736]
  We propose a general framework that unifies model-based and model-free reinforcement learning. We propose a novel estimation function with decomposable structural properties for optimization-based exploration. Under our framework, a new sample-efficient algorithm namely OPtimization-based ExploRation with Approximation (OPERA) is proposed.
  arXiv  Detail & Related papers  (2022-09-30T17:59:16Z)
• Learning Distributionally Robust Models at Scale via Composite Optimization [45.47760229170775]
  We show how different variants of DRO are simply instances of a finite-sum composite optimization for which we provide scalable methods. We also provide empirical results that demonstrate the effectiveness of our proposed algorithm with respect to the prior art in order to learn robust models from very large datasets.
  arXiv  Detail & Related papers  (2022-03-17T20:47:42Z)
• Conservative Objective Models for Effective Offline Model-Based Optimization [78.19085445065845]
  Computational design problems arise in a number of settings, from synthetic biology to computer architectures. We propose a method that learns a model of the objective function that lower bounds the actual value of the ground-truth objective on out-of-distribution inputs. COMs are simple to implement and outperform a number of existing methods on a wide range of MBO problems.
  arXiv  Detail & Related papers  (2021-07-14T17:55:28Z)
• Deep Magnification-Flexible Upsampling over 3D Point Clouds [103.09504572409449]
  We propose a novel end-to-end learning-based framework to generate dense point clouds. We first formulate the problem explicitly, which boils down to determining the weights and high-order approximation errors. Then, we design a lightweight neural network to adaptively learn unified and sorted weights as well as the high-order refinements.
  arXiv  Detail & Related papers  (2020-11-25T14:00:18Z)
• Deep Keypoint-Based Camera Pose Estimation with Geometric Constraints [80.60538408386016]
  Estimating relative camera poses from consecutive frames is a fundamental problem in visual odometry. We propose an end-to-end trainable framework consisting of learnable modules for detection, feature extraction, matching and outlier rejection.
  arXiv  Detail & Related papers  (2020-07-29T21:41:31Z)
• IVFS: Simple and Efficient Feature Selection for High Dimensional Topology Preservation [33.424663018395684]
  We propose a simple and effective feature selection algorithm to enhance sample similarity preservation. The proposed algorithm is able to well preserve the pairwise distances, as well as topological patterns, of the full data.
  arXiv  Detail & Related papers  (2020-04-02T23:05:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.