Interpretable Distribution Shift Detection using Optimal Transport

• URL: http://arxiv.org/abs/2208.02896v1
• Date: Thu, 4 Aug 2022 21:55:29 GMT
• Title: Interpretable Distribution Shift Detection using Optimal Transport
• Authors: Neha Hulkund, Nicolo Fusi, Jennifer Wortman Vaughan, David Alvarez-Melis
• Abstract summary: We propose a method to identify and characterize distribution shifts in classification datasets based on optimal transport. It allows the user to identify the extent to which each class is affected by the shift, and retrieves corresponding pairs of samples to provide insights on its nature.
• Score: 22.047388001308253
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a method to identify and characterize distribution shifts in classification datasets based on optimal transport. It allows the user to identify the extent to which each class is affected by the shift, and retrieves corresponding pairs of samples to provide insights on its nature. We illustrate its use on synthetic and natural shift examples. While the results we present are preliminary, we hope that this inspires future work on interpretable methods for analyzing distribution shifts.

Related papers

• Understanding Optimal Feature Transfer via a Fine-Grained Bias-Variance Analysis [10.79615566320291]
  In the transfer learning paradigm models learn useful representations (or features) during a data-rich pretraining stage, and then use the pretrained representation to improve model performance on data-scarce downstream tasks. In this work, we explore transfer learning with the goal of optimizing downstream performance.
  arXiv  Detail & Related papers  (2024-04-18T19:33:55Z)
• Identifiable Latent Neural Causal Models [82.14087963690561]
  Causal representation learning seeks to uncover latent, high-level causal representations from low-level observed data. We determine the types of distribution shifts that do contribute to the identifiability of causal representations. We translate our findings into a practical algorithm, allowing for the acquisition of reliable latent causal representations.
  arXiv  Detail & Related papers  (2024-03-23T04:13:55Z)
• Supervised Algorithmic Fairness in Distribution Shifts: A Survey [17.826312801085052]
  In real-world applications, machine learning models are often trained on a specific dataset but deployed in environments where the data distribution may shift. This shift can lead to unfair predictions, disproportionately affecting certain groups characterized by sensitive attributes, such as race and gender.
  arXiv  Detail & Related papers  (2024-02-02T11:26:18Z)
• Invariance assumptions for class distribution estimation [1.3053649021965603]
  We study the problem of class distribution estimation under dataset shift. On the training dataset, both features and class labels are observed while on the test dataset only the features can be observed.
  arXiv  Detail & Related papers  (2023-11-28T20:57:10Z)
• Predicate Classification Using Optimal Transport Loss in Scene Graph Generation [7.056402944499977]
  We propose a method to generate scene graphs using optimal transport as a measure for comparing two probability distributions. The experimental evaluation of the effectiveness demonstrates that the proposed method outperforms existing methods in terms of mean Recall@50 and 100.
  arXiv  Detail & Related papers  (2023-09-19T08:46:18Z)
• Distribution Shift Inversion for Out-of-Distribution Prediction [57.22301285120695]
  We propose a portable Distribution Shift Inversion algorithm for Out-of-Distribution (OoD) prediction. We show that our method provides a general performance gain when plugged into a wide range of commonly used OoD algorithms.
  arXiv  Detail & Related papers  (2023-06-14T08:00:49Z)
• Variational Classification [51.2541371924591]
  We derive a variational objective to train the model, analogous to the evidence lower bound (ELBO) used to train variational auto-encoders. Treating inputs to the softmax layer as samples of a latent variable, our abstracted perspective reveals a potential inconsistency. We induce a chosen latent distribution, instead of the implicit assumption found in a standard softmax layer.
  arXiv  Detail & Related papers  (2023-05-17T17:47:19Z)
• Explanation Shift: Detecting distribution shifts on tabular data via the explanation space [13.050516715665166]
  We investigate how model predictive performance and model explanation characteristics are affected under distribution shifts. We find that the modeling of explanation shifts can be a better indicator for the detection of predictive performance changes.
  arXiv  Detail & Related papers  (2022-10-22T06:47:13Z)
• Towards Explaining Distribution Shifts [9.036025934093965]
  A distribution shift can have fundamental consequences such as signaling a change in the operating environment or significantly reducing the accuracy of downstream models. Most prior work focuses on merely detecting if a shift has occurred and assumes any detected shift can be understood and handled appropriately by a human operator. We hope to aid in these manual mitigation tasks by explaining the distribution shift using interpretable transportation maps from the original distribution to the shifted one.
  arXiv  Detail & Related papers  (2022-10-19T03:38:57Z)
• Discriminative Attribution from Counterfactuals [64.94009515033984]
  We present a method for neural network interpretability by combining feature attribution with counterfactual explanations. We show that this method can be used to quantitatively evaluate the performance of feature attribution methods in an objective manner.
  arXiv  Detail & Related papers  (2021-09-28T00:53:34Z)
• Toward Scalable and Unified Example-based Explanation and Outlier Detection [128.23117182137418]
  We argue for a broader adoption of prototype-based student networks capable of providing an example-based explanation for their prediction. We show that our prototype-based networks beyond similarity kernels deliver meaningful explanations and promising outlier detection results without compromising classification accuracy.
  arXiv  Detail & Related papers  (2020-11-11T05:58:17Z)
• Generalization Properties of Optimal Transport GANs with Latent Distribution Learning [52.25145141639159]
  We study how the interplay between the latent distribution and the complexity of the pushforward map affects performance. Motivated by our analysis, we advocate learning the latent distribution as well as the pushforward map within the GAN paradigm.
  arXiv  Detail & Related papers  (2020-07-29T07:31:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.