PCNN: Probable-Class Nearest-Neighbor Explanations Improve Fine-Grained Image Classification Accuracy for AIs and Humans

• URL: http://arxiv.org/abs/2308.13651v5
• Date: Mon, 26 Aug 2024 21:11:26 GMT
• Title: PCNN: Probable-Class Nearest-Neighbor Explanations Improve Fine-Grained Image Classification Accuracy for AIs and Humans
• Authors: Giang, Nguyen, Valerie Chen, Mohammad Reza Taesiri, Anh Totti Nguyen,
• Abstract summary: Nearest neighbors (NN) are traditionally used to compute final decisions. In this paper, we show a novel utility of nearest neighbors: To improve predictions of a frozen, pretrained image classifier C. Our method consistently improves fine-grained image classification accuracy on CUB-200, Cars-196, and Dogs-120.
• Score: 7.655550161309149
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Nearest neighbors (NN) are traditionally used to compute final decisions, e.g., in Support Vector Machines or k-NN classifiers, and to provide users with explanations for the model's decision. In this paper, we show a novel utility of nearest neighbors: To improve predictions of a frozen, pretrained image classifier C. We leverage an image comparator S that (1) compares the input image with NN images from the top-K most probable classes given by C; and (2) uses scores from S to weight the confidence scores of C to refine predictions. Our method consistently improves fine-grained image classification accuracy on CUB-200, Cars-196, and Dogs-120. Also, a human study finds that showing users our probable-class nearest neighbors (PCNN) reduces over-reliance on AI, thus improving their decision accuracy over prior work which only shows only the most-probable (top-1) class examples.

Related papers

• Image-Caption Encoding for Improving Zero-Shot Generalization [12.906307770270026]
  We show that when an OOD data point is misclassified, the correct class can be typically found in the Top-K predicted classes. In order to steer the model prediction toward the correct class within the top predicted classes, we propose the Image-Caption (ICE) method. Our method can be easily combined with other SOTA methods to enhance Top-1 OOD accuracies by 0.5% on average and up to 3% on challenging datasets.
  arXiv  Detail & Related papers  (2024-02-05T01:14:07Z)
• RankDNN: Learning to Rank for Few-shot Learning [70.49494297554537]
  This paper introduces a new few-shot learning pipeline that casts relevance ranking for image retrieval as binary ranking relation classification. It provides a new perspective on few-shot learning and is complementary to state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-11-28T13:59:31Z)
• Visual correspondence-based explanations improve AI robustness and human-AI team accuracy [7.969008943697552]
  We propose two novel architectures of self-interpretable image classifiers that first explain, and then predict. Our models consistently improve (by 1 to 4 points) on out-of-distribution (OOD) datasets. For the first time, we show that it is possible to achieve complementary human-AI team accuracy (i.e., that is higher than either AI-alone or human-alone) in ImageNet and CUB image classification tasks.
  arXiv  Detail & Related papers  (2022-07-26T10:59:42Z)
• Rethinking Nearest Neighbors for Visual Classification [56.00783095670361]
  k-NN is a lazy learning method that aggregates the distance between the test image and top-k neighbors in a training set. We adopt k-NN with pre-trained visual representations produced by either supervised or self-supervised methods in two steps. Via extensive experiments on a wide range of classification tasks, our study reveals the generality and flexibility of k-NN integration.
  arXiv  Detail & Related papers  (2021-12-15T20:15:01Z)
• KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier [61.063988689601416]
  Pre-trained models are widely used in fine-tuning downstream tasks with linear classifiers optimized by the cross-entropy loss. These problems can be improved by learning representations that focus on similarities in the same class and contradictions when making predictions. We introduce the KNearest Neighbors in pre-trained model fine-tuning tasks in this paper.
  arXiv  Detail & Related papers  (2021-10-06T06:17:05Z)
• Overhead-MNIST: Machine Learning Baselines for Image Classification [0.0]
  Twenty-three machine learning algorithms were trained then scored to establish baseline comparison metrics. The Overhead-MNIST dataset is a collection of satellite images similar in style to the ubiquitous MNIST hand-written digits. We present results for the overall best performing algorithm as a baseline for edge deployability and future performance improvement.
  arXiv  Detail & Related papers  (2021-07-01T13:30:39Z)
• Instance-based learning using the Half-Space Proximal Graph [0.0]
  This paper presents a parameter-free instance-based learning algorithm using the em Half-Space Proximal (HSP) graph. The resulting classifier bettered $KNN$ for any $k$ in a battery of datasets.
  arXiv  Detail & Related papers  (2021-02-04T17:36:59Z)
• Almost Tight L0-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations [78.23408201652984]
  Top-k predictions are used in many real-world applications such as machine learning as a service, recommender systems, and web searches. Our work is based on randomized smoothing, which builds a provably robust classifier via randomizing an input. For instance, our method can build a classifier that achieves a certified top-3 accuracy of 69.2% on ImageNet when an attacker can arbitrarily perturb 5 pixels of a testing image.
  arXiv  Detail & Related papers  (2020-11-15T21:34:44Z)
• SCAN: Learning to Classify Images without Labels [73.69513783788622]
  We advocate a two-step approach where feature learning and clustering are decoupled. A self-supervised task from representation learning is employed to obtain semantically meaningful features. We obtain promising results on ImageNet, and outperform several semi-supervised learning methods in the low-data regime.
  arXiv  Detail & Related papers  (2020-05-25T18:12:33Z)
• I Am Going MAD: Maximum Discrepancy Competition for Comparing Classifiers Adaptively [135.7695909882746]
  We name the MAximum Discrepancy (MAD) competition. We adaptively sample a small test set from an arbitrarily large corpus of unlabeled images. Human labeling on the resulting model-dependent image sets reveals the relative performance of the competing classifiers.
  arXiv  Detail & Related papers  (2020-02-25T03:32:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.