PDiscoNet: Semantically consistent part discovery for fine-grained recognition

• URL: http://arxiv.org/abs/2309.03173v1
• Date: Wed, 6 Sep 2023 17:19:29 GMT
• Title: PDiscoNet: Semantically consistent part discovery for fine-grained recognition
• Authors: Robert van der Klis, Stephan Alaniz, Massimiliano Mancini, Cassio F. Dantas, Dino Ienco, Zeynep Akata, Diego Marcos
• Abstract summary: We propose PDiscoNet to discover object parts by using only image-level class labels along with priors encouraging the parts to be. Our results on CUB, CelebA, and PartImageNet show that the proposed method provides substantially better part discovery performance than previous methods.
• Score: 62.12602920807109
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fine-grained classification often requires recognizing specific object parts, such as beak shape and wing patterns for birds. Encouraging a fine-grained classification model to first detect such parts and then using them to infer the class could help us gauge whether the model is indeed looking at the right details better than with interpretability methods that provide a single attribution map. We propose PDiscoNet to discover object parts by using only image-level class labels along with priors encouraging the parts to be: discriminative, compact, distinct from each other, equivariant to rigid transforms, and active in at least some of the images. In addition to using the appropriate losses to encode these priors, we propose to use part-dropout, where full part feature vectors are dropped at once to prevent a single part from dominating in the classification, and part feature vector modulation, which makes the information coming from each part distinct from the perspective of the classifier. Our results on CUB, CelebA, and PartImageNet show that the proposed method provides substantially better part discovery performance than previous methods while not requiring any additional hyper-parameter tuning and without penalizing the classification performance. The code is available at https://github.com/robertdvdk/part_detection.

Related papers

• Rethinking Person Re-identification from a Projection-on-Prototypes Perspective [84.24742313520811]
  Person Re-IDentification (Re-ID) as a retrieval task, has achieved tremendous development over the past decade. We propose a new baseline ProNet, which innovatively reserves the function of the classifier at the inference stage. Experiments on four benchmarks demonstrate that our proposed ProNet is simple yet effective, and significantly beats previous baselines.
  arXiv  Detail & Related papers  (2023-08-21T13:38:10Z)
• Unicom: Universal and Compact Representation Learning for Image Retrieval [65.96296089560421]
  We cluster the large-scale LAION400M into one million pseudo classes based on the joint textual and visual features extracted by the CLIP model. To alleviate such conflict, we randomly select partial inter-class prototypes to construct the margin-based softmax loss. Our method significantly outperforms state-of-the-art unsupervised and supervised image retrieval approaches on multiple benchmarks.
  arXiv  Detail & Related papers  (2023-04-12T14:25:52Z)
• Sanity checks and improvements for patch visualisation in prototype-based image classification [0.0]
  We perform an in-depth analysis of the visualisation methods implemented in two popular self-explaining models for visual classification based on prototypes. We first show that such methods do not correctly identify the regions of interest inside of the images, and therefore do not reflect the model behaviour. We discuss the implications of our findings for other prototype-based models sharing the same visualisation method.
  arXiv  Detail & Related papers  (2023-01-20T15:13:04Z)
• Interclass Prototype Relation for Few-Shot Segmentation [0.0]
  With few-shot segmentation, the target class data distribution in the feature space is sparse and has low coverage because of the slight variations in the sample data. This study proposes the Interclass Prototype Relation Network (IPRNet) which improves the separation performance by reducing the similarity between other classes.
  arXiv  Detail & Related papers  (2022-11-16T05:27:52Z)
• Rethinking Semantic Segmentation: A Prototype View [126.59244185849838]
  We present a nonparametric semantic segmentation model based on non-learnable prototypes. Our framework yields compelling results over several datasets. We expect this work will provoke a rethink of the current de facto semantic segmentation model design.
  arXiv  Detail & Related papers  (2022-03-28T21:15:32Z)
• CAR: Class-aware Regularizations for Semantic Segmentation [20.947897583427192]
  We propose a universal Class-Aware Regularization (CAR) approach to optimize the intra-class variance and inter-class distance during feature learning. Our method can be easily applied to most existing segmentation models during training, including OCR and CPNet.
  arXiv  Detail & Related papers  (2022-03-14T15:02:48Z)
• APANet: Adaptive Prototypes Alignment Network for Few-Shot Semantic Segmentation [56.387647750094466]
  Few-shot semantic segmentation aims to segment novel-class objects in a given query image with only a few labeled support images. Most advanced solutions exploit a metric learning framework that performs segmentation through matching each query feature to a learned class-specific prototype. We present an adaptive prototype representation by introducing class-specific and class-agnostic prototypes.
  arXiv  Detail & Related papers  (2021-11-24T04:38:37Z)
• Unsupervised Part Discovery via Feature Alignment [15.67978793872039]
  We exploit the property that neural network features are largely invariant to nuisance variables. We find a set of similar images that show instances of the same object category in the same pose, through an affine alignment of their corresponding feature maps. During inference, part detection is simple and fast, without any extra modules or overheads other than a feed-forward neural network.
  arXiv  Detail & Related papers  (2020-12-01T07:25:00Z)
• Part-aware Prototype Network for Few-shot Semantic Segmentation [50.581647306020095]
  We propose a novel few-shot semantic segmentation framework based on the prototype representation. Our key idea is to decompose the holistic class representation into a set of part-aware prototypes. We develop a novel graph neural network model to generate and enhance the proposed part-aware prototypes.
  arXiv  Detail & Related papers  (2020-07-13T11:03:09Z)
• End-to-end Learning of a Fisher Vector Encoding for Part Features in Fine-grained Recognition [10.423464288613275]
  We assume that part-based methods suffer from a missing representation of local features. We propose integrating a Fisher vector encoding of part features into convolutional neural networks. Our approach improves state-of-the-art accuracies for three bird species classification datasets.
  arXiv  Detail & Related papers  (2020-07-04T12:17:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.