Adversarial Reconstruction Feedback for Robust Fine-grained Generalization

• URL: http://arxiv.org/abs/2507.21742v1
• Date: Tue, 29 Jul 2025 12:20:03 GMT
• Title: Adversarial Reconstruction Feedback for Robust Fine-grained Generalization
• Authors: Shijie Wang, Jian Shi, Haojie Li,
• Abstract summary: AdvRF is a novel adversarial reconstruction feedback framework for learning category-agnostic discrepancy representations.<n>It reformulates FGIR as a visual discrepancy reconstruction task via synergizing category-aware discrepancy localization from retrieval models.<n>It achieves impressive performance on both widely-used fine-grained and coarse-grained datasets.
• Score: 29.202222418152285
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing fine-grained image retrieval (FGIR) methods predominantly rely on supervision from predefined categories to learn discriminative representations for retrieving fine-grained objects. However, they inadvertently introduce category-specific semantics into the retrieval representation, creating semantic dependencies on predefined classes that critically hinder generalization to unseen categories. To tackle this, we propose AdvRF, a novel adversarial reconstruction feedback framework aimed at learning category-agnostic discrepancy representations. Specifically, AdvRF reformulates FGIR as a visual discrepancy reconstruction task via synergizing category-aware discrepancy localization from retrieval models with category-agnostic feature learning from reconstruction models. The reconstruction model exposes residual discrepancies overlooked by the retrieval model, forcing it to improve localization accuracy, while the refined signals from the retrieval model guide the reconstruction model to improve its reconstruction ability. Consequently, the retrieval model localizes visual differences, while the reconstruction model encodes these differences into category-agnostic representations. This representation is then transferred to the retrieval model through knowledge distillation for efficient deployment. Quantitative and qualitative evaluations demonstrate that our AdvRF achieves impressive performance on both widely-used fine-grained and coarse-grained datasets.

Related papers

• Reward Models Can Improve Themselves: Reward-Guided Adversarial Failure Mode Discovery for Robust Reward Modeling [27.11560841914813]
  We introduce REFORM, a self-improving reward modeling framework that enhances robustness by using the reward model itself to guide the generation of falsely scored responses.<n>We evaluate REFORM on two widely used preference datasets Anthropic Helpful Harmless (HH) and PKU Beavertails.
  arXiv  Detail & Related papers  (2025-07-08T21:56:33Z)
• A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends [67.43992456058541]
  Image restoration (IR) aims to recover high-quality images from inputs degraded by various factors such as noise, blur, compression, and adverse weather.<n>Traditional IR methods typically focus on specific types of degradation, which limits their effectiveness in real-world scenarios with complex distortions.<n>The all-in-one image restoration paradigm has recently emerged, offering a unified framework that adeptly addresses multiple degradation types.
  arXiv  Detail & Related papers  (2024-10-19T11:11:09Z)
• Model Will Tell: Training Membership Inference for Diffusion Models [15.16244745642374]
  Training Membership Inference (TMI) task aims to determine whether a specific sample has been used in the training process of a target model. In this paper, we explore a novel perspective for the TMI task by leveraging the intrinsic generative priors within the diffusion model.
  arXiv  Detail & Related papers  (2024-03-13T12:52:37Z)
• Distillation Enhanced Generative Retrieval [96.69326099136289]
  Generative retrieval is a promising new paradigm in text retrieval that generates identifier strings of relevant passages as the retrieval target. In this work, we identify a viable direction to further enhance generative retrieval via distillation and propose a feasible framework, named DGR. We conduct experiments on four public datasets, and the results indicate that DGR achieves state-of-the-art performance among the generative retrieval methods.
  arXiv  Detail & Related papers  (2024-02-16T15:48:24Z)
• Recursive Counterfactual Deconfounding for Object Recognition [20.128093193861165]
  We propose a Recursive Counterfactual Deconfounding model for object recognition in both closed-set and open-set scenarios. We show that the proposed RCD model performs better than 11 state-of-the-art baselines significantly in most cases.
  arXiv  Detail & Related papers  (2023-09-25T07:46:41Z)
• Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstruction [75.91471250967703]
  We introduce a novel sampling framework called Steerable Conditional Diffusion.<n>This framework adapts the diffusion model, concurrently with image reconstruction, based solely on the information provided by the available measurement.<n>We achieve substantial enhancements in out-of-distribution performance across diverse imaging modalities.
  arXiv  Detail & Related papers  (2023-08-28T08:47:06Z)
• Anti-aliasing Semantic Reconstruction for Few-Shot Semantic Segmentation [66.85202434812942]
  We reformulate few-shot segmentation as a semantic reconstruction problem. We convert base class features into a series of basis vectors which span a class-level semantic space for novel class reconstruction. Our proposed approach, referred to as anti-aliasing semantic reconstruction (ASR), provides a systematic yet interpretable solution for few-shot learning problems.
  arXiv  Detail & Related papers  (2021-06-01T02:17:36Z)
• Towards Novel Target Discovery Through Open-Set Domain Adaptation [73.81537683043206]
  Open-set domain adaptation (OSDA) considers that the target domain contains samples from novel categories unobserved in external source domain. We propose a novel framework to accurately identify the seen categories in target domain, and effectively recover the semantic attributes for unseen categories.
  arXiv  Detail & Related papers  (2021-05-06T04:22:29Z)
• Recurrent Feedback Improves Recognition of Partially Occluded Objects [1.452875650827562]
  We investigate if and how artificial neural networks also benefit from recurrence. We find that classification accuracy is significantly higher for recurrent models when compared to feedforward models of matched parametric complexity.
  arXiv  Detail & Related papers  (2021-04-21T16:18:34Z)
• Understanding invariance via feedforward inversion of discriminatively trained classifiers [30.23199531528357]
  Past research has discovered that some extraneous visual detail remains in the output logits. We develop a feedforward inversion model that produces remarkably high fidelity reconstructions. Our approach is based on BigGAN, with conditioning on logits instead of one-hot class labels.
  arXiv  Detail & Related papers  (2021-03-15T17:56:06Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.