Detail Reinforcement Diffusion Model: Augmentation Fine-Grained Visual Categorization in Few-Shot Conditions

• URL: http://arxiv.org/abs/2309.08097v2
• Date: Wed, 15 May 2024 07:51:37 GMT
• Title: Detail Reinforcement Diffusion Model: Augmentation Fine-Grained Visual Categorization in Few-Shot Conditions
• Authors: Tianxu Wu, Shuo Ye, Shuhuang Chen, Qinmu Peng, Xinge You,
• Abstract summary: Diffusion models have been widely adopted in data augmentation due to their outstanding diversity in data generation. We propose a novel approach termed the detail reinforcement diffusion model(DRDM) It leverages the rich knowledge of large models for fine-grained data augmentation and comprises two key components including discriminative semantic recombination (DSR) and spatial knowledge reference(SKR)
• Score: 11.121652649243119
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The challenge in fine-grained visual categorization lies in how to explore the subtle differences between different subclasses and achieve accurate discrimination. Previous research has relied on large-scale annotated data and pre-trained deep models to achieve the objective. However, when only a limited amount of samples is available, similar methods may become less effective. Diffusion models have been widely adopted in data augmentation due to their outstanding diversity in data generation. However, the high level of detail required for fine-grained images makes it challenging for existing methods to be directly employed. To address this issue, we propose a novel approach termed the detail reinforcement diffusion model~(DRDM), which leverages the rich knowledge of large models for fine-grained data augmentation and comprises two key components including discriminative semantic recombination (DSR) and spatial knowledge reference~(SKR). Specifically, DSR is designed to extract implicit similarity relationships from the labels and reconstruct the semantic mapping between labels and instances, which enables better discrimination of subtle differences between different subclasses. Furthermore, we introduce the SKR module, which incorporates the distributions of different datasets as references in the feature space. This allows the SKR to aggregate the high-dimensional distribution of subclass features in few-shot FGVC tasks, thus expanding the decision boundary. Through these two critical components, we effectively utilize the knowledge from large models to address the issue of data scarcity, resulting in improved performance for fine-grained visual recognition tasks. Extensive experiments demonstrate the consistent performance gain offered by our DRDM.

Related papers

• Enhancing Unsupervised Sentence Embeddings via Knowledge-Driven Data Augmentation and Gaussian-Decayed Contrastive Learning [37.54523122932728]
  We propose a pipeline-based data augmentation method via large language models (LLMs) To tackle the issue of low data diversity, our pipeline utilizes knowledge graphs (KGs) to extract entities and quantities. To address high data noise, the GCSE model uses a Gaussian-decayed function to limit the impact of false hard negative samples.
  arXiv  Detail & Related papers  (2024-09-19T16:29:58Z)
• On Learning Discriminative Features from Synthesized Data for Self-Supervised Fine-Grained Visual Recognition [21.137498023391178]
  Self-Supervised Learning (SSL) has become a prominent approach for acquiring visual representations across various tasks. We introduce a novel strategy that boosts SSL's ability to extract critical discriminative features vital for fine-grained visual recognition. This approach creates synthesized data pairs to guide the model to focus on discriminative features critical for FGVR.
  arXiv  Detail & Related papers  (2024-07-19T21:43:19Z)
• Exploring Data Efficiency in Zero-Shot Learning with Diffusion Models [38.36200871549062]
  Zero-Shot Learning (ZSL) aims to enable classifiers to identify unseen classes by enhancing data efficiency at the class level. This is achieved by generating image features from pre-defined semantics of unseen classes. In this paper, we demonstrate that limited seen examples generally result in deteriorated performance of generative models. This unified framework incorporates diffusion models to improve data efficiency at both the class and instance levels.
  arXiv  Detail & Related papers  (2024-06-05T04:37:06Z)
• Distribution-Aware Data Expansion with Diffusion Models [55.979857976023695]
  We propose DistDiff, a training-free data expansion framework based on the distribution-aware diffusion model. DistDiff consistently enhances accuracy across a diverse range of datasets compared to models trained solely on original data.
  arXiv  Detail & Related papers  (2024-03-11T14:07:53Z)
• Bridging Generative and Discriminative Models for Unified Visual Perception with Diffusion Priors [56.82596340418697]
  We propose a simple yet effective framework comprising a pre-trained Stable Diffusion (SD) model containing rich generative priors, a unified head (U-head) capable of integrating hierarchical representations, and an adapted expert providing discriminative priors. Comprehensive investigations unveil potential characteristics of Vermouth, such as varying granularity of perception concealed in latent variables at distinct time steps and various U-net stages. The promising results demonstrate the potential of diffusion models as formidable learners, establishing their significance in furnishing informative and robust visual representations.
  arXiv  Detail & Related papers  (2024-01-29T10:36:57Z)
• DiffAugment: Diffusion based Long-Tailed Visual Relationship Recognition [43.01467525231004]
  We introduce DiffAugment -- a method which augments the tail classes in the linguistic space by making use of WordNet. We demonstrate the effectiveness of hardness-aware diffusion in generating visual embeddings for the tail classes. We also propose a novel subject and object based seeding strategy for diffusion sampling which improves the discriminative capability of the generated visual embeddings.
  arXiv  Detail & Related papers  (2024-01-01T21:20:43Z)
• RGM: A Robust Generalizable Matching Model [49.60975442871967]
  We propose a deep model for sparse and dense matching, termed RGM (Robust Generalist Matching) To narrow the gap between synthetic training samples and real-world scenarios, we build a new, large-scale dataset with sparse correspondence ground truth. We are able to mix up various dense and sparse matching datasets, significantly improving the training diversity.
  arXiv  Detail & Related papers  (2023-10-18T07:30:08Z)
• Leveraging Diffusion Disentangled Representations to Mitigate Shortcuts in Underspecified Visual Tasks [92.32670915472099]
  We propose an ensemble diversification framework exploiting the generation of synthetic counterfactuals using Diffusion Probabilistic Models (DPMs) We show that diffusion-guided diversification can lead models to avert attention from shortcut cues, achieving ensemble diversity performance comparable to previous methods requiring additional data collection.
  arXiv  Detail & Related papers  (2023-10-03T17:37:52Z)
• Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models [77.83923746319498]
  We propose a framework called Diff-Instruct to instruct the training of arbitrary generative models. We show that Diff-Instruct results in state-of-the-art single-step diffusion-based models. Experiments on refining GAN models show that the Diff-Instruct can consistently improve the pre-trained generators of GAN models.
  arXiv  Detail & Related papers  (2023-05-29T04:22:57Z)
• R2-Trans:Fine-Grained Visual Categorization with Redundancy Reduction [21.11038841356125]
  Fine-grained visual categorization (FGVC) aims to discriminate similar subcategories, whose main challenge is the large intraclass diversities and subtle inter-class differences. We present a novel approach for FGVC, which can simultaneously make use of partial yet sufficient discriminative information in environmental cues and also compress the redundant information in class-token with respect to the target.
  arXiv  Detail & Related papers  (2022-04-21T13:35:38Z)
• Adversarial Feature Hallucination Networks for Few-Shot Learning [84.31660118264514]
  Adversarial Feature Hallucination Networks (AFHN) is based on conditional Wasserstein Generative Adversarial networks (cWGAN) Two novel regularizers are incorporated into AFHN to encourage discriminability and diversity of the synthesized features.
  arXiv  Detail & Related papers  (2020-03-30T02:43:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.