D2AF: A Dual-Driven Annotation and Filtering Framework for Visual Grounding

• URL: http://arxiv.org/abs/2505.24372v1
• Date: Fri, 30 May 2025 09:04:47 GMT
• Title: D2AF: A Dual-Driven Annotation and Filtering Framework for Visual Grounding
• Abstract summary: D2AF is a robust annotation framework for visual grounding using only input images.<n>By implementing dual-driven annotation strategies, we effectively generate detailed region-text pairs.<n>Our findings demonstrate that increasing data volume enhances model performance.
• Score: 36.321156992727055
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual Grounding is a task that aims to localize a target region in an image based on a free-form natural language description. With the rise of Transformer architectures, there is an increasing need for larger datasets to boost performance. However, the high cost of manual annotation poses a challenge, hindering the scale of data and the ability of large models to enhance their effectiveness. Previous pseudo label generation methods heavily rely on human-labeled captions of the original dataset, limiting scalability and diversity. To address this, we propose D2AF, a robust annotation framework for visual grounding using only input images. This approach overcomes dataset size limitations and enriches both the quantity and diversity of referring expressions. Our approach leverages multimodal large models and object detection models. By implementing dual-driven annotation strategies, we effectively generate detailed region-text pairs using both closed-set and open-set approaches. We further conduct an in-depth analysis of data quantity and data distribution. Our findings demonstrate that increasing data volume enhances model performance. However, the degree of improvement depends on how well the pseudo labels broaden the original data distribution. Based on these insights, we propose a consistency and distribution aware filtering method to further improve data quality by effectively removing erroneous and redundant data. This approach effectively eliminates noisy data, leading to improved performance. Experiments on three visual grounding tasks demonstrate that our method significantly improves the performance of existing models and achieves state-of-the-art results.

Related papers

• DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning [64.33887406863899]
  Multimodal deception detection aims to identify deceptive behavior by analyzing audiovisual cues for forensics and security.<n>Existing benchmarks provide only binary labels without intermediate reasoning cues.<n>We construct reasoning datasets with structured cue-level descriptions and reasoning chains.<n>With 1695 samples, it is the largest non-laboratory deception detection dataset.
  arXiv  Detail & Related papers  (2026-03-25T04:06:36Z)
• Learning Latent Representations for Image Translation using Frequency Distributed CycleGAN [7.610968152027164]
  Fd-CycleGAN is an image-to-image (I2I) translation framework that enhances latent representation learning to approximate real data distributions.<n>We conduct experiments on diverse datasets -- Horse2Zebra, Monet2Photo, and a synthetically augmented Strike-off dataset.<n>Our results suggest that frequency-guided latent learning significantly improves generalization in image translation tasks.
  arXiv  Detail & Related papers  (2025-08-05T12:59:37Z)
• Towards Explainable Bilingual Multimodal Misinformation Detection and Localization [64.37162720126194]
  BiMi is a framework that jointly performs region-level localization, cross-modal and cross-lingual consistency detection, and natural language explanation for misinformation analysis.<n>BiMiBench is a benchmark constructed by systematically editing real news images and subtitles.<n>BiMi outperforms strong baselines by up to +8.9 in classification accuracy, +15.9 in localization accuracy, and +2.5 in explanation BERTScore.
  arXiv  Detail & Related papers  (2025-06-28T15:43:06Z)
• Effective Dual-Region Augmentation for Reduced Reliance on Large Amounts of Labeled Data [1.0901840476380924]
  This paper introduces a novel dual-region augmentation approach designed to reduce reliance on large-scale labeled datasets.<n>Our method performs targeted data transformations by applying random noise perturbations to foreground objects.<n> Evaluations on the PACS dataset for SFDA demonstrate that our augmentation strategy consistently outperforms existing methods.<n>Experiments on Market-1501 and DukeMTMC-reID datasets validate the effectiveness of our approach for person ReID.
  arXiv  Detail & Related papers  (2025-04-17T16:42:33Z)
• Empowering Sparse-Input Neural Radiance Fields with Dual-Level Semantic Guidance from Dense Novel Views [66.1245505423179]
  We show that rendered semantics can be treated as a more robust form of augmented data than rendered RGB.<n>Our method enhances NeRF's performance by incorporating guidance derived from the rendered semantics.
  arXiv  Detail & Related papers  (2025-03-04T03:13:44Z)
• Boosting Salient Object Detection with Knowledge Distillated from Large Foundation Models [7.898092154590899]
  Salient Object Detection aims to identify and segment prominent regions within a scene.<n>Traditional models rely on manually annotated pseudo labels with precise pixel-level accuracy.<n>We develop a low-cost, high-precision annotation method to address the challenges.
  arXiv  Detail & Related papers  (2025-01-08T15:56:21Z)
• An Effective Deployment of Diffusion LM for Data Augmentation in Low-Resource Sentiment Classification [2.0930389307057427]
  Sentiment classification (SC) often suffers from low-resource challenges such as domain-specific contexts, imbalanced label distributions, and few-shot scenarios. We propose Diffusion LM to capture in-domain knowledge and generate pseudo samples by reconstructing strong label-related tokens.
  arXiv  Detail & Related papers  (2024-09-05T02:51:28Z)
• A Simple Background Augmentation Method for Object Detection with Diffusion Model [53.32935683257045]
  In computer vision, it is well-known that a lack of data diversity will impair model performance. We propose a simple yet effective data augmentation approach by leveraging advancements in generative models. Background augmentation, in particular, significantly improves the models' robustness and generalization capabilities.
  arXiv  Detail & Related papers  (2024-08-01T07:40:00Z)
• DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception [78.26734070960886]
  Current perceptive models heavily depend on resource-intensive datasets. We introduce perception-aware loss (P.A. loss) through segmentation, improving both quality and controllability. Our method customizes data augmentation by extracting and utilizing perception-aware attribute (P.A. Attr) during generation.
  arXiv  Detail & Related papers  (2024-03-20T04:58:03Z)
• Distribution-Aware Data Expansion with Diffusion Models [55.979857976023695]
  We propose DistDiff, a training-free data expansion framework based on the distribution-aware diffusion model. DistDiff consistently enhances accuracy across a diverse range of datasets compared to models trained solely on original data.
  arXiv  Detail & Related papers  (2024-03-11T14:07:53Z)
• D2 Pruning: Message Passing for Balancing Diversity and Difficulty in Data Pruning [70.98091101459421]
  Coreset selection seeks to select a subset of the training data so as to maximize the performance of models trained on this subset, also referred to as coreset. We propose a novel pruning algorithm, D2 Pruning, that uses forward and reverse message passing over this dataset graph for coreset selection. Results show that D2 Pruning improves coreset selection over previous state-of-the-art methods for up to 70% pruning rates.
  arXiv  Detail & Related papers  (2023-10-11T23:01:29Z)
• Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition [57.08108545219043]
  Recognizing novel sub-categories with scarce samples is an essential and challenging research topic in computer vision. Existing literature addresses this challenge by employing local-based representation approaches. This article proposes a novel model, Robust Saliency-aware Distillation (RSaD), for few-shot fine-grained visual recognition.
  arXiv  Detail & Related papers  (2023-05-12T00:13:17Z)
• Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition [94.56304526014875]
  We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER) Our method exploits self-supervised pretraining to learn good feature representations from the target data. We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
  arXiv  Detail & Related papers  (2022-10-11T08:24:50Z)
• Weakly Supervised Change Detection Using Guided Anisotropic Difusion [97.43170678509478]
  We propose original ideas that help us to leverage such datasets in the context of change detection. First, we propose the guided anisotropic diffusion (GAD) algorithm, which improves semantic segmentation results. We then show its potential in two weakly-supervised learning strategies tailored for change detection.
  arXiv  Detail & Related papers  (2021-12-31T10:03:47Z)
• Dense Contrastive Visual-Linguistic Pretraining [53.61233531733243]
  Several multimodal representation learning approaches have been proposed that jointly represent image and text. These approaches achieve superior performance by capturing high-level semantic information from large-scale multimodal pretraining. We propose unbiased Dense Contrastive Visual-Linguistic Pretraining to replace the region regression and classification with cross-modality region contrastive learning.
  arXiv  Detail & Related papers  (2021-09-24T07:20:13Z)
• Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentation [12.639524717464509]
  We propose an input-output balanced framework to handle the issue of long-tailed distribution. For the input space, we synthesize these tailed instances from mesh models and well simulate the position and density distribution of LiDAR scan. For the output space, a multi-head block is proposed to group different categories based on their shapes and instance amounts.
  arXiv  Detail & Related papers  (2021-03-26T05:42:11Z)
• MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding [74.33171794972688]
  We present algorithms to model phrase-object relevance by leveraging fine-grained visual representations and visually-aware language representations. Experiments conducted on the widely-adopted Flickr30k dataset show a significant improvement over existing weakly-supervised methods.
  arXiv  Detail & Related papers  (2020-10-12T00:43:52Z)
• Heavy-tailed Representations, Text Polarity Classification & Data Augmentation [11.624944730002298]
  We develop a novel method to learn a heavy-tailed embedding with desirable regularity properties. A classifier dedicated to the tails of the proposed embedding is obtained which performance outperforms the baseline. Numerical experiments on synthetic and real text data demonstrate the relevance of the proposed framework.
  arXiv  Detail & Related papers  (2020-03-25T19:24:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.