Diagnosing Aerial-View Object Detectors with Foundational Image Generative Models

• URL: http://arxiv.org/abs/2607.02718v1
• Date: Thu, 02 Jul 2026 19:11:57 GMT
• Title: Diagnosing Aerial-View Object Detectors with Foundational Image Generative Models
• Abstract summary: We introduce a synthetic diagnostic framework for aerial-view vehicle detection that combines text-guided generation, attribute-controlled editing, and automated attribute verification.<n>Our results show that controlled synthetic probing can predict real-domain performance gaps and guide efficient data collection.<n>The proposed diagnostic framework is modular and can incorporate alternative generative or vision-language models as capabilities evolve.
• Score: 45.7884146182982
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in large-scale image generative models enable photorealistic scene synthesis with controllable attributes. Beyond data augmentation, their potential as diagnostic tools for trained vision systems remains unexplored in the aerial and remote sensing domains. We introduce a synthetic diagnostic framework for aerial-view vehicle detection that combines text-guided generation, attribute-controlled editing, and automated attribute verification to construct a controllable synthetic testbed. This enables fine-grained evaluation of pretrained detectors under diverse scene types and environmental conditions that are difficult to isolate in real datasets. Across three detection architectures and three real aerial datasets, synthetic scene-wise performance trends closely match real-world weaknesses. Guided by these diagnostics, targeted supplementation with small real datasets from the identified weak categories yields improvements of up to 13% AP50 while requiring substantially fewer additional samples than non-targeted augmentation. Our results show that controlled synthetic probing can predict real-domain performance gaps and guide efficient data collection. The proposed diagnostic framework is modular and can incorporate alternative generative or vision-language models as capabilities evolve. Our code and datasets are available here: https://humansensinglab.github.io/AVODDiag/

Related papers

• A Real-Calibrated Synthetic-First Data Engine [0.0]
  Real-Calibrated Synthetic-First Data Engine is a modular data engineering framework that combines controllable diffusion generation and multi-stage curation/filtering within a unified pipeline.<n>We show that synthetic data improves a real-data baseline when used as near-zero-human-annotation-cost augmentation alongside real anchors, while synthetic-only training remains substantially below real-only performance.
  arXiv  Detail & Related papers  (2026-05-10T18:34:43Z)
• Layout-Guided Controllable Pathology Image Generation with In-Context Diffusion Transformers [57.54843029965778]
  Controllable pathology image synthesis requires reliable regulation of spatial layout, tissue morphology, and semantic detail.<n>We propose In-Context Diffusion Transformer (IC-DiT), a layout-aware generative model that incorporates spatial layouts, textual descriptions, and visual embeddings into a unified diffusion transformer.<n>IC-DiT achieves higher fidelity, stronger spatial controllability, and better diagnostic consistency than existing methods.
  arXiv  Detail & Related papers  (2026-03-11T06:14:11Z)
• Synth It Like KITTI: Synthetic Data Generation for Object Detection in Driving Scenarios [3.30184292168618]
  We propose a dataset generation pipeline based on the CARLA simulator for 3D object detection on LiDAR point clouds.<n>We are able to train an object detector on the synthetic data and demonstrate strong generalization capabilities to the KITTI dataset.
  arXiv  Detail & Related papers  (2025-02-20T22:27:42Z)
• Improving Object Detection by Modifying Synthetic Data with Explainable AI [3.0519884745675485]
  We propose a novel conceptual approach to improve the efficiency of designing synthetic images.<n>XAI techniques guide a human-in-the-loop process of modifying 3D mesh models used to generate these images.<n>We show that synthetic data can improve detection of vehicles in orientations unseen in training by 4.6%.
  arXiv  Detail & Related papers  (2024-12-02T13:24:43Z)
• AeroGen: Enhancing Remote Sensing Object Detection with Diffusion-Driven Data Generation [43.583735469794675]
  Remote sensing image object detection (RSIOD) aims to identify and locate specific objects within satellite or aerial imagery.<n>There is a scarcity of labeled data in current RSIOD datasets, which significantly limits the performance of current detection algorithms.<n>This paper proposes a layout-controllable diffusion generative model (i.e. AeroGen) tailored for RSIOD.
  arXiv  Detail & Related papers  (2024-11-23T09:04:33Z)
• BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation [57.40024206484446]
  We introduce the BEHAVIOR Vision Suite (BVS), a set of tools and assets to generate fully customized synthetic data for systematic evaluation of computer vision models. BVS supports a large number of adjustable parameters at the scene level. We showcase three example application scenarios.
  arXiv  Detail & Related papers  (2024-05-15T17:57:56Z)
• DODA: Adapting Object Detectors to Dynamic Agricultural Environments in Real-Time with Diffusion [4.549305421261851]
  DODA is a diffusion-based framework that can adapt the detector to a new domain in just 2 minutes.<n>We demonstrate DODA's effectiveness on the Global Wheat Head Detection dataset, where fine-tuning detectors on DODA-generated data yields significant improvements.
  arXiv  Detail & Related papers  (2024-03-27T08:16:33Z)
• DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception [78.26734070960886]
  Current perceptive models heavily depend on resource-intensive datasets. We introduce perception-aware loss (P.A. loss) through segmentation, improving both quality and controllability. Our method customizes data augmentation by extracting and utilizing perception-aware attribute (P.A. Attr) during generation.
  arXiv  Detail & Related papers  (2024-03-20T04:58:03Z)
• InstaGen: Enhancing Object Detection by Training on Synthetic Dataset [59.445498550159755]
  We present a novel paradigm to enhance the ability of object detector, e.g., expanding categories or improving detection performance. We integrate an instance-level grounding head into a pre-trained, generative diffusion model, to augment it with the ability of localising instances in the generated images. We conduct thorough experiments to show that, this enhanced version of diffusion model, termed as InstaGen, can serve as a data synthesizer.
  arXiv  Detail & Related papers  (2024-02-08T18:59:53Z)
• UAV-Sim: NeRF-based Synthetic Data Generation for UAV-based Perception [62.71374902455154]
  We leverage recent advancements in neural rendering to improve static and dynamic novelview UAV-based image rendering. We demonstrate a considerable performance boost when a state-of-the-art detection model is optimized primarily on hybrid sets of real and synthetic data.
  arXiv  Detail & Related papers  (2023-10-25T00:20:37Z)
• Domain Generalization via Ensemble Stacking for Face Presentation Attack Detection [4.61143637299349]
  Face Presentation Attack Detection (PAD) plays a pivotal role in securing face recognition systems against spoofing attacks. This work proposes a comprehensive solution that combines synthetic data generation and deep ensemble learning. Experimental results on four datasets demonstrate low half total error rates (HTERs) on three benchmark datasets.
  arXiv  Detail & Related papers  (2023-01-05T16:44:36Z)
• Generative Modeling Helps Weak Supervision (and Vice Versa) [87.62271390571837]
  We propose a model fusing weak supervision and generative adversarial networks. It captures discrete variables in the data alongside the weak supervision derived label estimate. It is the first approach to enable data augmentation through weakly supervised synthetic images and pseudolabels.
  arXiv  Detail & Related papers  (2022-03-22T20:24:21Z)
• Synthetic Data and Hierarchical Object Detection in Overhead Imagery [0.0]
  We develop novel synthetic data generation and augmentation techniques for enhancing low/zero-sample learning in satellite imagery. To test the effectiveness of synthetic imagery, we employ it in the training of detection models and our two stage model, and evaluate the resulting models on real satellite images.
  arXiv  Detail & Related papers  (2021-01-29T22:52:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.