Benchmarking Object Detectors under Real-World Distribution Shifts in Satellite Imagery

• URL: http://arxiv.org/abs/2503.19202v1
• Date: Mon, 24 Mar 2025 23:04:06 GMT
• Title: Benchmarking Object Detectors under Real-World Distribution Shifts in Satellite Imagery
• Authors: Sara Al-Emadi, Yin Yang, Ferda Ofli,
• Abstract summary: Domain Generalisation (DG) seeks to bridge the gap by enabling models to generalise to Out-Of-Distribution (OOD) data without access to target distributions during training.<n>We examine the generalisability and robustness of state-of-the-art object detectors under real-world distribution shifts.<n>We introduce Real-World Distribution Shifts (RWDS), a suite of three novel DG benchmarking datasets.
• Score: 9.723791642707738
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Object detectors have achieved remarkable performance in many applications; however, these deep learning models are typically designed under the i.i.d. assumption, meaning they are trained and evaluated on data sampled from the same (source) distribution. In real-world deployment, however, target distributions often differ from source data, leading to substantial performance degradation. Domain Generalisation (DG) seeks to bridge this gap by enabling models to generalise to Out-Of-Distribution (OOD) data without access to target distributions during training, enhancing robustness to unseen conditions. In this work, we examine the generalisability and robustness of state-of-the-art object detectors under real-world distribution shifts, focusing particularly on spatial domain shifts. Despite the need, a standardised benchmark dataset specifically designed for assessing object detection under realistic DG scenarios is currently lacking. To address this, we introduce Real-World Distribution Shifts (RWDS), a suite of three novel DG benchmarking datasets that focus on humanitarian and climate change applications. These datasets enable the investigation of domain shifts across (i) climate zones and (ii) various disasters and geographic regions. To our knowledge, these are the first DG benchmarking datasets tailored for object detection in real-world, high-impact contexts. We aim for these datasets to serve as valuable resources for evaluating the robustness and generalisation of future object detection models. Our datasets and code are available at https://github.com/RWGAI/RWDS.

Related papers

• Style-Adaptive Detection Transformer for Single-Source Domain Generalized Object Detection [7.768332621617199]
  Single-source Domain Generalization (SDG) in object detection aims to develop a detector using only data from a source domain. We propose an object-aware contrastive learning module to guide the detector in extracting domain-invariant features. Experiments demonstrate the superior performance and generalization capability of SA-DETR across five different weather scenarios.
  arXiv  Detail & Related papers  (2025-04-29T07:38:37Z)
• Object Style Diffusion for Generalized Object Detection in Urban Scene [69.04189353993907]
  We introduce a novel single-domain object detection generalization method, named GoDiff.<n>By integrating pseudo-target domain data with source domain data, we diversify the training dataset.<n> Experimental results demonstrate that our method not only enhances the generalization ability of existing detectors but also functions as a plug-and-play enhancement for other single-domain generalization methods.
  arXiv  Detail & Related papers  (2024-12-18T13:03:00Z)
• Distribution Shifts at Scale: Out-of-distribution Detection in Earth Observation [4.457854503856095]
  Out-of-distribution (OOD) detection addresses the challenge of identifying inputs that differ from in-distribution (ID) data.<n>We propose TARDIS, a post-hoc OOD detection method for scalable geospatial deployments.<n>Our method takes a pre-trained model, ID data WILD samples, and WILD samples, disentangling the latter into surrogate ID and surrogate OOD labels.<n>To demonstrate scalability, we deploy TARDIS on the Fields of the World dataset, offering actionable insights into pre-trained model behavior for large-scale deployments.
  arXiv  Detail & Related papers  (2024-12-18T00:10:44Z)
• HiGDA: Hierarchical Graph of Nodes to Learn Local-to-Global Topology for Semi-Supervised Domain Adaptation [0.18749305679160366]
  We introduce a Hierarchical Graph of Nodes designed to simultaneously present representations at both feature and category levels.<n>In this study, we introduce a local graph to identify the most relevant patches within an image, facilitating adaptability to defined main object representations.<n>At the category level, we employ a global graph to aggregate the features from samples within the same category, thereby enriching overall representations.
  arXiv  Detail & Related papers  (2024-12-16T14:35:52Z)
• Non-stationary Domain Generalization: Theory and Algorithm [11.781050299571692]
  In this paper, we study domain generalization in non-stationary environment. We first examine the impact of environmental non-stationarity on model performance. Then, we propose a novel algorithm based on adaptive invariant representation learning.
  arXiv  Detail & Related papers  (2024-05-10T21:32:43Z)
• Innovative Horizons in Aerial Imagery: LSKNet Meets DiffusionDet for Advanced Object Detection [55.2480439325792]
  We present an in-depth evaluation of an object detection model that integrates the LSKNet backbone with the DiffusionDet head. The proposed model achieves a mean average precision (MAP) of approximately 45.7%, which is a significant improvement. This advancement underscores the effectiveness of the proposed modifications and sets a new benchmark in aerial image analysis.
  arXiv  Detail & Related papers  (2023-11-21T19:49:13Z)
• One-Shot Domain Adaptive and Generalizable Semantic Segmentation with Class-Aware Cross-Domain Transformers [96.51828911883456]
  Unsupervised sim-to-real domain adaptation (UDA) for semantic segmentation aims to improve the real-world test performance of a model trained on simulated data. Traditional UDA often assumes that there are abundant unlabeled real-world data samples available during training for the adaptation. We explore the one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization problem, where only one real-world data sample is available.
  arXiv  Detail & Related papers  (2022-12-14T15:54:15Z)
• Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives [16.68091981866261]
  Unsupervised domain adaptation (UDA) is proposed to counter the performance drop on data in a target domain. UDA has yielded promising results on natural image processing, video analysis, natural language processing, time-series data analysis, medical image analysis, etc.
  arXiv  Detail & Related papers  (2022-08-15T20:05:07Z)
• On Certifying and Improving Generalization to Unseen Domains [87.00662852876177]
  Domain Generalization aims to learn models whose performance remains high on unseen domains encountered at test-time. It is challenging to evaluate DG algorithms comprehensively using a few benchmark datasets. We propose a universal certification framework that can efficiently certify the worst-case performance of any DG method.
  arXiv  Detail & Related papers  (2022-06-24T16:29:43Z)
• Collaborative Semantic Aggregation and Calibration for Federated Domain Generalization [28.573872986524794]
  DG aims to learn from multiple known source domains a model that can generalize well to unknown target domains. In this paper, we tackle the problem of federated domain generalization where the source datasets can only be accessed locally. We conduct data-free semantic aggregation by fusing the models trained on separated domains layer-by-layer.
  arXiv  Detail & Related papers  (2021-10-13T14:08:29Z)
• Salient Objects in Clutter [130.63976772770368]
  This paper identifies and addresses a serious design bias of existing salient object detection (SOD) datasets. This design bias has led to a saturation in performance for state-of-the-art SOD models when evaluated on existing datasets. We propose a new high-quality dataset and update the previous saliency benchmark.
  arXiv  Detail & Related papers  (2021-05-07T03:49:26Z)
• BREEDS: Benchmarks for Subpopulation Shift [98.90314444545204]
  We develop a methodology for assessing the robustness of models to subpopulation shift. We leverage the class structure underlying existing datasets to control the data subpopulations that comprise the training and test distributions. Applying this methodology to the ImageNet dataset, we create a suite of subpopulation shift benchmarks of varying granularity.
  arXiv  Detail & Related papers  (2020-08-11T17:04:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.