Bias Detection and Rotation-Robustness Mitigation in Vision-Language Models and Generative Image Models

• URL: http://arxiv.org/abs/2601.08860v1
• Date: Fri, 09 Jan 2026 00:36:11 GMT
• Title: Bias Detection and Rotation-Robustness Mitigation in Vision-Language Models and Generative Image Models
• Authors: Tarannum Mithila,
• Abstract summary: Vision-Language Models (VLMs) and generative image models have achieved remarkable performance across multimodal tasks.<n>This work investigates bias propagation and robustness in state-of-the-art vision-language and generative models.<n>We propose rotation-robust mitigation strategies that combine data augmentation, representation alignment, and model-level regularization.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language Models (VLMs) and generative image models have achieved remarkable performance across multimodal tasks, yet their robustness and fairness under input transformations remain insufficiently explored. This work investigates bias propagation and robustness degradation in state-of-the-art vision-language and generative models, with a particular focus on image rotation and distributional shifts. We analyze how rotation-induced perturbations affect model predictions, confidence calibration, and demographic bias patterns. To address these issues, we propose rotation-robust mitigation strategies that combine data augmentation, representation alignment, and model-level regularization. Experimental results across multiple datasets demonstrate that the proposed methods significantly improve robustness while reducing bias amplification without sacrificing overall performance. This study highlights critical limitations of current multimodal systems and provides practical mitigation techniques for building more reliable and fair AI models.

Related papers

• Did Models Sufficient Learn? Attribution-Guided Training via Subset-Selected Counterfactual Augmentation [61.248535801314375]
  Subset-Selected Counterfactual Augmentation (SS-CA)<n>We develop Counterfactual LIMA to identify minimal spatial region sets whose removal can selectively alter model predictions.<n>Experiments show that SS-CA improves generalization on in-distribution (ID) test data and achieves superior performance on out-of-distribution (OOD) benchmarks.
  arXiv  Detail & Related papers  (2025-11-15T08:39:22Z)
• Diffusion Models for Low-Light Image Enhancement: A Multi-Perspective Taxonomy and Performance Analysis [8.323736085126386]
  Low-light image enhancement (LLIE) is vital for safety-critical applications such as surveillance, autonomous navigation, and medical imaging.<n> diffusion models have emerged as a promising generative paradigm for LLIE due to their capacity to model complex image distributions via iterative denoising.<n>This survey aims to guide the next generation of diffusion-based LLIE research by highlighting trends and surfacing open research questions.
  arXiv  Detail & Related papers  (2025-10-07T14:30:36Z)
• Learning Robust Diffusion Models from Imprecise Supervision [75.53546939251146]
  DMIS is a unified framework for training robust Conditional Diffusion Models from Imprecise Supervision.<n>Our framework is derived from likelihood and decomposes the objective into generative and classification components.<n>Experiments on diverse forms of imprecise supervision, covering tasks covering image generation, weakly supervised learning, and dataset condensation demonstrate that DMIS consistently produces high-quality and class-discriminative samples.
  arXiv  Detail & Related papers  (2025-10-03T14:00:32Z)
• HRR: Hierarchical Retrospection Refinement for Generated Image Detection [16.958383381415445]
  We propose a diffusion model-based generative image detection framework termed Hierarchical Retrospection Refinement(HRR)<n>The HRR framework consistently delivers significant performance improvements, outperforming state-of-the-art methods in generated image detection task.
  arXiv  Detail & Related papers  (2025-02-25T05:13:44Z)
• Adversarial Transferability in Deep Denoising Models: Theoretical Insights and Robustness Enhancement via Out-of-Distribution Typical Set Sampling [6.189440665620872]
  Deep learning-based image denoising models demonstrate remarkable performance, but their lack of robustness analysis remains a significant concern.<n>A major issue is that these models are susceptible to adversarial attacks, where small, carefully crafted perturbations to input data can cause them to fail.<n>We propose a novel adversarial defense method: the Out-of-Distribution Typical Set Sampling Training strategy.
  arXiv  Detail & Related papers  (2024-12-08T13:47:57Z)
• Explanatory Model Monitoring to Understand the Effects of Feature Shifts on Performance [61.06245197347139]
  We propose a novel approach to explain the behavior of a black-box model under feature shifts. We refer to our method that combines concepts from Optimal Transport and Shapley Values as Explanatory Performance Estimation.
  arXiv  Detail & Related papers  (2024-08-24T18:28:19Z)
• What Matters When Repurposing Diffusion Models for General Dense Perception Tasks? [49.84679952948808]
  Recent works show promising results by simply fine-tuning T2I diffusion models for dense perception tasks.<n>We conduct a thorough investigation into critical factors that affect transfer efficiency and performance when using diffusion priors.<n>Our work culminates in the development of GenPercept, an effective deterministic one-step fine-tuning paradigm tailed for dense visual perception tasks.
  arXiv  Detail & Related papers  (2024-03-10T04:23:24Z)
• A data-centric approach to class-specific bias in image data augmentation [0.0]
  Data augmentation (DA) enhances model generalization in computer vision but may introduce biases, impacting class accuracy unevenly. We evaluate DA's class-specific bias across various datasets, including those distinct from ImageNet, through random cropping. This suggests a nuanced approach to model selection, emphasizing bias mitigation.
  arXiv  Detail & Related papers  (2024-03-07T00:32:47Z)
• Improving Adversarial Transferability via Model Alignment [25.43899674478279]
  We introduce a novel model alignment technique aimed at improving a given source model's ability in generating transferable adversarial perturbations. Experiments on the ImageNet dataset, using a variety of model architectures, demonstrate that perturbations generated from aligned source models exhibit significantly higher transferability.
  arXiv  Detail & Related papers  (2023-11-30T12:15:49Z)
• Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey [73.86861112002593]
  We present a comprehensive review of recent diffusion model-based methods on image restoration.<n>We classify and emphasize the innovative designs using diffusion models for both IR and blind/real-world IR.<n>We propose five potential and challenging directions for the future research of diffusion model-based IR.
  arXiv  Detail & Related papers  (2023-08-18T08:40:38Z)
• Fairness Increases Adversarial Vulnerability [50.90773979394264]
  This paper shows the existence of a dichotomy between fairness and robustness, and analyzes when achieving fairness decreases the model robustness to adversarial samples. Experiments on non-linear models and different architectures validate the theoretical findings in multiple vision domains. The paper proposes a simple, yet effective, solution to construct models achieving good tradeoffs between fairness and robustness.
  arXiv  Detail & Related papers  (2022-11-21T19:55:35Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.