Seeing the Unseen: How EMoE Unveils Bias in Text-to-Image Diffusion Models

• URL: http://arxiv.org/abs/2505.13273v1
• Date: Mon, 19 May 2025 15:53:32 GMT
• Title: Seeing the Unseen: How EMoE Unveils Bias in Text-to-Image Diffusion Models
• Authors: Lucas Berry, Axel Brando, Wei-Di Chang, Juan Camilo Gamboa Higuera, David Meger,
• Abstract summary: Estimating uncertainty in text-to-image diffusion models is challenging because of their large parameter counts and operation in complex, high-dimensional spaces.<n>We propose Epistemic Mixture of Experts (EMoE), a novel framework for efficiently estimating epistemic uncertainty in diffusion models.
• Score: 13.841466720774838
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Estimating uncertainty in text-to-image diffusion models is challenging because of their large parameter counts (often exceeding 100 million) and operation in complex, high-dimensional spaces with virtually infinite input possibilities. In this paper, we propose Epistemic Mixture of Experts (EMoE), a novel framework for efficiently estimating epistemic uncertainty in diffusion models. EMoE leverages pre-trained networks without requiring additional training, enabling direct uncertainty estimation from a prompt. We leverage a latent space within the diffusion process that captures epistemic uncertainty better than existing methods. Experimental results on the COCO dataset demonstrate EMoE's effectiveness, showing a strong correlation between uncertainty and image quality. Additionally, EMoE identifies under-sampled languages and regions with higher uncertainty, revealing hidden biases in the training set. This capability demonstrates the relevance of EMoE as a tool for addressing fairness and accountability in AI-generated content.

Related papers

• Provable Maximum Entropy Manifold Exploration via Diffusion Models [58.89696361871563]
  Exploration is critical for solving real-world decision-making problems such as scientific discovery.<n>We introduce a novel framework that casts exploration as entropy over approximate data manifold implicitly defined by a pre-trained diffusion model.<n>We develop an algorithm based on mirror descent that solves the exploration problem as sequential fine-tuning of a pre-trained diffusion model.
  arXiv  Detail & Related papers  (2025-06-18T11:59:15Z)
• Improved Immiscible Diffusion: Accelerate Diffusion Training by Reducing Its Miscibility [62.272571285823595]
  We show how immiscibility eases denoising and improves efficiency.<n>We propose a family of implementations including K-nearest neighbor (KNN) noise selection and image scaling to reduce miscibility.<n>This work establishes a potentially new direction for future research into high-efficiency diffusion training.
  arXiv  Detail & Related papers  (2025-05-24T05:38:35Z)
• EvidenceMoE: A Physics-Guided Mixture-of-Experts with Evidential Critics for Advancing Fluorescence Light Detection and Ranging in Scattering Media [3.9833452895674655]
  Fluorescence LiDAR (FLiDAR) is a technology employed for distance and depth estimation across medical, automotive, and other fields.<n>The complex nature of the acquired FLiDAR signal makes isolating photon time-of-flight and intrinsic fluorescence lifetime exceptionally difficult.<n>We present a Physics-Guided Mixture-of-Experts framework tailored for specialized modeling of diverse temporal components.
  arXiv  Detail & Related papers  (2025-05-23T16:38:13Z)
• Generalized Interpolating Discrete Diffusion [65.74168524007484]
  Masked diffusion is a popular choice due to its simplicity and effectiveness.<n>We derive the theoretical backbone of a family of general interpolating discrete diffusion processes.<n>Exploiting GIDD's flexibility, we explore a hybrid approach combining masking and uniform noise.
  arXiv  Detail & Related papers  (2025-03-06T14:30:55Z)
• Latent Distribution Decoupling: A Probabilistic Framework for Uncertainty-Aware Multimodal Emotion Recognition [7.25361375272096]
  Multimodal multi-label emotion recognition aims to identify the concurrent presence of multiple emotions in multimodal data.<n>Existing studies overlook the impact of textbfaleatoric uncertainty, which is the inherent noise in the multimodal data.<n>This paper proposes Latent emotional Distribution Decomposition with Uncertainty perception framework.
  arXiv  Detail & Related papers  (2025-02-19T18:53:23Z)
• Detecting Discrepancies Between AI-Generated and Natural Images Using Uncertainty [91.64626435585643]
  We propose a novel approach for detecting AI-generated images by leveraging predictive uncertainty to mitigate misuse and associated risks.<n>The motivation arises from the fundamental assumption regarding the distributional discrepancy between natural and AI-generated images.<n>We propose to leverage large-scale pre-trained models to calculate the uncertainty as the score for detecting AI-generated images.
  arXiv  Detail & Related papers  (2024-12-08T11:32:25Z)
• Shedding Light on Large Generative Networks: Estimating Epistemic Uncertainty in Diffusion Models [15.352556466952477]
  Generative diffusion models are notable for their large parameter count (exceeding 100 million) and operation within high-dimensional image spaces. We introduce an innovative framework, Diffusion Ensembles for Capturing Uncertainty (DECU), designed for estimating epistemic uncertainty for diffusion models.
  arXiv  Detail & Related papers  (2024-06-05T14:03:21Z)
• Estimating Epistemic and Aleatoric Uncertainty with a Single Model [5.871583927216653]
  We introduce a new approach to ensembling, hyper-diffusion models (HyperDM) HyperDM offers prediction accuracy on par with, and in some cases superior to, multi-model ensembles. We validate our method on two distinct real-world tasks: x-ray computed tomography reconstruction and weather temperature forecasting.
  arXiv  Detail & Related papers  (2024-02-05T19:39:52Z)
• Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model [61.53213964333474]
  We propose a unified framework Adv-Diffusion that can generate imperceptible adversarial identity perturbations in the latent space but not the raw pixel space. Specifically, we propose the identity-sensitive conditioned diffusion generative model to generate semantic perturbations in the surroundings. The designed adaptive strength-based adversarial perturbation algorithm can ensure both attack transferability and stealthiness.
  arXiv  Detail & Related papers  (2023-12-18T15:25:23Z)
• Measuring and Modeling Uncertainty Degree for Monocular Depth Estimation [50.920911532133154]
  The intrinsic ill-posedness and ordinal-sensitive nature of monocular depth estimation (MDE) models pose major challenges to the estimation of uncertainty degree. We propose to model the uncertainty of MDE models from the perspective of the inherent probability distributions. By simply introducing additional training regularization terms, our model, with surprisingly simple formations and without requiring extra modules or multiple inferences, can provide uncertainty estimations with state-of-the-art reliability.
  arXiv  Detail & Related papers  (2023-07-19T12:11:15Z)
• CamoDiffusion: Camouflaged Object Detection via Conditional Diffusion Models [72.93652777646233]
  Camouflaged Object Detection (COD) is a challenging task in computer vision due to the high similarity between camouflaged objects and their surroundings. We propose a new paradigm that treats COD as a conditional mask-generation task leveraging diffusion models. Our method, dubbed CamoDiffusion, employs the denoising process of diffusion models to iteratively reduce the noise of the mask.
  arXiv  Detail & Related papers  (2023-05-29T07:49:44Z)
• The role of MRI physics in brain segmentation CNNs: achieving acquisition invariance and instructive uncertainties [3.46329153611365]
  In this paper we demonstrate the efficacy of a physics-informed, uncertainty-aware, segmentation network. We show that the proposed approach also accurately extrapolates to out-of-distribution sequence samples. We demonstrate a significant improvement in terms of coefficients of variation, backed by uncertainty based volumetric validation.
  arXiv  Detail & Related papers  (2021-11-04T11:52:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.