Data Attribution for Diffusion Models: Timestep-induced Bias in
Influence Estimation
- URL: http://arxiv.org/abs/2401.09031v2
- Date: Sun, 21 Jan 2024 20:49:31 GMT
- Title: Data Attribution for Diffusion Models: Timestep-induced Bias in
Influence Estimation
- Authors: Tong Xie, Haoyu Li, Andrew Bai, Cho-Jui Hsieh
- Abstract summary: Diffusion models operate over a sequence of timesteps instead of instantaneous input-output relationships in previous contexts.
We present Diffusion-TracIn that incorporates this temporal dynamics and observe that samples' loss gradient norms are highly dependent on timestep.
We introduce Diffusion-ReTrac as a re-normalized adaptation that enables the retrieval of training samples more targeted to the test sample of interest.
- Score: 58.20016784231991
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Data attribution methods trace model behavior back to its training dataset,
offering an effective approach to better understand ''black-box'' neural
networks. While prior research has established quantifiable links between model
output and training data in diverse settings, interpreting diffusion model
outputs in relation to training samples remains underexplored. In particular,
diffusion models operate over a sequence of timesteps instead of instantaneous
input-output relationships in previous contexts, posing a significant challenge
to extend existing frameworks to diffusion models directly. Notably, we present
Diffusion-TracIn that incorporates this temporal dynamics and observe that
samples' loss gradient norms are highly dependent on timestep. This trend leads
to a prominent bias in influence estimation, and is particularly noticeable for
samples trained on large-norm-inducing timesteps, causing them to be generally
influential. To mitigate this effect, we introduce Diffusion-ReTrac as a
re-normalized adaptation that enables the retrieval of training samples more
targeted to the test sample of interest, facilitating a localized measurement
of influence and considerably more intuitive visualization. We demonstrate the
efficacy of our approach through various evaluation metrics and auxiliary
tasks, reducing the amount of generally influential samples to $\frac{1}{3}$ of
its original quantity.
Related papers
- MG-TSD: Multi-Granularity Time Series Diffusion Models with Guided Learning Process [26.661721555671626]
We introduce a novel Multi-Granularity Time Series (MG-TSD) model, which achieves state-of-the-art predictive performance.
Our approach does not rely on additional external data, making it versatile and applicable across various domains.
arXiv Detail & Related papers (2024-03-09T01:15:03Z) - Training Unbiased Diffusion Models From Biased Dataset [18.09610829650175]
This paper proposes time-dependent importance reweighting to mitigate the bias for diffusion models.
We demonstrate that the time-dependent density ratio becomes more precise than previous approaches.
While directly applying it to score-matching is intractable, we discover that using the time-dependent density ratio both for reweighting and score correction can lead to a tractable form of the objective function.
arXiv Detail & Related papers (2024-03-02T12:06:42Z) - Projection Regret: Reducing Background Bias for Novelty Detection via
Diffusion Models [72.07462371883501]
We propose emphProjection Regret (PR), an efficient novelty detection method that mitigates the bias of non-semantic information.
PR computes the perceptual distance between the test image and its diffusion-based projection to detect abnormality.
Extensive experiments demonstrate that PR outperforms the prior art of generative-model-based novelty detection methods by a significant margin.
arXiv Detail & Related papers (2023-12-05T09:44:47Z) - Debias the Training of Diffusion Models [53.49637348771626]
We provide theoretical evidence that the prevailing practice of using a constant loss weight strategy in diffusion models leads to biased estimation during the training phase.
We propose an elegant and effective weighting strategy grounded in the theoretically unbiased principle.
These analyses are expected to advance our understanding and demystify the inner workings of diffusion models.
arXiv Detail & Related papers (2023-10-12T16:04:41Z) - Ensemble Modeling for Multimodal Visual Action Recognition [50.38638300332429]
We propose an ensemble modeling approach for multimodal action recognition.
We independently train individual modality models using a variant of focal loss tailored to handle the long-tailed distribution of the MECCANO [21] dataset.
arXiv Detail & Related papers (2023-08-10T08:43:20Z) - Exploring Continual Learning of Diffusion Models [24.061072903897664]
We evaluate the continual learning (CL) properties of diffusion models.
We provide insights into the dynamics of forgetting, which exhibit diverse behavior across diffusion timesteps.
arXiv Detail & Related papers (2023-03-27T15:52:14Z) - How Much is Enough? A Study on Diffusion Times in Score-based Generative
Models [76.76860707897413]
Current best practice advocates for a large T to ensure that the forward dynamics brings the diffusion sufficiently close to a known and simple noise distribution.
We show how an auxiliary model can be used to bridge the gap between the ideal and the simulated forward dynamics, followed by a standard reverse diffusion process.
arXiv Detail & Related papers (2022-06-10T15:09:46Z) - Efficient Causal Inference from Combined Observational and
Interventional Data through Causal Reductions [68.6505592770171]
Unobserved confounding is one of the main challenges when estimating causal effects.
We propose a novel causal reduction method that replaces an arbitrary number of possibly high-dimensional latent confounders.
We propose a learning algorithm to estimate the parameterized reduced model jointly from observational and interventional data.
arXiv Detail & Related papers (2021-03-08T14:29:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.