Expressive Score-Based Priors for Distribution Matching with Geometry-Preserving Regularization

• URL: http://arxiv.org/abs/2506.14607v1
• Date: Tue, 17 Jun 2025 15:08:16 GMT
• Title: Expressive Score-Based Priors for Distribution Matching with Geometry-Preserving Regularization
• Authors: Ziyu Gong, Jim Lim, David I. Inouye,
• Abstract summary: gradient-based DM training only requires the prior's score function -- not its density.<n>This approach eliminates biases from fixed priors, enabling more effective use of geometry-preserving regularization.<n>Our method also demonstrates better stability and computational efficiency compared to other diffusion-based priors.
• Score: 10.432302605566331
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Distribution matching (DM) is a versatile domain-invariant representation learning technique that has been applied to tasks such as fair classification, domain adaptation, and domain translation. Non-parametric DM methods struggle with scalability and adversarial DM approaches suffer from instability and mode collapse. While likelihood-based methods are a promising alternative, they often impose unnecessary biases through fixed priors or require explicit density models (e.g., flows) that can be challenging to train. We address this limitation by introducing a novel approach to training likelihood-based DM using expressive score-based prior distributions. Our key insight is that gradient-based DM training only requires the prior's score function -- not its density -- allowing us to train the prior via denoising score matching. This approach eliminates biases from fixed priors (e.g., in VAEs), enabling more effective use of geometry-preserving regularization, while avoiding the challenge of learning an explicit prior density model (e.g., a flow-based prior). Our method also demonstrates better stability and computational efficiency compared to other diffusion-based priors (e.g., LSGM). Furthermore, experiments demonstrate superior performance across multiple tasks, establishing our score-based method as a stable and effective approach to distribution matching. Source code available at https://github.com/inouye-lab/SAUB.

Related papers

• Enhanced OoD Detection through Cross-Modal Alignment of Multi-Modal Representations [2.992602379681373]
  We show that multi-modal fine-tuning can achieve notable OoDD performance.<n>We propose a training objective that enhances cross-modal alignment by regularizing the distances between image and text embeddings of ID data.
  arXiv  Detail & Related papers  (2025-03-24T16:00:21Z)
• Asymmetric Co-Training for Source-Free Few-Shot Domain Adaptation [5.611768906855499]
  We propose an asymmetric co-training (ACT) method specifically designed for the SFFSDA scenario.<n>We use a two-step optimization process to train the target model.<n>Our findings suggest that adapting a source pre-trained model using only a small amount of labeled target data offers a practical and dependable solution.
  arXiv  Detail & Related papers  (2025-02-20T02:58:45Z)
• What Has Been Overlooked in Contrastive Source-Free Domain Adaptation: Leveraging Source-Informed Latent Augmentation within Neighborhood Context [28.634315143647385]
  Source-free domain adaptation (SFDA) involves adapting a model originally trained using a labeled dataset to perform effectively on an unlabeled dataset.<n>This adaptation is especially crucial when significant disparities in data distributions exist between the two domains.<n>We introduce a straightforward yet highly effective latent augmentation method tailored for contrastive SFDA.
  arXiv  Detail & Related papers  (2024-12-18T20:09:46Z)
• Amortizing intractable inference in diffusion models for vision, language, and control [89.65631572949702]
  This paper studies amortized sampling of the posterior over data, $mathbfxsim prm post(mathbfx)propto p(mathbfx)r(mathbfx)$, in a model that consists of a diffusion generative model prior $p(mathbfx)$ and a black-box constraint or function $r(mathbfx)$.<n>We prove the correctness of a data-free learning objective, relative trajectory balance, for training a diffusion model that samples from
  arXiv  Detail & Related papers  (2024-05-31T16:18:46Z)
• Training Data Attribution via Approximate Unrolled Differentiation [8.87519936904341]
  Methods based on implicit differentiation, such as influence functions, can be made computationally efficient, but fail to account for underspecification. We introduce Source, an approximate unrolling-based TDA method that is computed using an influence-function-like formula.
  arXiv  Detail & Related papers  (2024-05-20T17:17:44Z)
• Diffusion Model for Dense Matching [34.13580888014]
  The objective for establishing dense correspondence between paired images consists of two terms: a data term and a prior term. We propose DiffMatch, a novel conditional diffusion-based framework designed to explicitly model both the data and prior terms. Our experimental results demonstrate significant performance improvements of our method over existing approaches.
  arXiv  Detail & Related papers  (2023-05-30T14:58:24Z)
• Source-Free Domain Adaptation via Distribution Estimation [106.48277721860036]
  Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different. Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data. In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation.
  arXiv  Detail & Related papers  (2022-04-24T12:22:19Z)
• Continual Test-Time Domain Adaptation [94.51284735268597]
  Test-time domain adaptation aims to adapt a source pre-trained model to a target domain without using any source data. CoTTA is easy to implement and can be readily incorporated in off-the-shelf pre-trained models.
  arXiv  Detail & Related papers  (2022-03-25T11:42:02Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• Tune it the Right Way: Unsupervised Validation of Domain Adaptation via Soft Neighborhood Density [125.64297244986552]
  We propose an unsupervised validation criterion that measures the density of soft neighborhoods by computing the entropy of the similarity distribution between points. Our criterion is simpler than competing validation methods, yet more effective.
  arXiv  Detail & Related papers  (2021-08-24T17:41:45Z)
• KL Guided Domain Adaptation [88.19298405363452]
  Domain adaptation is an important problem and often needed for real-world applications. A common approach in the domain adaptation literature is to learn a representation of the input that has the same distributions over the source and the target domain. We show that with a probabilistic representation network, the KL term can be estimated efficiently via minibatch samples.
  arXiv  Detail & Related papers  (2021-06-14T22:24:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.