Target Score Matching

• URL: http://arxiv.org/abs/2402.08667v1
• Date: Tue, 13 Feb 2024 18:48:28 GMT
• Title: Target Score Matching
• Authors: Valentin De Bortoli, Michael Hutchinson, Peter Wirnsberger, Arnaud Doucet
• Abstract summary: We show that it is possible to leverage knowledge of the target score. We present a Target Score Identity and corresponding Target Score Matching regression loss.
• Score: 36.80075781966174
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Denoising Score Matching estimates the score of a noised version of a target distribution by minimizing a regression loss and is widely used to train the popular class of Denoising Diffusion Models. A well known limitation of Denoising Score Matching, however, is that it yields poor estimates of the score at low noise levels. This issue is particularly unfavourable for problems in the physical sciences and for Monte Carlo sampling tasks for which the score of the clean original target is known. Intuitively, estimating the score of a slightly noised version of the target should be a simple task in such cases. In this paper, we address this shortcoming and show that it is indeed possible to leverage knowledge of the target score. We present a Target Score Identity and corresponding Target Score Matching regression loss which allows us to obtain score estimates admitting favourable properties at low noise levels.

Related papers

• Language Generation with Strictly Proper Scoring Rules [70.340673452404]
  We propose a strategy for adapting scoring rules to language generation, allowing for language modeling with any non-local scoring rules. We train language generation models using two classic strictly proper scoring rules, the Brier score and the Spherical score, as alternatives to the logarithmic score.
  arXiv  Detail & Related papers  (2024-05-29T09:09:00Z)
• Towards noise contrastive estimation with soft targets for conditional models [0.6138671548064356]
  We propose a loss function that is compatible with probabilistic targets. Our new soft target InfoNCE loss is conceptually simple, efficient to compute, and can be motivated through the framework of noise contrastive estimation. We observe that soft target InfoNCE performs on par with strong soft target cross-entropy baselines and outperforms hard target NLL and InfoNCE losses on popular benchmarks.
  arXiv  Detail & Related papers  (2024-04-22T10:45:59Z)
• Improving Interpretability of Scores in Anomaly Detection Based on Gaussian-Bernoulli Restricted Boltzmann Machine [0.0]
  In GBRBM-based anomaly detection, normal and anomalous data are classified based on a score that is identical to an energy function of the marginal GBRBM. We propose a measure that improves score's interpretability based on its cumulative distribution. We also establish a guideline for setting the threshold using the interpretable measure.
  arXiv  Detail & Related papers  (2024-03-19T12:13:52Z)
• Sample Complexity Bounds for Score-Matching: Causal Discovery and Generative Modeling [82.36856860383291]
  We demonstrate that accurate estimation of the score function is achievable by training a standard deep ReLU neural network. We establish bounds on the error rate of recovering causal relationships using the score-matching-based causal discovery method.
  arXiv  Detail & Related papers  (2023-10-27T13:09:56Z)
• Label Noise: Correcting the Forward-Correction [0.0]
  Training neural network classifiers on datasets with label noise poses a risk of overfitting them to the noisy labels. We propose an approach to tackling overfitting caused by label noise. Motivated by this observation, we propose imposing a lower bound on the training loss to mitigate overfitting.
  arXiv  Detail & Related papers  (2023-07-24T19:41:19Z)
• Denoising Likelihood Score Matching for Conditional Score-based Data Generation [22.751924447125955]
  We propose a novel training objective called Denoising Likelihood Score Matching (DLSM) loss to match the gradients of the true log likelihood density. Our experimental evidence shows that the proposed method outperforms the previous methods noticeably in terms of several key evaluation metrics.
  arXiv  Detail & Related papers  (2022-03-27T04:37:54Z)
• Rethinking Counting and Localization in Crowds:A Purely Point-Based Framework [59.578339075658995]
  We propose a purely point-based framework for joint crowd counting and individual localization. We design an intuitive solution under this framework, which is called Point to Point Network (P2PNet)
  arXiv  Detail & Related papers  (2021-07-27T11:41:50Z)
• Deconfounding Scores: Feature Representations for Causal Effect Estimation with Weak Overlap [140.98628848491146]
  We introduce deconfounding scores, which induce better overlap without biasing the target of estimation. We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data. In particular, we show that this technique could be an attractive alternative to standard regularizations.
  arXiv  Detail & Related papers  (2021-04-12T18:50:11Z)
• Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification [137.9939571408506]
  We estimate and exploit the credibility of the assigned pseudo-label of each sample to alleviate the influence of noisy labels. Our uncertainty-guided optimization brings significant improvement and achieves the state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-16T04:09:04Z)
• Probabilistic Anchor Assignment with IoU Prediction for Object Detection [9.703212439661097]
  In object detection, determining which anchors to assign as positive or negative samples, known as anchor assignment, has been revealed as a core procedure that can significantly affect a model's performance. We propose a novel anchor assignment strategy that adaptively separates anchors into positive and negative samples for a ground truth bounding box according to the model's learning status.
  arXiv  Detail & Related papers  (2020-07-16T04:26:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.