Generalized Score Matching for General Domains

• URL: http://arxiv.org/abs/2009.11428v1
• Date: Thu, 24 Sep 2020 00:53:04 GMT
• Title: Generalized Score Matching for General Domains
• Authors: Shiqing Yu, Mathias Drton, Ali Shojaie
• Abstract summary: Estimation of density functions supported on general domains arises when the data is naturally restricted to a proper subset of the real space. We offer a natural generalization of score matching that accommodates densities supported on a very general class of domains.
• Score: 6.982738885923204
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Estimation of density functions supported on general domains arises when the data is naturally restricted to a proper subset of the real space. This problem is complicated by typically intractable normalizing constants. Score matching provides a powerful tool for estimating densities with such intractable normalizing constants, but as originally proposed is limited to densities on $\mathbb{R}^m$ and $\mathbb{R}_+^m$. In this paper, we offer a natural generalization of score matching that accommodates densities supported on a very general class of domains. We apply the framework to truncated graphical and pairwise interaction models, and provide theoretical guarantees for the resulting estimators. We also generalize a recently proposed method from bounded to unbounded domains, and empirically demonstrate the advantages of our method.

Related papers

• Violation of steering inequality for generalized equiangular measurements [0.6906005491572401]
  We study bipartite quantum steering using a general class of measurement operators known as the generalized equiangular measurement (GEAM) Our approach allows for the construction of steering inequalities that are applicable not only to informationally complete measurements but also to incomplete or lossy scenarios.
  arXiv  Detail & Related papers  (2025-04-11T12:54:00Z)
• An Information-Theoretic Framework for Out-of-Distribution Generalization with Applications to Stochastic Gradient Langevin Dynamics [22.64636825553921]
  We study the Out-of-Distribution generalization in machine learning and propose a general framework that establishes information-theoretic generalization bounds. Our framework interpolates freely between Integral Probability Metric and $f$-divergence, which naturally recovers some known results. By focusing on $f$-divergence and combining it with the Mutual Information Conditional (CMI) methods, we derive a family of CMI-based generalization bounds.
  arXiv  Detail & Related papers  (2024-03-29T00:29:57Z)
• Universal coding, intrinsic volumes, and metric complexity [3.4265828682659705]
  In this assignment, the goal is to predict, or equivalently, a sequence of real-valued observations almost as well as the best distribution with mean-valued subsets of $mathbbRn. As part of our analysis, we also characterize the minimax redundancy for a general constraint set. We contrast our findings with classical results in information theory.
  arXiv  Detail & Related papers  (2023-03-13T16:54:04Z)
• Cross-Domain Ensemble Distillation for Domain Generalization [17.575016642108253]
  We propose a simple yet effective method for domain generalization, named cross-domain ensemble distillation (XDED) Our method generates an ensemble of the output logits from training data with the same label but from different domains and then penalizes each output for the mismatch with the ensemble. We show that models learned by our method are robust against adversarial attacks and image corruptions.
  arXiv  Detail & Related papers  (2022-11-25T12:32:36Z)
• Instance-Dependent Generalization Bounds via Optimal Transport [51.71650746285469]
  Existing generalization bounds fail to explain crucial factors that drive the generalization of modern neural networks. We derive instance-dependent generalization bounds that depend on the local Lipschitz regularity of the learned prediction function in the data space. We empirically analyze our generalization bounds for neural networks, showing that the bound values are meaningful and capture the effect of popular regularization methods during training.
  arXiv  Detail & Related papers  (2022-11-02T16:39:42Z)
• On the Importance of Gradient Norm in PAC-Bayesian Bounds [92.82627080794491]
  We propose a new generalization bound that exploits the contractivity of the log-Sobolev inequalities. We empirically analyze the effect of this new loss-gradient norm term on different neural architectures.
  arXiv  Detail & Related papers  (2022-10-12T12:49:20Z)
• Constrained Maximum Cross-Domain Likelihood for Domain Generalization [14.91361835243516]
  Domain generalization aims to learn a generalizable model on multiple source domains, which is expected to perform well on unseen test domains. In this paper, we propose a novel domain generalization method, which minimizes the KL-divergence between posterior distributions from different domains. Experiments on four standard benchmark datasets, i.e., Digits-DG, PACS, Office-Home and miniDomainNet, highlight the superior performance of our method.
  arXiv  Detail & Related papers  (2022-10-09T03:41:02Z)
• Compound Domain Generalization via Meta-Knowledge Encoding [55.22920476224671]
  We introduce Style-induced Domain-specific Normalization (SDNorm) to re-normalize the multi-modal underlying distributions. We harness the prototype representations, the centroids of classes, to perform relational modeling in the embedding space. Experiments on four standard Domain Generalization benchmarks reveal that COMEN exceeds the state-of-the-art performance without the need of domain supervision.
  arXiv  Detail & Related papers  (2022-03-24T11:54:59Z)
• Provably Strict Generalisation Benefit for Invariance in Kernel Methods [0.0]
  We build on the function space perspective of Elesedy and Zaidi arXiv:2102.10333 to derive a strictly non-zero generalisation benefit. We find that generalisation is governed by a notion of effective dimension that arises from the interplay between the kernel and the group.
  arXiv  Detail & Related papers  (2021-06-04T08:55:28Z)
• Adaptive Domain-Specific Normalization for Generalizable Person Re-Identification [81.30327016286009]
  We propose a novel adaptive domain-specific normalization approach (AdsNorm) for generalizable person Re-ID. In this work, we propose a novel adaptive domain-specific normalization approach (AdsNorm) for generalizable person Re-ID.
  arXiv  Detail & Related papers  (2021-05-07T02:54:55Z)
• An Online Learning Approach to Interpolation and Extrapolation in Domain Generalization [53.592597682854944]
  We recast generalization over sub-groups as an online game between a player minimizing risk and an adversary presenting new test. We show that ERM is provably minimax-optimal for both tasks.
  arXiv  Detail & Related papers  (2021-02-25T19:06:48Z)
• Failures of model-dependent generalization bounds for least-norm interpolation [39.97534972432276]
  We consider bounds on the generalization performance of the least-norm linear regressor. For a variety of natural joint distributions on training examples, any valid generalization bound must sometimes be very loose.
  arXiv  Detail & Related papers  (2020-10-16T16:30:05Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.