A First Step Towards Even More Sparse Encodings of Probability Distributions

• URL: http://arxiv.org/abs/2603.29691v1
• Date: Tue, 31 Mar 2026 12:46:05 GMT
• Title: A First Step Towards Even More Sparse Encodings of Probability Distributions
• Abstract summary: We propose a method for extracting first-order formulas from probability distributions that require significantly less values.<n>This reduction and minimization allows for increasing the sparsity in the encoding while also generalizing a given distribution.<n>Our evaluation shows that sparsity can increase immensely by extracting a small set of short formulas while preserving core information.
• Score: 3.6431130041427964
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real world scenarios can be captured with lifted probability distributions. However, distributions are usually encoded in a table or list, requiring an exponential number of values. Hence, we propose a method for extracting first-order formulas from probability distributions that require significantly less values by reducing the number of values in a distribution and then extracting, for each value, a logical formula to be further minimized. This reduction and minimization allows for increasing the sparsity in the encoding while also generalizing a given distribution. Our evaluation shows that sparsity can increase immensely by extracting a small set of short formulas while preserving core information.

Related papers

• Rethinking Likelihood distributions: Student's t Likelihood Boosts Bayesian Neural Network Performance [0.0]
  We investigate whether alternative assumptions for the likelihood distribution can outperform the commonly used Gaussian.<n>Student's t yields better predictive performance than a Gaussian likelihood distribution, independent of the data distribution and architecture.<n>In some cases, Student's t can also lead to shorter training times, while still being easy to implement.
  arXiv  Detail & Related papers  (2026-07-28T07:32:02Z)
• Distribution Prompting: Understanding the Expressivity of Language Models Through the Next-Token Distributions They Can Produce [16.149411731945968]
  We show that some distributions are significantly harder to elicit than others.<n>We find that distributions with very low or very high entropy are easier to approximate than those with moderate entropy.
  arXiv  Detail & Related papers  (2025-05-18T05:49:48Z)
• A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraints [87.08677547257733]
  Neuro-symbolic AI bridges the gap between purely symbolic and neural approaches to learning. We show how to maximize the likelihood of a symbolic constraint w.r.t the neural network's output distribution. We also evaluate our approach on Sudoku and shortest-path prediction cast as autoregressive generation.
  arXiv  Detail & Related papers  (2023-12-06T20:58:07Z)
• Domain Generalization by Functional Regression [3.209698860006188]
  We study domain generalization as a problem of functional regression. Our concept leads to a new algorithm for learning a linear operator from marginal distributions of inputs to the corresponding conditional distributions of outputs given inputs.
  arXiv  Detail & Related papers  (2023-02-09T16:07:21Z)
• A Dimensionality Reduction Method for Finding Least Favorable Priors with a Focus on Bregman Divergence [108.28566246421742]
  This paper develops a dimensionality reduction method that allows us to move the optimization to a finite-dimensional setting with an explicit bound on the dimension. In order to make progress on the problem, we restrict ourselves to Bayesian risks induced by a relatively large class of loss functions, namely Bregman divergences.
  arXiv  Detail & Related papers  (2022-02-23T16:22:28Z)
• Evidential Softmax for Sparse Multimodal Distributions in Deep Generative Models [38.26333732364642]
  We present $textitev-softmax$, a sparse normalization function that preserves the multimodality of probability distributions. We evaluate our method on a variety of generative models, including variational autoencoders and auto-regressive architectures.
  arXiv  Detail & Related papers  (2021-10-27T05:32:25Z)
• Robust Learning of Optimal Auctions [84.13356290199603]
  We study the problem of learning revenue-optimal multi-bidder auctions from samples when the samples of bidders' valuations can be adversarially corrupted or drawn from distributions that are adversarially perturbed. We propose new algorithms that can learn a mechanism whose revenue is nearly optimal simultaneously for all true distributions'' that are $alpha$-close to the original distribution in Kolmogorov-Smirnov distance.
  arXiv  Detail & Related papers  (2021-07-13T17:37:21Z)
• Approximating Probability Distributions by ReLU Networks [15.899159309486679]
  How many neurons are needed to approximate a target probability distribution using a neural network with a given input distribution and approximation error? This paper examines this question for the case when the input distribution is uniform, and the target distribution belongs to the class of histogram distributions. We obtain a new upper bound on the number of required neurons, which is strictly better than previously existing upper bounds.
  arXiv  Detail & Related papers  (2021-01-25T09:31:20Z)
• Distributional Reinforcement Learning via Moment Matching [54.16108052278444]
  We formulate a method that learns a finite set of statistics from each return distribution via neural networks. Our method can be interpreted as implicitly matching all orders of moments between a return distribution and its Bellman target. Experiments on the suite of Atari games show that our method outperforms the standard distributional RL baselines.
  arXiv  Detail & Related papers  (2020-07-24T05:18:17Z)
• Generative Semantic Hashing Enhanced via Boltzmann Machines [61.688380278649056]
  Existing generative-hashing methods mostly assume a factorized form for the posterior distribution. We propose to employ the distribution of Boltzmann machine as the retrievalal posterior. We show that by effectively modeling correlations among different bits within a hash code, our model can achieve significant performance gains.
  arXiv  Detail & Related papers  (2020-06-16T01:23:39Z)
• Log-Likelihood Ratio Minimizing Flows: Towards Robust and Quantifiable Neural Distribution Alignment [52.02794488304448]
  We propose a new distribution alignment method based on a log-likelihood ratio statistic and normalizing flows. We experimentally verify that minimizing the resulting objective results in domain alignment that preserves the local structure of input domains.
  arXiv  Detail & Related papers  (2020-03-26T22:10:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.