The Gaussian-Multinoulli Restricted Boltzmann Machine: A Potts Model Extension of the GRBM

• URL: http://arxiv.org/abs/2505.11635v1
• Date: Fri, 16 May 2025 18:59:59 GMT
• Title: The Gaussian-Multinoulli Restricted Boltzmann Machine: A Potts Model Extension of the GRBM
• Authors: Nikhil Kapasi, William Whitehead, Luke Theogarajan,
• Abstract summary: We introduce a generative energy-based model that extends the Gaussian-Bernoulli Restricted Boltzmann Machine (GB-RBM)<n>This modification enables a scalablely richer latent space and supports learning over multivalued, interpretable latent concepts.<n>We demonstrate that GM-RBMs model complex multimodal distributions more effectively than binary RBMs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many real-world tasks, from associative memory to symbolic reasoning, demand discrete, structured representations that standard continuous latent models struggle to express naturally. We introduce the Gaussian-Multinoulli Restricted Boltzmann Machine (GM-RBM), a generative energy-based model that extends the Gaussian-Bernoulli RBM (GB-RBM) by replacing binary hidden units with $q$-state Potts variables. This modification enables a combinatorially richer latent space and supports learning over multivalued, interpretable latent concepts. We formally derive GM-RBM's energy function, learning dynamics, and conditional distributions, showing that it preserves tractable inference and training through contrastive divergence. Empirically, we demonstrate that GM-RBMs model complex multimodal distributions more effectively than binary RBMs, outperforming them on tasks involving analogical recall and structured memory. Our results highlight GM-RBMs as a scalable framework for discrete latent inference with enhanced expressiveness and interoperability.

Related papers

• Exploring the Energy Landscape of RBMs: Reciprocal Space Insights into Bosons, Hierarchical Learning and Symmetry Breaking [0.07499722271664146]
  We focus on Restricted Boltzmann Machines (RBMs), known for their universal approximation capabilities for discrete distributions.<n>By introducing a reciprocal space formulation, we reveal a connection between RBMs, diffusion processes, and coupled Bosons.<n>Our findings bridge the gap between disparate generative frameworks and also shed light on the processes underpinning learning in generative models.
  arXiv  Detail & Related papers  (2025-03-27T14:28:37Z)
• Expressive equivalence of classical and quantum restricted Boltzmann machines [1.1639171061272031]
  We propose a semi-quantum restricted Boltzmann machine (sqRBM) for classical data.<n> sqRBM is commuting in the visible subspace while remaining non-commuting in the hidden subspace.<n>Our theoretical analysis predicts that, to learn a given probability distribution, an RBM requires three times as many hidden units as an sqRBM.
  arXiv  Detail & Related papers  (2025-02-24T19:00:02Z)
• Bellman Diffusion: Generative Modeling as Learning a Linear Operator in the Distribution Space [72.52365911990935]
  We introduce Bellman Diffusion, a novel DGM framework that maintains linearity in MDPs through gradient and scalar field modeling. Our results show that Bellman Diffusion achieves accurate field estimations and is a capable image generator, converging 1.5x faster than the traditional histogram-based baseline in distributional RL tasks.
  arXiv  Detail & Related papers  (2024-10-02T17:53:23Z)
• A Non-negative VAE:the Generalized Gamma Belief Network [49.970917207211556]
  The gamma belief network (GBN) has demonstrated its potential for uncovering multi-layer interpretable latent representations in text data. We introduce the generalized gamma belief network (Generalized GBN) in this paper, which extends the original linear generative model to a more expressive non-linear generative model. We also propose an upward-downward Weibull inference network to approximate the posterior distribution of the latent variables.
  arXiv  Detail & Related papers  (2024-08-06T18:18:37Z)
• Monotone deep Boltzmann machines [86.50247625239406]
  Deep Boltzmann machines (DBMs) are multi-layered probabilistic models governed by a pairwise energy function. We develop a new class of restricted model, the monotone DBM, which allows for arbitrary self-connection in each layer. We show that a particular choice of activation results in a fixed-point iteration that gives a variational mean-field solution.
  arXiv  Detail & Related papers  (2023-07-11T03:02:44Z)
• Neural Boltzmann Machines [2.179313476241343]
  Conditional generative models are capable of using contextual information as input to create new imaginative outputs. Conditional Restricted Boltzmann Machines (CRBMs) are one class of conditional generative models that have proven to be especially adept at modeling noisy discrete or continuous data. We generalize CRBMs by converting each of the CRBM parameters to their own neural networks that are allowed to be functions of the conditional inputs.
  arXiv  Detail & Related papers  (2023-05-15T04:03:51Z)
• Guiding Energy-based Models via Contrastive Latent Variables [81.68492940158436]
  An energy-based model (EBM) is a popular generative framework that offers both explicit density and architectural flexibility. There often exists a large gap between EBMs and other generative frameworks like GANs in terms of generation quality. We propose a novel and effective framework for improving EBMs via contrastive representation learning.
  arXiv  Detail & Related papers  (2023-03-06T10:50:25Z)
• Exact representations of many body interactions with RBM neural networks [77.34726150561087]
  We exploit the representation power of RBMs to provide an exact decomposition of many-body contact interactions into one-body operators. This construction generalizes the well known Hirsch's transform used for the Hubbard model to more complicated theories such as Pionless EFT in nuclear physics.
  arXiv  Detail & Related papers  (2020-05-07T15:59:29Z)
• Training Deep Energy-Based Models with f-Divergence Minimization [113.97274898282343]
  Deep energy-based models (EBMs) are very flexible in distribution parametrization but computationally challenging. We propose a general variational framework termed f-EBM to train EBMs using any desired f-divergence. Experimental results demonstrate the superiority of f-EBM over contrastive divergence, as well as the benefits of training EBMs using f-divergences other than KL.
  arXiv  Detail & Related papers  (2020-03-06T23:11:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.