DO-EM: Density Operator Expectation Maximization

• URL: http://arxiv.org/abs/2507.22786v1
• Date: Wed, 30 Jul 2025 15:51:20 GMT
• Title: DO-EM: Density Operator Expectation Maximization
• Authors: Adit Vishnu, Abhay Shastry, Dhruva Kashyap, Chiranjib Bhattacharyya,
• Abstract summary: We develop an Expectation-Maximization framework to learn latent variable models defined through textbfDOMs on classical hardware.<n>We show that the textbfDO-EM algorithm ensures non-decreasing log-likelihood across iterations for a broad class of models.<n>We present Quantum Interleaved Deep Boltzmann Machines (textbfQiDBMs), a textbfDOM that can be trained with the same resources as a DBM.
• Score: 10.697014584408963
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Density operators, quantum generalizations of probability distributions, are gaining prominence in machine learning due to their foundational role in quantum computing. Generative modeling based on density operator models (\textbf{DOMs}) is an emerging field, but existing training algorithms -- such as those for the Quantum Boltzmann Machine -- do not scale to real-world data, such as the MNIST dataset. The Expectation-Maximization algorithm has played a fundamental role in enabling scalable training of probabilistic latent variable models on real-world datasets. \textit{In this paper, we develop an Expectation-Maximization framework to learn latent variable models defined through \textbf{DOMs} on classical hardware, with resources comparable to those used for probabilistic models, while scaling to real-world data.} However, designing such an algorithm is nontrivial due to the absence of a well-defined quantum analogue to conditional probability, which complicates the Expectation step. To overcome this, we reformulate the Expectation step as a quantum information projection (QIP) problem and show that the Petz Recovery Map provides a solution under sufficient conditions. Using this formulation, we introduce the Density Operator Expectation Maximization (DO-EM) algorithm -- an iterative Minorant-Maximization procedure that optimizes a quantum evidence lower bound. We show that the \textbf{DO-EM} algorithm ensures non-decreasing log-likelihood across iterations for a broad class of models. Finally, we present Quantum Interleaved Deep Boltzmann Machines (\textbf{QiDBMs}), a \textbf{DOM} that can be trained with the same resources as a DBM. When trained with \textbf{DO-EM} under Contrastive Divergence, a \textbf{QiDBM} outperforms larger classical DBMs in image generation on the MNIST dataset, achieving a 40--60\% reduction in the Fr\'echet Inception Distance.

Related papers

• MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillation [74.34220141721231]
  We present MPQ-DMv2, an improved textbfMixed textbfPrecision textbfQuantization framework for extremely low-bit textbfDiffusion textbfModels.
  arXiv  Detail & Related papers  (2025-07-06T08:16:50Z)
• Maximal Entropy Formalism and the Restricted Boltzmann Machine [1.7563879056963012]
  We introduce Restricted Boltzmann Machines (RBMs) as probabilistic models to approximate quantum states.<n>We employ quantum sampling techniques to enhance RBM training, enabling scalable and high-fidelity quantum state reconstruction.<n>Our method applies to the general and previously unaddressed case of reconstructing arbitrary mixed quantum states from incomplete and potentially non-commuting sets of expectations.
  arXiv  Detail & Related papers  (2025-04-04T17:57:20Z)
• Expressive equivalence of classical and quantum restricted Boltzmann machines [1.1639171061272031]
  We propose a semi-quantum restricted Boltzmann machine (sqRBM) for classical data.<n> sqRBM is commuting in the visible subspace while remaining non-commuting in the hidden subspace.<n>Our theoretical analysis predicts that, to learn a given probability distribution, an RBM requires three times as many hidden units as an sqRBM.
  arXiv  Detail & Related papers  (2025-02-24T19:00:02Z)
• Probabilistic neural operators for functional uncertainty quantification [14.08907045605149]
  We introduce the probabilistic neural operator (PNO), a framework for learning probability distributions over the output function space of neural operators.<n>PNO extends neural operators with generative modeling based on strictly proper scoring rules, integrating uncertainty information directly into the training process.
  arXiv  Detail & Related papers  (2025-02-18T14:42:11Z)
• Preconditioned Inexact Stochastic ADMM for Deep Model [35.37705488695026]
  This paper develops an algorithm, PISA, which enables scalable parallel computing and supports various preconditions.<n>It converges under the sole assumption of Lipschitz continuity of the gradient on a bounded region, removing the need for other conditions commonly imposed by methods.<n>It demonstrates its superior numerical performance compared to various state-of-the-art iterations.
  arXiv  Detail & Related papers  (2025-02-15T12:28:51Z)
• Conditional Distribution Quantization in Machine Learning [83.54039134248231]
  Conditional expectation mathbbE(Y mid X) often fails to capture the complexity of multimodal conditional distributions mathcalL(Y mid X)<n>We propose using n-point conditional quantizations--functional mappings of X that are learnable via gradient descent--to approximate mathcalL(Y mid X)
  arXiv  Detail & Related papers  (2025-02-11T00:28:24Z)
• Accelerated zero-order SGD under high-order smoothness and overparameterized regime [79.85163929026146]
  We present a novel gradient-free algorithm to solve convex optimization problems. Such problems are encountered in medicine, physics, and machine learning. We provide convergence guarantees for the proposed algorithm under both types of noise.
  arXiv  Detail & Related papers  (2024-11-21T10:26:17Z)
• $ζ$-QVAE: A Quantum Variational Autoencoder utilizing Regularized Mixed-state Latent Representations [1.0687104237121408]
  A major challenge in quantum computing is its application to large real-world datasets due to scarce quantum hardware resources.<n>We present a fully quantum framework, $zeta$-QVAE, which encompasses all the capabilities of classical VAEs.<n>Our results indicate that $zeta$-QVAE learns representations that better utilize the capacity of the latent space.
  arXiv  Detail & Related papers  (2024-02-27T18:37:01Z)
• Monotone deep Boltzmann machines [86.50247625239406]
  Deep Boltzmann machines (DBMs) are multi-layered probabilistic models governed by a pairwise energy function. We develop a new class of restricted model, the monotone DBM, which allows for arbitrary self-connection in each layer. We show that a particular choice of activation results in a fixed-point iteration that gives a variational mean-field solution.
  arXiv  Detail & Related papers  (2023-07-11T03:02:44Z)
• Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models [69.22568644711113]
  We introduce probabilistic unrolling, a method that combines Monte Carlo sampling with iterative linear solvers to circumvent matrix inversions. Our theoretical analyses reveal that unrolling and backpropagation through the iterations of the solver can accelerate gradient estimation for maximum likelihood estimation. In experiments on simulated and real data, we demonstrate that probabilistic unrolling learns latent Gaussian models up to an order of magnitude faster than gradient EM, with minimal losses in model performance.
  arXiv  Detail & Related papers  (2023-06-05T21:08:34Z)
• Ensemble Multi-Quantiles: Adaptively Flexible Distribution Prediction for Uncertainty Quantification [4.728311759896569]
  We propose a novel, succinct, and effective approach for distribution prediction to quantify uncertainty in machine learning. It incorporates adaptively flexible distribution prediction of $mathbbP(mathbfy|mathbfX=x)$ in regression tasks. On extensive regression tasks from UCI datasets, we show that EMQ achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-11-26T11:45:32Z)
• Do Quantum Circuit Born Machines Generalize? [58.720142291102135]
  We present the first work in the literature that presents the QCBM's generalization performance as an integral evaluation metric for quantum generative models. We show that the QCBM is able to effectively learn the reweighted dataset and generate unseen samples with higher quality than those in the training set.
  arXiv  Detail & Related papers  (2022-07-27T17:06:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.