Thermodynamic Performance Limits for Score-Based Diffusion Models

• URL: http://arxiv.org/abs/2510.06174v1
• Date: Tue, 07 Oct 2025 17:35:18 GMT
• Title: Thermodynamic Performance Limits for Score-Based Diffusion Models
• Authors: Nathan X. Kodama, Michael Hinczewski,
• Abstract summary: We establish a connection between score-based diffusion models and non-equilibrium thermodynamics by deriving performance limits based on entropy rates.<n>Our framework connects generative modeling performance to fundamental physical principles through thermodynamics.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We establish a fundamental connection between score-based diffusion models and non-equilibrium thermodynamics by deriving performance limits based on entropy rates. Our main theoretical contribution is a lower bound on the negative log-likelihood of the data that relates model performance to entropy rates of diffusion processes. We numerically validate this bound on a synthetic dataset and investigate its tightness. By building a bridge to entropy rates - system, intrinsic, and exchange entropy - we provide new insights into the thermodynamic operation of these models, drawing parallels to Maxwell's demon and implications for thermodynamic computing hardware. Our framework connects generative modeling performance to fundamental physical principles through stochastic thermodynamics.

Related papers

• Entropy-Based Block Pruning for Efficient Large Language Models [81.18339597023187]
  We propose an entropy-based pruning strategy to enhance efficiency while maintaining performance.<n> Empirical analysis reveals that the entropy of hidden representations decreases in the early blocks but progressively increases across most subsequent blocks.
  arXiv  Detail & Related papers  (2025-04-04T03:42:34Z)
• Thermodynamics-Consistent Graph Neural Networks [50.0791489606211]
  We propose excess Gibbs free energy graph neural networks (GE-GNNs) for predicting composition-dependent activity coefficients of binary mixtures. The GE-GNN architecture ensures thermodynamic consistency by predicting the molar excess Gibbs free energy. We demonstrate high accuracy and thermodynamic consistency of the activity coefficient predictions.
  arXiv  Detail & Related papers  (2024-07-08T06:58:56Z)
• Finite-temperature properties of string-net models [0.0]
  We compute the partition function of the string-net model and investigate several thermodynamical quantities. In the thermodynamic limit, we show that the partition function is dominated by the contribution of special particles, dubbed pure fluxons. We also analyze the behavior of Wegner-Wilson loops associated to excitations and show that they obey an area law.
  arXiv  Detail & Related papers  (2024-06-28T07:51:58Z)
• Deep generative modelling of canonical ensemble with differentiable thermal properties [0.9421843976231371]
  We propose a variational modelling method with differentiable temperature for canonical ensembles. Using a deep generative model, the free energy is estimated and minimized simultaneously in a continuous temperature range. The training process requires no dataset, and works with arbitrary explicit density generative models.
  arXiv  Detail & Related papers  (2024-04-29T03:41:49Z)
• Quantum thermodynamics of the spin-boson model using the principle of minimal dissipation [41.94295877935867]
  We investigate the influence of the environment on quantities such as work, heat and entropy production. The results reveal significant differences to the weak-coupling forms of work, heat and entropy production.
  arXiv  Detail & Related papers  (2024-04-18T12:11:18Z)
• A thermodynamically consistent physics-informed deep learning material model for short fiber/polymer nanocomposites [0.0]
  This work proposes a physics-informed deep learning (PIDL)-based model for investigating the viscoelastic-viscoplastic behavior of short fiber-reinforced nanoparticles-filled epoxies under various ambient conditions. The PIDL model can accurately predict the mechanical behavior of epoxy-based nanocomposites for different volume fractions of fibers and nanoparticles under various hygrothermal conditions.
  arXiv  Detail & Related papers  (2024-03-27T07:22:32Z)
• Heat and Work in Quantum Thermodynamics: a Cybernetic Approach [0.0]
  We present a new proposal for distinguishing heat from work based on a control-theoretic observability decomposition. We derive a Hermitian operator representing instantaneous dissipation of observable energy, and suggest a generalization of the von-Neumann entropy.
  arXiv  Detail & Related papers  (2024-03-04T13:26:48Z)
• Gibbs-Duhem-Informed Neural Networks for Binary Activity Coefficient Prediction [45.84205238554709]
  We propose Gibbs-Duhem-informed neural networks for the prediction of binary activity coefficients at varying compositions. We include the Gibbs-Duhem equation explicitly in the loss function for training neural networks.
  arXiv  Detail & Related papers  (2023-05-31T07:36:45Z)
• Fast Thermalization from the Eigenstate Thermalization Hypothesis [69.68937033275746]
  Eigenstate Thermalization Hypothesis (ETH) has played a major role in understanding thermodynamic phenomena in closed quantum systems. This paper establishes a rigorous link between ETH and fast thermalization to the global Gibbs state. Our results explain finite-time thermalization in chaotic open quantum systems.
  arXiv  Detail & Related papers  (2021-12-14T18:48:31Z)
• Open-system approach to nonequilibrium quantum thermodynamics at arbitrary coupling [77.34726150561087]
  We develop a general theory describing the thermodynamical behavior of open quantum systems coupled to thermal baths. Our approach is based on the exact time-local quantum master equation for the reduced open system states.
  arXiv  Detail & Related papers  (2021-09-24T11:19:22Z)
• Uhlmann Fidelity and Fidelity Susceptibility for Integrable Spin Chains at Finite Temperature: Exact Results [68.8204255655161]
  We show that the proper inclusion of the odd parity subspace leads to the enhancement of maximal fidelity susceptibility in the intermediate range of temperatures. The correct low-temperature behavior is captured by an approximation involving the two lowest many-body energy eigenstates.
  arXiv  Detail & Related papers  (2021-05-11T14:08:02Z)
• Qubit thermodynamics far from equilibrium: two perspectives about the nature of heat and work in the quantum regime [68.8204255655161]
  We develop an alternative theoretical framework for the thermodynamic analysis of two-level systems. We observe the appearance of a new term of work, which represents the energy cost of rotating the Bloch vector in presence of the external field that defines the local Hamiltonian. In order to illustrate our findings we study, from both perspectives, matter-radiation interaction processes for two different systems.
  arXiv  Detail & Related papers  (2021-03-16T09:31:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.