Nonequilbrium physics of generative diffusion models
- URL: http://arxiv.org/abs/2405.11932v1
- Date: Mon, 20 May 2024 10:16:26 GMT
- Title: Nonequilbrium physics of generative diffusion models
- Authors: Zhendong Yu, Haiping Huang,
- Abstract summary: We provide a transparent physics analysis of the diffusion models, deriving the fluctuation theorem, entropy production, Franz-Parisi potential to understand intrinsic phase transitions.
Our analysis is rooted in non-equlibrium physics and concepts from equilibrium physics, i.e., treating both forward and backward dynamics as a Langevin dynamics.
This unified principle is expected to guide machine learning practitioners to design better algorithms and theoretical physicists to link the machine learning to non-equilibrium thermodynamics.
- Score: 2.5690340428649328
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Generative diffusion models apply the concept of Langevin dynamics in physics to machine leaning, attracting a lot of interest from industrial application, but a complete picture about inherent mechanisms is still lacking. In this paper, we provide a transparent physics analysis of the diffusion models, deriving the fluctuation theorem, entropy production, Franz-Parisi potential to understand the intrinsic phase transitions discovered recently. Our analysis is rooted in non-equlibrium physics and concepts from equilibrium physics, i.e., treating both forward and backward dynamics as a Langevin dynamics, and treating the reverse diffusion generative process as a statistical inference, where the time-dependent state variables serve as quenched disorder studied in spin glass theory. This unified principle is expected to guide machine learning practitioners to design better algorithms and theoretical physicists to link the machine learning to non-equilibrium thermodynamics.
Related papers
- Pi-fusion: Physics-informed diffusion model for learning fluid dynamics [3.5197612808072147]
We propose Pi-fusion, a physics-informed diffusion model for predicting the temporal evolution of velocity and pressure field in fluid dynamics.
We introduce a training strategy based on reciprocal learning to learn the quasiperiodical pattern of fluid motion.
Experimental results show that the proposed approach significantly outperforms existing methods for predicting temporal evolution of velocity and pressure field.
arXiv Detail & Related papers (2024-06-06T03:14:59Z) - The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instability [6.4322891559626125]
We show that generative diffusion models undergo second-order phase transitions corresponding to universality breaking phenomena.
We argue that the critical instability that arises from the phase transitions lies at the heart of their generative capabilities.
arXiv Detail & Related papers (2023-10-26T15:15:01Z) - End-To-End Latent Variational Diffusion Models for Inverse Problems in
High Energy Physics [61.44793171735013]
We introduce a novel unified architecture, termed latent variation models, which combines the latent learning of cutting-edge generative art approaches with an end-to-end variational framework.
Our unified approach achieves a distribution-free distance to the truth of over 20 times less than non-latent state-of-the-art baseline.
arXiv Detail & Related papers (2023-05-17T17:43:10Z) - Entanglement R\'enyi Entropies from Ballistic Fluctuation Theory: the
free fermionic case [0.0]
We investigate entanglement entropy using its connection with the large-deviation theory for thermodynamic and hydrodynamic fluctuations.
We show that both the equilibrium behavior and the dynamics of R'enyi entanglement entropies can be fully derived from the Ballistic Fluctuation Theory.
arXiv Detail & Related papers (2023-01-05T23:09:48Z) - Spreading of a local excitation in a Quantum Hierarchical Model [62.997667081978825]
We study the dynamics of the quantum Dyson hierarchical model in its paramagnetic phase.
An initial state made by a local excitation of the paramagnetic ground state is considered.
A localization mechanism is found and the excitation remains close to its initial position at arbitrary times.
arXiv Detail & Related papers (2022-07-14T10:05:20Z) - NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural
Radiance Fields [65.07940731309856]
Deep learning has shown great potential for modeling the physical dynamics of complex particle systems such as fluids.
In this paper, we consider a partially observable scenario known as fluid dynamics grounding.
We propose a differentiable two-stage network named NeuroFluid.
It is shown to reasonably estimate the underlying physics of fluids with different initial shapes, viscosity, and densities.
arXiv Detail & Related papers (2022-03-03T15:13:29Z) - Machine learning nonequilibrium electron forces for adiabatic spin
dynamics [0.0]
We develop a deep-learning neural network that learns the forces in a driven s-d model computed from the nonequilibrium Green's function method.
We show that the Landau-Lifshitz dynamics simulations with forces predicted from the neural-net model accurately reproduce the voltage-driven domain-wall propagation.
arXiv Detail & Related papers (2021-12-22T18:37:56Z) - The problem of engines in statistical physics [62.997667081978825]
Engines are open systems that can generate work cyclically, at the expense of an external disequilibrium.
Recent advances in the theory of open quantum systems point to a more realistic description of autonomous engines.
We show how the external loading force and the thermal noise may be incorporated into the relevant equations of motion.
arXiv Detail & Related papers (2021-08-17T03:59:09Z) - Anomalous statistics of laser-cooled atoms in dissipative optical
lattices [0.0]
A century ago it was realized that an extension to this type of dynamics can be obtained in the form of anomalous" diffusion.
Laser-cooled atomic ensembles can be used as a test bed for such dynamics.
arXiv Detail & Related papers (2021-07-20T14:19:58Z) - Physics-Integrated Variational Autoencoders for Robust and Interpretable
Generative Modeling [86.9726984929758]
We focus on the integration of incomplete physics models into deep generative models.
We propose a VAE architecture in which a part of the latent space is grounded by physics.
We demonstrate generative performance improvements over a set of synthetic and real-world datasets.
arXiv Detail & Related papers (2021-02-25T20:28:52Z) - Analog cosmological reheating in an ultracold Bose gas [58.720142291102135]
We quantum-simulate the reheating-like dynamics of a generic cosmological single-field model in an ultracold Bose gas.
Expanding spacetime as well as the background oscillating inflaton field are mimicked in the non-relativistic limit.
The proposed experiment has the potential of exploring the evolution up to late times even beyond the weak coupling regime.
arXiv Detail & Related papers (2020-08-05T18:00:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.