Latent Thermodynamic Flows: Unified Representation Learning and Generative Modeling of Temperature-Dependent Behaviors from Limited Data

• URL: http://arxiv.org/abs/2507.03174v1
• Date: Thu, 03 Jul 2025 21:02:36 GMT
• Title: Latent Thermodynamic Flows: Unified Representation Learning and Generative Modeling of Temperature-Dependent Behaviors from Limited Data
• Authors: Yunrui Qiu, Richard John, Lukas Herron, Pratyush Tiwary,
• Abstract summary: We introduce Latent Thermodynamic Flows (LaTF), an end-to-end framework that tightly integrates representation learning and generative modeling.<n>LaTF unifies the State Predictive Information Bottleneck (SPIB) with NFs to simultaneously learn low-dimensional latent representations.<n>We demonstrate LaTF's effectiveness across diverse systems, including a model potential, the Chignolin protein, and cluster of Lennard Jones particles.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Accurate characterization of the equilibrium distributions of complex molecular systems and their dependence on environmental factors such as temperature is essential for understanding thermodynamic properties and transition mechanisms. Projecting these distributions onto meaningful low-dimensional representations enables interpretability and downstream analysis. Recent advances in generative AI, particularly flow models such as Normalizing Flows (NFs), have shown promise in modeling such distributions, but their scope is limited without tailored representation learning. In this work, we introduce Latent Thermodynamic Flows (LaTF), an end-to-end framework that tightly integrates representation learning and generative modeling. LaTF unifies the State Predictive Information Bottleneck (SPIB) with NFs to simultaneously learn low-dimensional latent representations, referred to as Collective Variables (CVs), classify metastable states, and generate equilibrium distributions across temperatures beyond the training data. The two components of representation learning and generative modeling are optimized jointly, ensuring that the learned latent features capture the system's slow, important degrees of freedom while the generative model accurately reproduces the system's equilibrium behavior. We demonstrate LaTF's effectiveness across diverse systems, including a model potential, the Chignolin protein, and cluster of Lennard Jones particles, with thorough evaluations and benchmarking using multiple metrics and extensive simulations. Finally, we apply LaTF to a RNA tetraloop system, where despite using simulation data from only two temperatures, LaTF reconstructs the temperature-dependent structural ensemble and melting behavior, consistent with experimental and prior extensive computational results.

Related papers

• Understanding temperature tuning in energy-based models [5.75145367989177]
  We show that learning from sparse data causes models to systematically overestimate high-energy state probabilities.<n>More generally, we characterize how the optimal sampling temperature depends on the interplay between data size and the system's underlying energy landscape.<n>Our framework thus casts post-hoc temperature tuning as a diagnostic tool that reveals properties of the true data distribution and the limits of the learned model.
  arXiv  Detail & Related papers  (2025-12-09T22:06:30Z)
• Optimized Machine Learning Methods for Studying the Thermodynamic Behavior of Complex Spin Systems [0.0]
  convolutional neural networks (CNNs) are an efficient and versatile tool for the analysis of critical and low-temperature phase states in spin system models.<n>It is shown that CNNs substantially reduce the root-mean-square error (RMSE) compared with fully connected architectures.
  arXiv  Detail & Related papers  (2025-12-08T11:33:24Z)
• Combined Representation and Generation with Diffusive State Predictive Information Bottleneck [0.0]
  We combine a time-lagged information bottleneck designed to characterize molecular important representations and a diffusion model in one joint training objective.<n>The resulting protocol, which we term Diffusive State Predictive Information Bottleneck (D-SPIB), enables the balancing of representation learning and generation aims in one flexible architecture.<n>We benchmark D-SPIB on multiple molecular tasks and showcase its potential for exploring physical conditions outside the training set.
  arXiv  Detail & Related papers  (2025-10-10T18:46:21Z)
• Thermodynamic Performance Limits for Score-Based Diffusion Models [0.0]
  We establish a connection between score-based diffusion models and non-equilibrium thermodynamics by deriving performance limits based on entropy rates.<n>Our framework connects generative modeling performance to fundamental physical principles through thermodynamics.
  arXiv  Detail & Related papers  (2025-10-07T17:35:18Z)
• Learning Kinetic Monte Carlo stochastic dynamics with Deep Generative Adversarial Networks [0.0]
  We show that Generative Adversarial Networks (GANs) may be fruitfully exploited to learn dynamics.<n>We showcase the application to a two-dimensional, many-particle system, focusing on surface-step fluctuations and on the related time-dependent roughness.
  arXiv  Detail & Related papers  (2025-07-29T12:48:03Z)
• CLOUD: A Scalable and Physics-Informed Foundation Model for Crystal Representation Learning [0.0]
  We introduce CLOUD (Crystal Language mOdel for Unified and Differentiable materials modeling), a transformer-based framework trained on a novel Symmetry-Consistent SCOPE (SCOPE)<n>CLOUD is pre-trained on over six million crystal structures and achieves competitive performance in predicting a wide range of material properties.<n>As proof of concept of differentiable materials modeling, CLOUD is applied to predict the phonon internal energy and heat capacity.
  arXiv  Detail & Related papers  (2025-06-19T15:45:24Z)
• Models of Heavy-Tailed Mechanistic Universality [62.107333654304014]
  We propose a family of random matrix models to explore attributes that give rise to heavy-tailed behavior in trained neural networks.<n>Under this model, spectral densities with power laws on tails arise through a combination of three independent factors.<n> Implications of our model on other appearances of heavy tails, including neural scaling laws, trajectories, and the five-plus-one phases of neural network training, are discussed.
  arXiv  Detail & Related papers  (2025-06-04T00:55:01Z)
• PIGPVAE: Physics-Informed Gaussian Process Variational Autoencoders [42.8983261737774]
  We propose a novel generative model that learns from limited data by incorporating physical constraints to enhance performance.<n>We extend the VAE architecture by incorporating physical models in the generative process, enabling it to capture underlying dynamics more effectively.<n>We demonstrate that PIGPVAE can produce realistic samples beyond the observed distribution, highlighting its robustness and usefulness under distribution shifts.
  arXiv  Detail & Related papers  (2025-05-25T21:12:01Z)
• Latent Space Energy-based Neural ODEs [73.01344439786524]
  This paper introduces novel deep dynamical models designed to represent continuous-time sequences.<n>We train the model using maximum likelihood estimation with Markov chain Monte Carlo.<n> Experimental results on oscillating systems, videos and real-world state sequences (MuJoCo) demonstrate that our model with the learnable energy-based prior outperforms existing counterparts.
  arXiv  Detail & Related papers  (2024-09-05T18:14:22Z)
• Efficient mapping of phase diagrams with conditional Boltzmann Generators [4.437335677401287]
  We develop deep generative machine learning models based on the Boltzmann Generator approach for entire phase diagrams. By training a single normalizing flow to transform the equilibrium distribution sampled at only one reference thermodynamic state to a wide range of target temperatures and pressures, we can efficiently generate equilibrium samples. We demonstrate our approach by predicting the solid-liquid coexistence line for a Lennard-Jones system in excellent agreement with state-of-the-art free energy methods.
  arXiv  Detail & Related papers  (2024-06-18T08:05:04Z)
• Deep generative modelling of canonical ensemble with differentiable thermal properties [0.9421843976231371]
  We propose a variational modelling method with differentiable temperature for canonical ensembles. Using a deep generative model, the free energy is estimated and minimized simultaneously in a continuous temperature range. The training process requires no dataset, and works with arbitrary explicit density generative models.
  arXiv  Detail & Related papers  (2024-04-29T03:41:49Z)
• Model-free quantification of completeness, uncertainties, and outliers in atomistic machine learning using information theory [4.59916193837551]
  atomistic machine learning (ML) often relies on unsupervised learning or model predictions to analyze information contents. Here, we introduce a theoretical framework that provides a rigorous, model-free tool to quantify information contents in atomistic simulations.
  arXiv  Detail & Related papers  (2024-04-18T17:50:15Z)
• Heat Death of Generative Models in Closed-Loop Learning [63.83608300361159]
  We study the learning dynamics of generative models that are fed back their own produced content in addition to their original training dataset. We show that, unless a sufficient amount of external data is introduced at each iteration, any non-trivial temperature leads the model to degenerate.
  arXiv  Detail & Related papers  (2024-04-02T21:51:39Z)
• Synthetic location trajectory generation using categorical diffusion models [50.809683239937584]
  Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data. We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
  arXiv  Detail & Related papers  (2024-02-19T15:57:39Z)
• Stochastic Thermodynamics of Learning Parametric Probabilistic Models [0.0]
  We introduce two information-theoretic metrics: Memorized-information (M-info) and Learned-information (L-info), which trace the flow of information during the learning process of Parametric Probabilistic Models (PPMs) We demonstrate that the accumulation of L-info during the learning process is associated with entropy production, and parameters serve as a heat reservoir in this process, capturing learned information in the form of M-info.
  arXiv  Detail & Related papers  (2023-10-04T01:32:55Z)
• Towards Predicting Equilibrium Distributions for Molecular Systems with Deep Learning [60.02391969049972]
  We introduce a novel deep learning framework, called Distributional Graphormer (DiG), in an attempt to predict the equilibrium distribution of molecular systems. DiG employs deep neural networks to transform a simple distribution towards the equilibrium distribution, conditioned on a descriptor of a molecular system.
  arXiv  Detail & Related papers  (2023-06-08T17:12:08Z)
• Capturing Actionable Dynamics with Structured Latent Ordinary Differential Equations [68.62843292346813]
  We propose a structured latent ODE model that captures system input variations within its latent representation. Building on a static variable specification, our model learns factors of variation for each input to the system, thus separating the effects of the system inputs in the latent space.
  arXiv  Detail & Related papers  (2022-02-25T20:00:56Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA [11.919315372249802]
  We consider the identifiability theory of probabilistic models. We show that our model can be used for the estimation of the components in the framework of Independently Modulated Component Analysis.
  arXiv  Detail & Related papers  (2020-02-26T14:43:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.