Self-Supervised Evolution Operator Learning for High-Dimensional Dynamical Systems

• URL: http://arxiv.org/abs/2505.18671v1
• Date: Sat, 24 May 2025 12:18:19 GMT
• Title: Self-Supervised Evolution Operator Learning for High-Dimensional Dynamical Systems
• Authors: Giacomo Turri, Luigi Bonati, Kai Zhu, Massimiliano Pontil, Pietro Novelli,
• Abstract summary: We introduce an encoder-only approach to learn the evolution operators of non-scale dynamical systems.<n>We show it across multiple scientific domains: explaining the folding dynamics of small proteins in host sites, and autonomously finding patterns in climate data.
• Score: 24.014942930679855
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce an encoder-only approach to learn the evolution operators of large-scale non-linear dynamical systems, such as those describing complex natural phenomena. Evolution operators are particularly well-suited for analyzing systems that exhibit complex spatio-temporal patterns and have become a key analytical tool across various scientific communities. As terabyte-scale weather datasets and simulation tools capable of running millions of molecular dynamics steps per day are becoming commodities, our approach provides an effective tool to make sense of them from a data-driven perspective. The core of it lies in a remarkable connection between self-supervised representation learning methods and the recently established learning theory of evolution operators. To show the usefulness of the proposed method, we test it across multiple scientific domains: explaining the folding dynamics of small proteins, the binding process of drug-like molecules in host sites, and autonomously finding patterns in climate data. Code and data to reproduce the experiments are made available open source.

Related papers

• AI4X Roadmap: Artificial Intelligence for the advancement of scientific pursuit and its future directions [65.44445343399126]
  We look at AI-enabled science across biology, chemistry, climate science, mathematics, materials science, physics, self-driving laboratories and unconventional computing.<n>Several shared themes emerge: the need for diverse and trustworthy data, transferable electronic-structure and interatomic models, AI systems integrated into end-to-end scientific synthesis.<n>Across domains, we highlight how large foundation models, active learning and self-driving laboratories can close loops between prediction and validation.
  arXiv  Detail & Related papers  (2025-11-26T02:10:28Z)
• Foundation Models for Discovery and Exploration in Chemical Space [57.97784111110166]
  MIST is a family of molecular foundation models trained on large unlabeled datasets.<n>We demonstrate the ability of these models to solve real-world problems across chemical space.
  arXiv  Detail & Related papers  (2025-10-20T17:56:01Z)
• Deep Learning in Astrophysics [0.2700171473617699]
  Deep learning has generated diverse perspectives in astronomy, with ongoing discussions between proponents and skeptics motivating this review.<n>We examine how neural networks complement classical statistics, extending our data analytical toolkit for modern surveys.<n>This review demonstrates how deep learning incorporates domain knowledge through architectural design, with built-in assumptions guiding models toward physically meaningful solutions.
  arXiv  Detail & Related papers  (2025-10-12T17:31:46Z)
• Agentic Exploration of Physics Models [0.0]
  We introduce SciExplorer, an agent that enables exploration of systems without any domain-specific blueprints.<n>We test SciExplorer on a broad set of models spanning mechanical dynamical systems, wave evolution, and quantum many-body physics.<n>The demonstrated effectiveness of this setup opens the door to similar scientific exploration in other domains.
  arXiv  Detail & Related papers  (2025-09-29T16:07:05Z)
• A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers [251.23085679210206]
  Scientific Large Language Models (Sci-LLMs) are transforming how knowledge is represented, integrated, and applied in scientific research.<n>This survey reframes the development of Sci-LLMs as a co-evolution between models and their underlying data substrate.<n>We formulate a unified taxonomy of scientific data and a hierarchical model of scientific knowledge.
  arXiv  Detail & Related papers  (2025-08-28T18:30:52Z)
• A Survey of Self-Evolving Agents: On Path to Artificial Super Intelligence [87.08051686357206]
  Large Language Models (LLMs) have demonstrated strong capabilities but remain fundamentally static.<n>As LLMs are increasingly deployed in open-ended, interactive environments, this static nature has become a critical bottleneck.<n>This survey provides the first systematic and comprehensive review of self-evolving agents.
  arXiv  Detail & Related papers  (2025-07-28T17:59:05Z)
• Molecular Machine Learning Using Euler Characteristic Transforms [12.108680020079925]
  Shape of a molecule determines its physicochemical and biological properties.<n>We propose using the Euler Characteristic Transform (ECT) as a geometrical-topological descriptor.<n>ECT enables the extraction of multiscale structural features, offering a novel way to represent and encode molecular shape in the feature space.
  arXiv  Detail & Related papers  (2025-07-04T10:57:40Z)
• Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond [61.18736646013446]
  In pursuit of a deeper understanding of its surprising behaviors, we investigate the utility of a simple yet accurate model of a trained neural network. Across three case studies, we illustrate how it can be applied to derive new empirical insights on a diverse range of prominent phenomena.
  arXiv  Detail & Related papers  (2024-10-31T22:54:34Z)
• Physical Consistency Bridges Heterogeneous Data in Molecular Multi-Task Learning [79.75718786477638]
  We exploit the specialty of molecular tasks that there are physical laws connecting them, and design consistency training approaches. We demonstrate that the more accurate energy data can improve the accuracy of structure prediction. We also find that consistency training can directly leverage force and off-equilibrium structure data to improve structure prediction.
  arXiv  Detail & Related papers  (2024-10-14T03:11:33Z)
• Learning minimal representations of stochastic processes with variational autoencoders [52.99137594502433]
  We introduce an unsupervised machine learning approach to determine the minimal set of parameters required to describe a process. Our approach enables for the autonomous discovery of unknown parameters describing processes.
  arXiv  Detail & Related papers  (2023-07-21T14:25:06Z)
• Learning Latent Dynamics via Invariant Decomposition and (Spatio-)Temporal Transformers [0.6767885381740952]
  We propose a method for learning dynamical systems from high-dimensional empirical data. We focus on the setting in which data are available from multiple different instances of a system. We study behaviour through simple theoretical analyses and extensive experiments on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-06-21T07:52:07Z)
• Implicit Geometry and Interaction Embeddings Improve Few-Shot Molecular Property Prediction [53.06671763877109]
  We develop molecular embeddings that encode complex molecular characteristics to improve the performance of few-shot molecular property prediction. Our approach leverages large amounts of synthetic data, namely the results of molecular docking calculations. On multiple molecular property prediction benchmarks, training from the embedding space substantially improves Multi-Task, MAML, and Prototypical Network few-shot learning performance.
  arXiv  Detail & Related papers  (2023-02-04T01:32:40Z)
• Bridging the Gap to Real-World Object-Centric Learning [66.55867830853803]
  We show that reconstructing features from models trained in a self-supervised manner is a sufficient training signal for object-centric representations to arise in a fully unsupervised way. Our approach, DINOSAUR, significantly out-performs existing object-centric learning models on simulated data.
  arXiv  Detail & Related papers  (2022-09-29T15:24:47Z)
• A Deep Learning Approach to Analyzing Continuous-Time Systems [20.89961728689037]
  We show that deep learning can be used to analyze complex processes. Our approach relaxes standard assumptions that are implausible for many natural systems. We demonstrate substantial improvements on behavioral and neuroimaging data.
  arXiv  Detail & Related papers  (2022-09-25T03:02:31Z)
• Transferring Chemical and Energetic Knowledge Between Molecular Systems with Machine Learning [5.27145343046974]
  We propose a novel methodology for transferring knowledge obtained from simple molecular systems to a more complex one. We focus on the classification of high and low free-energy states. Our results show a remarkable AUC of 0.92 for transfer learning from tri-alanine to the deca-alanine system.
  arXiv  Detail & Related papers  (2022-05-06T16:21:00Z)
• GEM: Group Enhanced Model for Learning Dynamical Control Systems [78.56159072162103]
  We build effective dynamical models that are amenable to sample-based learning. We show that learning the dynamics on a Lie algebra vector space is more effective than learning a direct state transition model. This work sheds light on a connection between learning of dynamics and Lie group properties, which opens doors for new research directions.
  arXiv  Detail & Related papers  (2021-04-07T01:08:18Z)
• Deep learning reveals hidden interactions in complex systems [0.0]
  AgentNet is a model-free data-driven framework consisting of deep neural networks to reveal hidden interactions in complex systems. A demonstration with empirical data from a flock of birds showed that AgentNet could identify hidden interaction ranges exhibited by real birds.
  arXiv  Detail & Related papers  (2020-01-03T02:25:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.