Representation Learning for Spatiotemporal Physical Systems

• URL: http://arxiv.org/abs/2603.13227v1
• Date: Fri, 13 Mar 2026 17:59:51 GMT
• Title: Representation Learning for Spatiotemporal Physical Systems
• Abstract summary: Machine learning approaches to physical systems have primarily focused on next-frame prediction.<n>We look at scientific tasks further downstream of predicting the next frame, such as estimation of a system's governing physical parameters.<n>We evaluate the effectiveness of general-purpose self-supervised methods in learning physics-grounded representations.
• Score: 43.674371165000544
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning approaches to spatiotemporal physical systems have primarily focused on next-frame prediction, with the goal of learning an accurate emulator for the system's evolution in time. However, these emulators are computationally expensive to train and are subject to performance pitfalls, such as compounding errors during autoregressive rollout. In this work, we take a different perspective and look at scientific tasks further downstream of predicting the next frame, such as estimation of a system's governing physical parameters. Accuracy on these tasks offers a uniquely quantifiable glimpse into the physical relevance of the representations of these models. We evaluate the effectiveness of general-purpose self-supervised methods in learning physics-grounded representations that are useful for downstream scientific tasks. Surprisingly, we find that not all methods designed for physical modeling outperform generic self-supervised learning methods on these tasks, and methods that learn in the latent space (e.g., joint embedding predictive architectures, or JEPAs) outperform those optimizing pixel-level prediction objectives. Code is available at https://github.com/helenqu/physical-representation-learning.

Related papers

• Adaptive Sensing of Continuous Physical Systems for Machine Learning [1.2891210250935148]
  We propose a general computing framework for adaptive information extraction from dynamical systems.<n>We show that adaptive spatial sensing significantly improves prediction accuracy on canonical chaotic benchmarks.
  arXiv  Detail & Related papers  (2026-03-04T02:13:36Z)
• From Physics to Machine Learning and Back: Part II - Learning and Observational Bias in PHM [52.64097278841485]
  Review examines how incorporating learning and observational biases through physics-informed modeling and data strategies can guide models toward physically consistent and reliable predictions.<n>Fast adaptation methods including meta-learning and few-shot learning are reviewed alongside domain generalization techniques.
  arXiv  Detail & Related papers  (2025-09-25T14:15:43Z)
• What Makes Pre-Trained Visual Representations Successful for Robust Manipulation? [57.92924256181857]
  We find that visual representations designed for manipulation and control tasks do not necessarily generalize under subtle changes in lighting and scene texture. We find that emergent segmentation ability is a strong predictor of out-of-distribution generalization among ViT models.
  arXiv  Detail & Related papers  (2023-11-03T18:09:08Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• Which priors matter? Benchmarking models for learning latent dynamics [70.88999063639146]
  Several methods have proposed to integrate priors from classical mechanics into machine learning models. We take a sober look at the current capabilities of these models. We find that the use of continuous and time-reversible dynamics benefits models of all classes.
  arXiv  Detail & Related papers  (2021-11-09T23:48:21Z)
• An Extensible Benchmark Suite for Learning to Simulate Physical Systems [60.249111272844374]
  We introduce a set of benchmark problems to take a step towards unified benchmarks and evaluation protocols. We propose four representative physical systems, as well as a collection of both widely used classical time-based and representative data-driven methods.
  arXiv  Detail & Related papers  (2021-08-09T17:39:09Z)
• Fast Modeling and Understanding Fluid Dynamics Systems with Encoder-Decoder Networks [0.0]
  We show that an accurate deep-learning-based proxy model can be taught efficiently by a finite-volume-based simulator. Compared to traditional simulation, the proposed deep learning approach enables much faster forward computation. We quantify the sensitivity of the deep learning model to key physical parameters and hence demonstrate that the inversion problems can be solved with great acceleration.
  arXiv  Detail & Related papers  (2020-06-09T17:14:08Z)
• Accurately Solving Physical Systems with Graph Learning [22.100386288615006]
  We introduce a novel method to accelerate iterative solvers for physical systems with graph networks. Unlike existing methods that aim to learn physical systems in an end-to-end manner, our approach guarantees long-term stability. Our method improves the run time performance of traditional iterative solvers.
  arXiv  Detail & Related papers  (2020-06-06T15:48:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.