AE-ViT: Stable Long-Horizon Parametric Partial Differential Equations Modeling

• URL: http://arxiv.org/abs/2604.06475v1
• Date: Tue, 07 Apr 2026 21:19:45 GMT
• Title: AE-ViT: Stable Long-Horizon Parametric Partial Differential Equations Modeling
• Abstract summary: We propose a joint model consisting of a convolutional encoder, a transformer operating on latent representations, and a decoder for reconstruction.<n>Our approach combines the efficiency of latent evolution with the fidelity of full-field models, outperforming DL-ROMs, latent transformers, and plain ViTs in multi-field prediction.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Learning Reduced Order Models (ROMs) are becoming increasingly popular as surrogate models for parametric partial differential equations (PDEs) due to their ability to handle high-dimensional data, approximate highly nonlinear mappings, and utilize GPUs. Existing approaches typically learn evolution either on the full solution field, which requires capturing long-range spatial interactions at high computational cost, or on compressed latent representations obtained from autoencoders, which reduces the cost but often yields latent vectors that are difficult to evolve, since they primarily encode spatial information. Moreover, in parametric PDEs, the initial condition alone is not sufficient to determine the trajectory, and most current approaches are not evaluated on jointly predicting multiple solution components with differing magnitudes and parameter sensitivities. To address these challenges, we propose a joint model consisting of a convolutional encoder, a transformer operating on latent representations, and a decoder for reconstruction. The main novelties are joint training with multi-stage parameter injection and coordinate channel injection. Parameters are injected at multiple stages to improve conditioning. Physical coordinates are encoded to provide spatial information. This allows the model to dynamically adapt its computations to the specific PDE parameters governing each system, rather than learning a single fixed response. Experiments on the Advection-Diffusion-Reaction equation and Navier-Stokes flow around the cylinder wake demonstrate that our approach combines the efficiency of latent evolution with the fidelity of full-field models, outperforming DL-ROMs, latent transformers, and plain ViTs in multi-field prediction, reducing the relative rollout error by approximately $5$ times.

Related papers

• Equation Recast for Canonical Operator Learning Across Parametric PDEs [59.44573033543284]
  We introduce equation recast, which reformulates parametric operator learning as the learning of a single canonical operator.<n>In high-fidelity tokamak simulations for nuclear fusion, the framework unifies electron-temperature data across four device geometries.
  arXiv  Detail & Related papers  (2026-09-02T13:15:50Z)
• Efficient Dilated Squeeze and Excitation Neural Operator for Differential Equations [2.5796735709994643]
  D-SENO is a lightweight operator learning framework for efficiently solving a wide range of partial differential equations.<n>D-SENO combines dilated convolution (DC) blocks with squeeze-and-excitation (SE) modules to jointly capture wide receptive fields and dynamics.
  arXiv  Detail & Related papers  (2026-01-24T10:38:06Z)
• Parallel Diffusion Solver via Residual Dirichlet Policy Optimization [88.7827307535107]
  Diffusion models (DMs) have achieved state-of-the-art generative performance but suffer from high sampling latency due to their sequential denoising nature.<n>Existing solver-based acceleration methods often face significant image quality degradation under a low-dimensional budget.<n>We propose the Ensemble Parallel Direction solver (dubbed as EPD-EPr), a novel ODE solver that mitigates these errors by incorporating multiple gradient parallel evaluations in each step.
  arXiv  Detail & Related papers  (2025-12-28T05:48:55Z)
• Towards Fast Coarse-graining and Equation Discovery with Foundation Inference Models [6.403678133359229]
  latent dynamics in high-dimensional recordings are often characterized by a much smaller set of effective variables.<n>Most machine learning approaches tackle these tasks jointly by training autoencoders together with models that enforce dynamical consistency.<n>We propose to decouple the two problems by leveraging the recently introduced Foundation Inference Models (FIMs)<n>A proof of concept on a double-well system with semicircle diffusion, embedded into synthetic video data, illustrates the potential of this approach for fast and reusable coarse-graining pipelines.
  arXiv  Detail & Related papers  (2025-10-14T15:17:23Z)
• Enhanced accuracy through ensembling of randomly initialized auto-regressive models for time-dependent PDEs [0.0]
  Autoregressive inference with machine learning models suffer from error accumulation over successive predictions, limiting their long-term accuracy.<n>We propose a deep ensemble framework to address this challenge, where multiple ML surrogate models are trained in parallel and aggregated during inference.<n>We validate the framework on three PDE-driven dynamical systems - stress evolution in heterogeneous microstructures, Gray-Scott reaction-diffusion, and planetary-scale shallow water system.
  arXiv  Detail & Related papers  (2025-07-05T02:25:12Z)
• CALM-PDE: Continuous and Adaptive Convolutions for Latent Space Modeling of Time-dependent PDEs [21.325228694415085]
  CALM-PDE is a model class that efficiently solves arbitrarily discretized PDEs in a compressed latent space.<n>We introduce a novel continuous convolution-based encoder-decoder architecture that uses an epsilon-neighborhood-constrained kernel.
  arXiv  Detail & Related papers  (2025-05-19T10:31:30Z)
• Complexity Matters: Rethinking the Latent Space for Generative Modeling [65.64763873078114]
  In generative modeling, numerous successful approaches leverage a low-dimensional latent space, e.g., Stable Diffusion. In this study, we aim to shed light on this under-explored topic by rethinking the latent space from the perspective of model complexity.
  arXiv  Detail & Related papers  (2023-07-17T07:12:29Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Solving High-Dimensional PDEs with Latent Spectral Models [74.1011309005488]
  We present Latent Spectral Models (LSM) toward an efficient and precise solver for high-dimensional PDEs. Inspired by classical spectral methods in numerical analysis, we design a neural spectral block to solve PDEs in the latent space. LSM achieves consistent state-of-the-art and yields a relative gain of 11.5% averaged on seven benchmarks.
  arXiv  Detail & Related papers  (2023-01-30T04:58:40Z)
• Non-linear Independent Dual System (NIDS) for Discretization-independent Surrogate Modeling over Complex Geometries [0.0]
  Non-linear independent dual system (NIDS) is a deep learning surrogate model for discretization-independent, continuous representation of PDE solutions. NIDS can be used for prediction over domains with complex, variable geometries and mesh topologies. Test cases include a vehicle problem with complex geometry and data scarcity, enabled by a training method.
  arXiv  Detail & Related papers  (2021-09-14T23:38:41Z)
• Large-scale Neural Solvers for Partial Differential Equations [48.7576911714538]
  Solving partial differential equations (PDE) is an indispensable part of many branches of science as many processes can be modelled in terms of PDEs. Recent numerical solvers require manual discretization of the underlying equation as well as sophisticated, tailored code for distributed computing. We examine the applicability of continuous, mesh-free neural solvers for partial differential equations, physics-informed neural networks (PINNs) We discuss the accuracy of GatedPINN with respect to analytical solutions -- as well as state-of-the-art numerical solvers, such as spectral solvers.
  arXiv  Detail & Related papers  (2020-09-08T13:26:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.