Zero-shot generalization of transformer neural operators to larger domains

• URL: http://arxiv.org/abs/2606.14597v1
• Date: Fri, 12 Jun 2026 16:17:31 GMT
• Title: Zero-shot generalization of transformer neural operators to larger domains
• Abstract summary: Transformer-based neural operators have shown remarkable performance for approxing solution operators of partial differential equations on complex geometries.<n>Existing approaches implicitly assume a fixed domain size, which limits their ability to generalize at inference.<n>We investigate domain extension, namely zero-shot inference on spatial domains that are significantly larger than those encountered during training.
• Score: 5.254567635692465
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transformer-based neural operators have shown remarkable performance for approximating solution operators of partial differential equations on complex geometries. However, existing approaches implicitly assume a fixed domain size, which limits their ability to generalize at inference. In this work, we investigate domain extension, namely zero-shot inference on spatial domains that are significantly larger than those encountered during training. We argue that this setting fundamentally requires spatial locality and translation equivariance. We propose to implement this locality via a decomposable bias in the attention logits computation, enabling finely controllable locality while remaining fully decomposable into query-key inner products and directly compatible with optimized attention kernels. Combined with rotary positional embeddings, it enables expressive embeddings with controllable spatial support without altering the transformer architecture. We empirically show that our approach substantially improves zero-shot generalization to larger domains across two PDE benchmarks and a 3D industrial atmospheric flow application. Our code and datasets are available at https://github.com/cerea-daml/domain-extension.

Related papers

• EqGINO: Equivariant Geometry-Informed Fourier Neural Operators for 3D PDEs [12.013314817711054]
  We introduce EqGINO, a geometrically robust framework that enforces isotropy in the spectral domain.<n>Our method robustly models coordinate-invariant physical laws on complex irregular 3D geometries.
  arXiv  Detail & Related papers  (2026-06-02T07:23:06Z)
• A Turn Toward Better Alignment: Few-Shot Generative Adaptation with Equivariant Feature Rotation [67.2019317630466]
  Few-shot image generation aims to effectively adapt a source generative model to a target domain using very few training images.<n>We propose Equivariant Feature Rotation (EFR), a novel adaptation strategy that aligns source and target domains at two complementary levels.<n>Our method significantly enhances the generative performance within the targeted domain.
  arXiv  Detail & Related papers  (2025-12-24T13:48:22Z)
• From Local Interactions to Global Operators: Scalable Gaussian Process Operator for Physical Systems [7.807210884802377]
  We introduce a novel, scalable GPO that capitalizes on sparsity, locality, and structural information through judicious kernel design.<n>We demonstrate that our framework consistently achieves high accuracy across varying discretization scales.
  arXiv  Detail & Related papers  (2025-06-18T22:40:52Z)
• Mondrian: Transformer Operators via Domain Decomposition [2.1392064955842014]
  We introduce bfMondrian, transformer operators that decompose a domain into non-overlapping text.<n>Within each subdomain, it replaces standard layers with expressive neural operators, and attention is computed via softmax-based inner products over functions.<n>Mondrian achieves strong performance on Allen-Cahn and Navier-Stokes PDEs, demonstrating resolution scaling without retraining.
  arXiv  Detail & Related papers  (2025-06-09T20:52:04Z)
• GMapLatent: Geometric Mapping in Latent Space [51.317738404571514]
  Cross-domain generative models based on encoder-decoder AI architectures have attracted much attention in generating realistic images.<n>We introduce a canonical latent space representation based on geometric mapping to align the cross-domain latent spaces in a rigorous and precise manner.<n>Experiments on gray-scale and color images validate the efficiency, efficacy and applicability of GMapLatent.
  arXiv  Detail & Related papers  (2025-03-30T12:02:36Z)
• A general reduced-order neural operator for spatio-temporal predictive learning on complex spatial domains [1.708086375224371]
  This paper focuses on the unequal-domain mappings in predictive learning for complex processes (PL-STP) Recent advances in deep learning have revealed the great potential of neural operators (NOs) to learn operators directly from observational data. Existing NOs require input space and output space to be the same domain, which pose challenges in ensuring predictive accuracy and stability for unequal-domain mappings.
  arXiv  Detail & Related papers  (2024-09-09T11:02:27Z)
• Neural Operators with Localized Integral and Differential Kernels [77.76991758980003]
  We present a principled approach to operator learning that can capture local features under two frameworks. We prove that we obtain differential operators under an appropriate scaling of the kernel values of CNNs. To obtain local integral operators, we utilize suitable basis representations for the kernels based on discrete-continuous convolutions.
  arXiv  Detail & Related papers  (2024-02-26T18:59:31Z)
• Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEs [93.82811501035569]
  We introduce a new data efficient and highly parallelizable operator learning approach with reduced memory requirement and better generalization. MG-TFNO scales to large resolutions by leveraging local and global structures of full-scale, real-world phenomena. We demonstrate superior performance on the turbulent Navier-Stokes equations where we achieve less than half the error with over 150x compression.
  arXiv  Detail & Related papers  (2023-09-29T20:18:52Z)
• Domain-aware Triplet loss in Domain Generalization [0.0]
  Domain shift is caused by discrepancies in the distributions of the testing and training data. We design a domainaware triplet loss for domain generalization to help the model to cluster similar semantic features. Our algorithm is designed to disperse domain information in the embedding space.
  arXiv  Detail & Related papers  (2023-03-01T14:02:01Z)
• Exploring Sequence Feature Alignment for Domain Adaptive Detection Transformers [141.70707071815653]
  We propose a novel Sequence Feature Alignment (SFA) method that is specially designed for the adaptation of detection transformers. SFA consists of a domain query-based feature alignment (DQFA) module and a token-wise feature alignment (TDA) module. Experiments on three challenging benchmarks show that SFA outperforms state-of-the-art domain adaptive object detection methods.
  arXiv  Detail & Related papers  (2021-07-27T07:17:12Z)
• Bi-Directional Generation for Unsupervised Domain Adaptation [61.73001005378002]
  Unsupervised domain adaptation facilitates the unlabeled target domain relying on well-established source domain information. Conventional methods forcefully reducing the domain discrepancy in the latent space will result in the destruction of intrinsic data structure. We propose a Bi-Directional Generation domain adaptation model with consistent classifiers interpolating two intermediate domains to bridge source and target domains.
  arXiv  Detail & Related papers  (2020-02-12T09:45:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.