LLT: Local Linear Transformer for PDE Operator Learning

• URL: http://arxiv.org/abs/2607.07718v1
• Date: Sat, 04 Jul 2026 11:07:09 GMT
• Title: LLT: Local Linear Transformer for PDE Operator Learning
• Abstract summary: We introduce Local Linear Transformer (LLT) for PDE operator learning.<n>LLT combines linear global attention with local spatial mixing, and incorporates coordinate and geometry information.<n>We evaluate LLT on several PDE problems, including elasticity, plasticity, air flow, pipe flow, and Darcy flow.
• Score: 0.10742675209112622
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural operators have become a common approach for learning PDE solution maps and accelerating numerical simulations. Transformer-based neural operators are of particular interest, since attention can learn long-range dependencies in the computational domain. However, standard attention has two major limitations when applied to PDEs: it scales quadratically with the number of computational nodes, and it lacks an explicit bias toward local interactions. To address these issues, we introduce Local Linear Transformer (LLT) for PDE operator learning. The architecture combines linear global attention with local spatial mixing, and incorporates coordinate and geometry information. We evaluate LLT on several PDE problems, including elasticity, plasticity, airfoil flow, pipe flow, and Darcy flow. The reference data for these problems span finite-element, finite-volume, and finite-difference discretizations on structured and unstructured meshes. Compared with other neural-operator and transformer baselines from prior studies, LLT achieves competitive or lower relative $L_2$ error across these problems. On matched structured discretizations, wall-clock time per training iteration is reduced by factors of 1.8 to 2.5 relative to Transolver. We also scale the approach and apply it to a three-dimensional car aerodynamics dataset with 32,186 unstructured mesh points per sample. Together, these results indicate that LLT provides an accurate and computationally efficient operator for PDE problems across discretizations, mesh types, and problem settings.

Related papers

• Local gradient neural operator [12.997669738865484]
  We propose a lightweight and interpretable alternative for field temporal evolution prediction and source identification in typical mechanical problems.<n>The method builds on priors from nonlinear gradient discretization and uses multilayer perceptron convolutional layers to learn translation-invariant local kernels.<n>We evaluate the method on PDE benchmarks covering linear and nonlinear, static and dynamic, and low and high dimensional cases.
  arXiv  Detail & Related papers  (2026-09-07T16:53:47Z)
• CompNO: A Novel Foundation Model approach for solving Partial Differential Equations [0.0]
  Partial differential equations govern a wide range of physical phenomena, but their numerical solution remains computationally demanding.<n>Recent Scientific Foundation Models (SFMs) aim to alleviate this cost by learning universal surrogates from large collections of simulated systems.<n>We introduce Compositional Neural Operators (CompNO), a compositional neural operator framework for parametric PDEs.
  arXiv  Detail & Related papers  (2026-01-12T10:04:48Z)
• Tensor Gaussian Processes: Efficient Solvers for Nonlinear PDEs [16.38975732337055]
  TGPS is a machine learning solver for partial differential equations.<n>It reduces the task to learning a collection of one-dimensional GPs.<n>It achieves superior accuracy and efficiency compared to existing approaches.
  arXiv  Detail & Related papers  (2025-10-15T17:23:21Z)
• Geometric Operator Learning with Optimal Transport [77.16909146519227]
  We propose integrating optimal transport (OT) into operator learning for partial differential equations (PDEs) on complex geometries.<n>For 3D simulations focused on surfaces, our OT-based neural operator embeds the surface geometry into a 2D parameterized latent space.<n> Experiments with Reynolds-averaged Navier-Stokes equations (RANS) on the ShapeNet-Car and DrivAerNet-Car datasets show that our method achieves better accuracy and also reduces computational expenses.
  arXiv  Detail & Related papers  (2025-07-26T21:28:25Z)
• LaPON: A Lagrange's-mean-value-theorem-inspired operator network for solving PDEs and its application on NSE [8.014720523981385]
  We propose LaPON, an operator network inspired by the Lagrange's mean value theorem.<n>It embeds prior knowledge directly into the neural network architecture instead of the loss function.<n>LaPON provides a scalable and reliable solution for high-fidelity fluid dynamics simulation.
  arXiv  Detail & Related papers  (2025-05-18T10:45:17Z)
• Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum [78.27945336558987]
  Decentralized server (DFL) eliminates reliance on client-client architecture.<n>Non-smooth regularization is often incorporated into machine learning tasks.<n>We propose a novel novel DNCFL algorithm to solve these problems.
  arXiv  Detail & Related papers  (2025-04-17T08:32:25Z)
• Enabling Automatic Differentiation with Mollified Graph Neural Operators [73.52999622724101]
  We propose the mollified graph neural operator ($m$GNO), the first method to leverage automatic differentiation and compute exact gradients on arbitrary geometries.<n>For a PDE example on regular grids, $m$GNO paired with autograd reduced the L2 relative data error by 20x compared to finite differences.<n>It can also solve PDEs on unstructured point clouds seamlessly, using physics losses only, at resolutions vastly lower than those needed for finite differences to be accurate enough.
  arXiv  Detail & Related papers  (2025-04-11T06:16:30Z)
• Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs [85.40198664108624]
  We propose Codomain Attention Neural Operator (CoDA-NO) to solve multiphysics problems with PDEs. CoDA-NO tokenizes functions along the codomain or channel space, enabling self-supervised learning or pretraining of multiple PDE systems. We find CoDA-NO to outperform existing methods by over 36% on complex downstream tasks with limited data.
  arXiv  Detail & Related papers  (2024-03-19T08:56:20Z)
• Solving High-Dimensional PDEs with Latent Spectral Models [74.1011309005488]
  We present Latent Spectral Models (LSM) toward an efficient and precise solver for high-dimensional PDEs. Inspired by classical spectral methods in numerical analysis, we design a neural spectral block to solve PDEs in the latent space. LSM achieves consistent state-of-the-art and yields a relative gain of 11.5% averaged on seven benchmarks.
  arXiv  Detail & Related papers  (2023-01-30T04:58:40Z)
• LordNet: An Efficient Neural Network for Learning to Solve Parametric Partial Differential Equations without Simulated Data [47.49194807524502]
  We propose LordNet, a tunable and efficient neural network for modeling entanglements. The experiments on solving Poisson's equation and (2D and 3D) Navier-Stokes equation demonstrate that the long-range entanglements can be well modeled by the LordNet.
  arXiv  Detail & Related papers  (2022-06-19T14:41:08Z)
• Lie Point Symmetry Data Augmentation for Neural PDE Solvers [69.72427135610106]
  We present a method, which can partially alleviate this problem, by improving neural PDE solver sample complexity. In the context of PDEs, it turns out that we are able to quantitatively derive an exhaustive list of data transformations. We show how it can easily be deployed to improve neural PDE solver sample complexity by an order of magnitude.
  arXiv  Detail & Related papers  (2022-02-15T18:43:17Z)
• Non-linear Independent Dual System (NIDS) for Discretization-independent Surrogate Modeling over Complex Geometries [0.0]
  Non-linear independent dual system (NIDS) is a deep learning surrogate model for discretization-independent, continuous representation of PDE solutions. NIDS can be used for prediction over domains with complex, variable geometries and mesh topologies. Test cases include a vehicle problem with complex geometry and data scarcity, enabled by a training method.
  arXiv  Detail & Related papers  (2021-09-14T23:38:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.