Mondrian: Transformer Operators via Domain Decomposition

• URL: http://arxiv.org/abs/2506.08226v1
• Date: Mon, 09 Jun 2025 20:52:04 GMT
• Title: Mondrian: Transformer Operators via Domain Decomposition
• Authors: Arthur Feeney, Kuei-Hsiang Huang, Aparna Chandramowlishwaran,
• Abstract summary: We introduce bfMondrian, transformer operators that decompose a domain into non-overlapping text.<n>Within each subdomain, it replaces standard layers with expressive neural operators, and attention is computed via softmax-based inner products over functions.<n>Mondrian achieves strong performance on Allen-Cahn and Navier-Stokes PDEs, demonstrating resolution scaling without retraining.
• Score: 2.1392064955842014
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Operator learning enables data-driven modeling of partial differential equations (PDEs) by learning mappings between function spaces. However, scaling transformer-based operator models to high-resolution, multiscale domains remains a challenge due to the quadratic cost of attention and its coupling to discretization. We introduce \textbf{Mondrian}, transformer operators that decompose a domain into non-overlapping subdomains and apply attention over sequences of subdomain-restricted functions. Leveraging principles from domain decomposition, Mondrian decouples attention from discretization. Within each subdomain, it replaces standard layers with expressive neural operators, and attention across subdomains is computed via softmax-based inner products over functions. The formulation naturally extends to hierarchical windowed and neighborhood attention, supporting both local and global interactions. Mondrian achieves strong performance on Allen-Cahn and Navier-Stokes PDEs, demonstrating resolution scaling without retraining. These results highlight the promise of domain-decomposed attention for scalable and general-purpose neural operators.

Related papers

• Non-overlapping, Schwarz-type Domain Decomposition Method for Physics and Equality Constrained Artificial Neural Networks [0.24578723416255746]
  We present a non-overlapping, Schwarz-type domain decomposition method with a generalized interface condition. Our approach employs physics and equality-constrained artificial neural networks (PECANN) within each subdomain. A distinct advantage our domain decomposition method is its ability to learn solutions to both Poisson's and Helmholtz equations.
  arXiv  Detail & Related papers  (2024-09-20T16:48:55Z)
• A general reduced-order neural operator for spatio-temporal predictive learning on complex spatial domains [1.708086375224371]
  This paper focuses on the unequal-domain mappings in predictive learning for complex processes (PL-STP) Recent advances in deep learning have revealed the great potential of neural operators (NOs) to learn operators directly from observational data. Existing NOs require input space and output space to be the same domain, which pose challenges in ensuring predictive accuracy and stability for unequal-domain mappings.
  arXiv  Detail & Related papers  (2024-09-09T11:02:27Z)
• Learning the boundary-to-domain mapping using Lifting Product Fourier Neural Operators for partial differential equations [5.5927988408828755]
  We present a novel FNO-based architecture, named Lifting Product FNO (or LP-FNO) which can map arbitrary boundary functions to a solution in the entire domain. We demonstrate the efficacy and resolution independence of the proposed LP-FNO for the 2D Poisson equation.
  arXiv  Detail & Related papers  (2024-06-24T15:45:37Z)
• StyDeSty: Min-Max Stylization and Destylization for Single Domain Generalization [85.18995948334592]
  Single domain generalization (single DG) aims at learning a robust model generalizable to unseen domains from only one training domain. State-of-the-art approaches have mostly relied on data augmentations, such as adversarial perturbation and style enhancement, to synthesize new data. We propose emphStyDeSty, which explicitly accounts for the alignment of the source and pseudo domains in the process of data augmentation.
  arXiv  Detail & Related papers  (2024-06-01T02:41:34Z)
• Neural Operators with Localized Integral and Differential Kernels [77.76991758980003]
  We present a principled approach to operator learning that can capture local features under two frameworks. We prove that we obtain differential operators under an appropriate scaling of the kernel values of CNNs. To obtain local integral operators, we utilize suitable basis representations for the kernels based on discrete-continuous convolutions.
  arXiv  Detail & Related papers  (2024-02-26T18:59:31Z)
• Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEs [93.82811501035569]
  We introduce a new data efficient and highly parallelizable operator learning approach with reduced memory requirement and better generalization. MG-TFNO scales to large resolutions by leveraging local and global structures of full-scale, real-world phenomena. We demonstrate superior performance on the turbulent Navier-Stokes equations where we achieve less than half the error with over 150x compression.
  arXiv  Detail & Related papers  (2023-09-29T20:18:52Z)
• A Generalized Schwarz-type Non-overlapping Domain Decomposition Method using Physics-constrained Neural Networks [0.9137554315375919]
  We present a meshless Schwarz-type non-overlapping domain decomposition based on artificial neural networks. Our method is applicable to both the Laplace's and Helmholtz equations.
  arXiv  Detail & Related papers  (2023-07-23T21:18:04Z)
• Efficient Hierarchical Domain Adaptation for Pretrained Language Models [77.02962815423658]
  Generative language models are trained on diverse, general domain corpora. We introduce a method to scale domain adaptation to many diverse domains using a computationally efficient adapter approach.
  arXiv  Detail & Related papers  (2021-12-16T11:09:29Z)
• Neural Operator: Learning Maps Between Function Spaces [75.93843876663128]
  We propose a generalization of neural networks to learn operators, termed neural operators, that map between infinite dimensional function spaces. We prove a universal approximation theorem for our proposed neural operator, showing that it can approximate any given nonlinear continuous operator. An important application for neural operators is learning surrogate maps for the solution operators of partial differential equations.
  arXiv  Detail & Related papers  (2021-08-19T03:56:49Z)
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection [90.18752912204778]
  Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications. We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training. Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
  arXiv  Detail & Related papers  (2021-06-10T05:01:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.