MSPT: Efficient Large-Scale Physical Modeling via Parallelized Multi-Scale Attention

• URL: http://arxiv.org/abs/2512.01738v1
• Date: Mon, 01 Dec 2025 14:43:46 GMT
• Title: MSPT: Efficient Large-Scale Physical Modeling via Parallelized Multi-Scale Attention
• Authors: Pedro M. P. Curvo, Jan-Willem van de Meent, Maksim Zhdanov,
• Abstract summary: We introduce the Multi-Scale Patch Transformer (MSPT), an architecture that combines local point attention within patches with global attention to coarse patch-level representations.<n>We validate our method on standard PDE benchmarks (elasticity, plasticity, fluid dynamics, porous flow) and large-scale aerodynamic datasets.
• Score: 8.605284164957984
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A key scalability challenge in neural solvers for industrial-scale physics simulations is efficiently capturing both fine-grained local interactions and long-range global dependencies across millions of spatial elements. We introduce the Multi-Scale Patch Transformer (MSPT), an architecture that combines local point attention within patches with global attention to coarse patch-level representations. To partition the input domain into spatially-coherent patches, we employ ball trees, which handle irregular geometries efficiently. This dual-scale design enables MSPT to scale to millions of points on a single GPU. We validate our method on standard PDE benchmarks (elasticity, plasticity, fluid dynamics, porous flow) and large-scale aerodynamic datasets (ShapeNet-Car, Ahmed-ML), achieving state-of-the-art accuracy with substantially lower memory footprint and computational cost.

Related papers

• MeshGraphNet-Transformer: Scalable Mesh-based Learned Simulation for Solid Mechanics [0.0]
  We present MeshGraphNet-Transformer (MGN-T), a novel architecture that combines the global modeling capabilities of Transformers with the geometric inductive bias of MeshGraphNets.<n>MGN-T overcomes a key limitation of standard MGN, the inefficient long-range information propagation caused by iterative message passing on large, high-resolution meshes.<n>We demonstrate that MGN-T successfully handles industrial-scale meshes for impact dynamics, a setting in which standard MGN fails due message-passing under-reaching.
  arXiv  Detail & Related papers  (2026-01-30T17:02:47Z)
• THOR: A Versatile Foundation Model for Earth Observation Climate and Society Applications [9.852915112122567]
  THOR is a "computeadaptive" foundation model that solves both input heterogeneity and deployment rigidity.<n>We pre-train THOR with a novel randomized patch and input image size strategy.<n>This allows a single set of pre-trained weights to be deployed at inference with any patch size, enabling a dynamic trade-off between computational cost and feature resolution without retraining.
  arXiv  Detail & Related papers  (2026-01-22T14:38:00Z)
• Adaptive Mesh-Quantization for Neural PDE Solvers [51.26961483962011]
  Graph Neural Networks can handle the irregular meshes required for complex geometries and boundary conditions, but still apply uniform computational effort across all nodes.<n>We propose Adaptive Mesh Quantization: spatially adaptive quantization across mesh node, edge, and cluster features, dynamically adjusting the bit-width used by a quantized model.<n>We demonstrate our framework's effectiveness by integrating it with two state-of-the-art models, MP-PDE and GraphViT, to evaluate performance across multiple tasks.
  arXiv  Detail & Related papers  (2025-11-23T14:47:24Z)
• GFocal: A Global-Focal Neural Operator for Solving PDEs on Arbitrary Geometries [5.323843026995587]
  Transformer-based neural operators have emerged as promising surrogate solvers for partial differential equations.<n>We propose GFocal, a method that enforces simultaneous global and local feature learning and fusion.<n>Experiments show that GFocal achieves state-of-the-art performance with an average 15.2% relative gain in five out of six benchmarks.
  arXiv  Detail & Related papers  (2025-08-06T14:02:39Z)
• Scaling Intelligence: Designing Data Centers for Next-Gen Language Models [0.6168147650666682]
  Large Language Models (LLMs), such as GPT-4 with 1.8 trillion parameters, demand a fundamental rethinking of data center architecture.<n>Our work provides a comprehensive co-design framework that jointly explores FLOPS, bandwidth and capacity, multiple network topologies.<n>We quantify the benefits of overlapping compute and communication, leveraging hardware-accelerated collectives, widening the scale-out domain, and increasing memory capacity.
  arXiv  Detail & Related papers  (2025-06-17T22:29:37Z)
• On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentation [52.96632954620623]
  We introduce a novel geometry-aware PEFT module specifically designed for 3D point cloud transformers.<n>Our approach sets a new benchmark for efficient, scalable, and geometry-aware fine-tuning of large-scale 3D point cloud models.
  arXiv  Detail & Related papers  (2025-05-28T15:08:36Z)
• Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries [67.63077028746191]
  Transolver++ is a highly parallel and efficient neural solver that can solve PDEs on million-scale geometries.<n>Transolver++ increases the single- GPU input capacity to million-scale points for the first time.<n>It achieves over 20% performance gain in million-scale high-fidelity industrial simulations.
  arXiv  Detail & Related papers  (2025-02-04T15:33:50Z)
• GERA: Geometric Embedding for Efficient Point Registration Analysis [20.690695788384517]
  We propose a novel point cloud registration network that leverages a pure geometric architecture, constructing geometric information offline. Our method is the first to replace 3D coordinate inputs with offline-constructed geometric encoding, improving generalization and stability.
  arXiv  Detail & Related papers  (2024-10-01T11:19:56Z)
• Towards Universal Mesh Movement Networks [13.450178050669964]
  We introduce the Universal Mesh Movement Network (UM2N)<n>UM2N can be applied in a non-intrusive, zero-shot manner to move meshes with different size distributions and structures.<n>We evaluate our method on advection and Navier-Stokes based examples, as well as a real-world tsunami simulation case.
  arXiv  Detail & Related papers  (2024-06-29T09:35:12Z)
• Transolver: A Fast Transformer Solver for PDEs on General Geometries [66.82060415622871]
  We present Transolver, which learns intrinsic physical states hidden behind discretized geometries. By calculating attention to physics-aware tokens encoded from slices, Transovler can effectively capture intricate physical correlations. Transolver achieves consistent state-of-the-art with 22% relative gain across six standard benchmarks and also excels in large-scale industrial simulations.
  arXiv  Detail & Related papers  (2024-02-04T06:37:38Z)
• DilateFormer: Multi-Scale Dilated Transformer for Visual Recognition [62.95223898214866]
  We explore effective Vision Transformers to pursue a preferable trade-off between the computational complexity and size of the attended receptive field. With a pyramid architecture, we construct a Multi-Scale Dilated Transformer (DilateFormer) by stacking MSDA blocks at low-level stages and global multi-head self-attention blocks at high-level stages. Our experiment results show that our DilateFormer achieves state-of-the-art performance on various vision tasks.
  arXiv  Detail & Related papers  (2023-02-03T14:59:31Z)
• Joint Spatial-Temporal and Appearance Modeling with Transformer for Multiple Object Tracking [59.79252390626194]
  We propose a novel solution named TransSTAM, which leverages Transformer to model both the appearance features of each object and the spatial-temporal relationships among objects. The proposed method is evaluated on multiple public benchmarks including MOT16, MOT17, and MOT20, and it achieves a clear performance improvement in both IDF1 and HOTA.
  arXiv  Detail & Related papers  (2022-05-31T01:19:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.