MMET: A Multi-Input and Multi-Scale Transformer for Efficient PDEs Solving

• URL: http://arxiv.org/abs/2506.17230v1
• Date: Sat, 24 May 2025 19:50:11 GMT
• Title: MMET: A Multi-Input and Multi-Scale Transformer for Efficient PDEs Solving
• Authors: Yichen Luo, Jia Wang, Dapeng Lan, Yu Liu, Zhibo Pang,
• Abstract summary: Multi-input and Multi-scale Efficient Transformer (MMET) is a novel framework designed to address the above challenges.<n>MMET decouples mesh and query points as two sequences and feeds them into the encoder and decoder, respectively.<n>This work highlights the potential of MMET as a robust and scalable solution for real-time PDE solving in engineering and physics-based applications.
• Score: 7.676857294785697
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Partial Differential Equations (PDEs) are fundamental for modeling physical systems, yet solving them in a generic and efficient manner using machine learning-based approaches remains challenging due to limited multi-input and multi-scale generalization capabilities, as well as high computational costs. This paper proposes the Multi-input and Multi-scale Efficient Transformer (MMET), a novel framework designed to address the above challenges. MMET decouples mesh and query points as two sequences and feeds them into the encoder and decoder, respectively, and uses a Gated Condition Embedding (GCE) layer to embed input variables or functions with varying dimensions, enabling effective solutions for multi-scale and multi-input problems. Additionally, a Hilbert curve-based reserialization and patch embedding mechanism decrease the input length. This significantly reduces the computational cost when dealing with large-scale geometric models. These innovations enable efficient representations and support multi-scale resolution queries for large-scale and multi-input PDE problems. Experimental evaluations on diverse benchmarks spanning different physical fields demonstrate that MMET outperforms SOTA methods in both accuracy and computational efficiency. This work highlights the potential of MMET as a robust and scalable solution for real-time PDE solving in engineering and physics-based applications, paving the way for future explorations into pre-trained large-scale models in specific domains. This work is open-sourced at https://github.com/YichenLuo-0/MMET.

Related papers

• CodePDE: An Inference Framework for LLM-driven PDE Solver Generation [57.15474515982337]
  Partial differential equations (PDEs) are fundamental to modeling physical systems.<n>Traditional numerical solvers rely on expert knowledge to implement and are computationally expensive.<n>We introduce CodePDE, the first inference framework for generating PDE solvers using large language models.
  arXiv  Detail & Related papers  (2025-05-13T17:58:08Z)
• Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries [67.63077028746191]
  Transolver++ is a highly parallel and efficient neural solver that can solve PDEs on million-scale geometries.<n>Transolver++ increases the single- GPU input capacity to million-scale points for the first time.<n>It achieves over 20% performance gain in million-scale high-fidelity industrial simulations.
  arXiv  Detail & Related papers  (2025-02-04T15:33:50Z)
• M2NO: Multiresolution Operator Learning with Multiwavelet-based Algebraic Multigrid Method [13.93532934867225]
  We introduce the Multiwavelet-based Algebraic Multigrid Neural Operator (M2NO), a novel deep learning framework. By exploiting the inherent similarities between these two approaches, M2NO enhances precision and flexibility across various PDE benchmarks. M2NO excels in handling high-resolution and super-resolution tasks, consistently outperforming competing models.
  arXiv  Detail & Related papers  (2024-06-07T10:47:40Z)
• Unisolver: PDE-Conditional Transformers Are Universal PDE Solvers [55.0876373185983]
  We present Unisolver, a novel Transformer model trained on diverse data and conditioned on diverse PDEs.<n>Unisolver achieves consistent state-of-the-art on three challenging large-scale benchmarks, showing impressive performance and generalizability.
  arXiv  Detail & Related papers  (2024-05-27T15:34:35Z)
• Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey [18.00772798876708]
  Efficient Fine-Tuning (PEFT) provides a practical solution by efficiently adjusting the large models over the various downstream tasks. PEFT refers to the process of adjusting the parameters of a pre-trained large model to adapt it to a specific task or domain. We present comprehensive studies of various PEFT algorithms, examining their performance and computational overhead.
  arXiv  Detail & Related papers  (2024-03-21T17:55:50Z)
• Kolmogorov n-Widths for Multitask Physics-Informed Machine Learning (PIML) Methods: Towards Robust Metrics [8.90237460752114]
  This topic encompasses a broad array of methods and models aimed at solving a single or a collection of PDE problems, called multitask learning. PIML is characterized by the incorporation of physical laws into the training process of machine learning models in lieu of large data when solving PDE problems.
  arXiv  Detail & Related papers  (2024-02-16T23:21:40Z)
• A Multi-Head Ensemble Multi-Task Learning Approach for Dynamical Computation Offloading [62.34538208323411]
  We propose a multi-head ensemble multi-task learning (MEMTL) approach with a shared backbone and multiple prediction heads (PHs) MEMTL outperforms benchmark methods in both the inference accuracy and mean square error without requiring additional training data.
  arXiv  Detail & Related papers  (2023-09-02T11:01:16Z)
• Challenges and opportunities for machine learning in multiscale computational modeling [0.0]
  Solving for complex multiscale systems remains computationally onerous due to the high dimensionality of the solution space. Machine learning (ML) has emerged as a promising solution that can either serve as a surrogate for, accelerate or augment traditional numerical methods. This paper provides a perspective on the opportunities and challenges of using ML for complex multiscale modeling and simulation.
  arXiv  Detail & Related papers  (2023-03-22T02:04:39Z)
• Solving High-Dimensional PDEs with Latent Spectral Models [74.1011309005488]
  We present Latent Spectral Models (LSM) toward an efficient and precise solver for high-dimensional PDEs. Inspired by classical spectral methods in numerical analysis, we design a neural spectral block to solve PDEs in the latent space. LSM achieves consistent state-of-the-art and yields a relative gain of 11.5% averaged on seven benchmarks.
  arXiv  Detail & Related papers  (2023-01-30T04:58:40Z)
• A composable autoencoder-based iterative algorithm for accelerating numerical simulations [0.0]
  CoAE-MLSim is an unsupervised, lower-dimensional, local method that is motivated from key ideas used in commercial PDE solvers. It is tested for a variety of complex engineering cases to demonstrate its computational speed, accuracy, scalability, and generalization across different PDE conditions.
  arXiv  Detail & Related papers  (2021-10-07T20:22:37Z)
• Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning [89.31889875864599]
  We propose an efficient model-based reinforcement learning algorithm for learning in multi-agent systems. Our main theoretical contributions are the first general regret bounds for model-based reinforcement learning for MFC. We provide a practical parametrization of the core optimization problem.
  arXiv  Detail & Related papers  (2021-07-08T18:01:02Z)
• A Bayesian Multiscale Deep Learning Framework for Flows in Random Media [0.0]
  Fine-scale simulation of complex systems governed by multiscale partial differential equations (PDEs) is computationally expensive and various multiscale methods have been developed for addressing such problems. In this work, we introduce a novel hybrid deep-learning and multiscale approach for multiscale PDEs with limited training data. For demonstration purposes, we focus on a porous media flow problem. We use an image-to-image supervised deep learning model to learn the mapping between the input permeability field and the multiscale basis functions.
  arXiv  Detail & Related papers  (2021-03-08T23:11:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.