Eradicating Negative Transfer in Multi-Physics Foundation Models via Sparse Mixture-of-Experts Routing

• URL: http://arxiv.org/abs/2605.15179v1
• Date: Thu, 14 May 2026 17:58:15 GMT
• Title: Eradicating Negative Transfer in Multi-Physics Foundation Models via Sparse Mixture-of-Experts Routing
• Abstract summary: Shodh-MoE is a sparse-activated latent transformer architecture for multi-physics transport.<n>A Top-1 router dynamically assigns localized latent patches to expertworks.<n>Results support sparse expert routing as a practical architectural mechanism for mitigating multi-physics interference in universal neural operators.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scaling Scientific Machine Learning (SciML) toward universal foundation models is bottlenecked by negative transfer: the simultaneous co-training of disparate partial differential equation (PDE) regimes can induce gradient conflict, unstable optimization, and plasticity loss in dense neural operators. In particular, broadband open-channel fluid dynamics and boundary-dominated porous media flows impose incompatible spectral and geometric demands on a single dense parameter path. We introduce Shodh-MoE, a sparse-activated latent transformer architecture for multi-physics transport. Shodh-MoE operates on compressed 16^3 physical latents produced by a physics-informed autoencoder with an intra-tokenizer Helmholtz-style velocity parameterization, restricting decoded states to divergence-free velocity manifolds. The model guarantees exact mass conservation, achieving a physically verifiable velocity divergence of ~2.8 x 10^-10 (evaluated post-hoc in FP64) on 128^3 grids. A Top-1 soft-semantic router dynamically assigns localized latent patches to expert subnetworks, enabling specialized parameter paths for distinct physical mechanisms while preserving shared experts for universal symmetries. In a 20,000-step distributed pretraining run over mixed three-dimensional physical tensors, routing telemetry shows autonomous domain bifurcation: held-out validation tokens from the open-channel domain route exclusively to Expert 0, while porous-media tokens route exclusively to Expert 1. The model converges simultaneously across both regimes, achieving latent validation MSEs of 2.46 x 10^-5 and 9.76 x 10^-6, and decoded physical MSEs of 2.48 x 10^-6 and 1.76 x 10^-6. These results support sparse expert routing as a practical architectural mechanism for mitigating multi-physics interference in universal neural operators.

Related papers

• Learning Metamaterial Eigenmodes with Wavelet-Encoded Fourier Neural Operators [57.64362886310329]
  We show that a machine learning surrogate based on neural operators can learn and predict multiple eigenmodes of the elastic wave equation.<n>For metamaterial design, the resulting surrogate accelerates the simulation stage of the design cycle by three orders of magnitude relative to finite element analysis on a consumer-grade CPU.<n>These results also carry broader implications for designing input encodings in other multi-mode PDE solvers based on spectral neural operators.
  arXiv  Detail & Related papers  (2026-09-08T01:21:46Z)
• Physics-Informed Neural Network Surrogate for Oxygen Vacancy Dynamics in epitaxial $\mathrm{SrTiO_3}$ on Si memristors via Dynamic Spectral Optimization [0.0]
  Physics-informed neural networks (PINNs) offer a promising framework for semiconductor device modeling.<n>Here, we demonstrate a cascaded PINN architecture coupled with a custom second-order spatial parameter solvers.<n>By isolating trained sub-neural-networks, our model circumvents condition exceeding $1016$m Si substrate.
  arXiv  Detail & Related papers  (2026-09-02T07:42:51Z)
• Manifold-Orthogonal Dual-spectrum Extrapolation for Parameterized Physics-Informed Neural Networks [5.601252185916962]
  Physics-informed neural networks (PINNs) have achieved notable success in modeling dynamical systems governed by partial differential equations (PDEs)<n>We propose Manifold-Orthogonal Dual-spectrum Extrapolation (MODE), a lightweight micro-architecture designed for physics operator adaptation.
  arXiv  Detail & Related papers  (2026-03-14T04:52:50Z)
• Agentic Exploration of PDE Spaces using Latent Foundation Models for Parameterized Simulations [5.931951549967812]
  Flow and more broadly physical phenomena governed by partial differential equations (PDEs) are inherently continuous, high-dimensional and often chaotic in nature.<n>Traditionally, researchers have explored these rich PDE solution spaces using laboratory experiments and/or computationally expensive numerical simulations.<n>We address this by coupling multi-agent LLMs with latent foundation models (LFMs), a generative model over parametrised simulations, that learns explicit, compact and disentangled latent representations of flow fields.
  arXiv  Detail & Related papers  (2026-02-26T20:39:45Z)
• Structure-Informed Estimation for Pilot-Limited MIMO Channels via Tensor Decomposition [51.56484100374058]
  This paper formulates pilot-limited channel estimation as low-rank tensor completion from sparse observations.<n>Experiments on synthetic channels demonstrate 10-20,dB normalized mean-square error (NMSE) improvement over least-squares (LS)<n> evaluations on DeepMIMO ray-tracing channels show 24-44% additional NMSE reduction over pure tensor-based methods.
  arXiv  Detail & Related papers  (2026-02-03T23:38:05Z)
• PhyG-MoE: A Physics-Guided Mixture-of-Experts Framework for Energy-Efficient GNSS Interference Recognition [49.955269674859004]
  This paper introduces PhyG-MoE (Physics-Guided Mixture-of-Experts), a framework designed to align model capacity with signal complexity.<n>Unlike static architectures, the proposed system employs a spectrum-based gating mechanism that routes signals based on their spectral feature entanglement.<n>A high-capacity TransNeXt expert is activated on-demand to disentangle complex features in saturated scenarios, while lightweight experts handle fundamental signals to minimize latency.
  arXiv  Detail & Related papers  (2026-01-19T07:57:52Z)
• SKANet: A Cognitive Dual-Stream Framework with Adaptive Modality Fusion for Robust Compound GNSS Interference Classification [47.20483076887704]
  Global Navigation Satellite Systems (GNSS) face growing threats from sophisticated jamming interference.<n>We propose a cognitive deep learning framework built upon a dual-stream architecture that integrates Time-Frequency Images (TFIs) and Power Spectral Density (PSD)<n>We show that SKANet achieves an overall accuracy of 96.99%, exhibiting superior robustness for compound jamming classification.
  arXiv  Detail & Related papers  (2026-01-19T07:42:45Z)
• 1d-qt-ideal-solver: 1D Idealized Quantum Tunneling Solver with Absorbing Boundaries [0.0]
  1d-qt-ideal-solver is an open-source Python library for simulating quantum tunneling dynamics.<n>Numba just-in-time compilation achieves performance comparable to compiled languages.
  arXiv  Detail & Related papers  (2025-12-27T16:13:44Z)
• Rapid ground state energy estimation with a Sparse Pauli Dynamics-enabled Variational Double Bracket Flow [0.0]
  Ground state energy estimation for strongly correlated quantum systems remains a central challenge in computational physics.<n>We present a variational double bracket flow algorithm that leverages Sparse Pauli Dynamics to efficiently approximate ground state energies.<n>For a 10x10 Heisenberg lattice (100 qubits), vDBF obtains accurate results in approximately 10 minutes on a single CPU thread, compared to over 50 hours on 64 threads for DMRG.
  arXiv  Detail & Related papers  (2025-11-26T18:22:13Z)
• PHASE-Net: Physics-Grounded Harmonic Attention System for Efficient Remote Photoplethysmography Measurement [63.007237197267834]
  Existing deep learning methods are mostly physiological monitoring and lack theoretical robustness.<n>We propose a physics-informed r paradigm derived from the Navier-Stokes equations of hemodynamics, showing that the pulse signal follows a second-order system.<n>This provides a theoretical justification for using a Temporal Conal Network (TCN)<n>Phase-Net achieves state-of-the-art performance with strong efficiency, offering a theoretically grounded and deployment-ready r solution.
  arXiv  Detail & Related papers  (2025-09-29T14:36:45Z)
• Artificial Intelligence for Direct Prediction of Molecular Dynamics Across Chemical Space [0.0]
  Molecular dynamics (MD) is a powerful tool for exploring the behavior of atomistic systems, but its reliance on sequential numerical integration limits simulation efficiency.<n>We present a novel neural network architecture, MDtrajNet, and a pre-trained foundational model, MDtrajNet-1.<n>MDtrajNet directly generates MD trajectories across chemical space, bypassing force calculations and integration.
  arXiv  Detail & Related papers  (2025-05-22T06:56:19Z)
• RadioDiff-$k^2$: Helmholtz Equation Informed Generative Diffusion Model for Multi-Path Aware Radio Map Construction [76.24833675757033]
  We propose a physics-informed generative learning approach, named RadioDiff-$k2$, for accurate and efficient multipath-aware radio map (RM) construction.<n>We show that the proposed RadioDiff-$k2$ framework achieves state-of-the-art (SOTA) performance in both image-level RM construction and localization tasks.
  arXiv  Detail & Related papers  (2025-04-22T06:28:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.