M\textsuperscript{4}Fuse: Lightweight State-Space MoE with a Cross-Scale Gating Bridge for Brain Tumor Segmentation

• URL: http://arxiv.org/abs/2605.02444v1
• Date: Mon, 04 May 2026 10:45:03 GMT
• Title: M\textsuperscript{4}Fuse: Lightweight State-Space MoE with a Cross-Scale Gating Bridge for Brain Tumor Segmentation
• Abstract summary: Mtextsuperscript4Fuse is a lightweight network that prioritizes discriminative brain tumor cues over exhaustive appearance reconstruction.<n>On the BraTS 2019 and BraTS 2021 benchmarks, Mtextsuperscript4Fuse outperforms other lightweight excellent methods in both parameter count and performance.
• Score: 7.426738885878874
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Encoder-decoder imbalance and the reliance on large input volumes make many 3D brain tumor segmentation models both compute-heavy and brittle. We present M\textsuperscript{4}Fuse, a lightweight network that prioritizes discriminative brain tumor cues over exhaustive appearance reconstruction. Our method balances encoder and decoder capacity and replaces depth expansion with a synergistic design: it propagates long-range context with linear complexity via a grouped state space mixer, denoises and aligns skip features using a cross-scale dual-stage gating bridge, and absorbs cross-site acquisition shifts with a sample-level mixture-of-experts. On the BraTS2019 and BraTS2021 benchmarks, M\textsuperscript{4}Fuse outperforms other lightweight excellent methods in both parameter count and performance. Even at a challenging input resolution of \(64\times128\times128\) (half that of existing excellent models), M\textsuperscript{4}Fuse reduces parameters by 62.63\% and improves average performance by 0.09\%. Ablations of key components validate the method's exceptional parameter-to-accuracy efficiency and robustness across diverse data centers.

Related papers

• TRIUNE-Net: Harmonizing Scale, Shape, and Efficiency in Pancreatic Tumor Segmentation [6.974505071267636]
  Pancreatic tumor segmentation in 3D CT volumes is challenged by extreme variability across both the pancreas and tumor.<n>We propose TRIUNENet, a lightweight unified lightweight in resource scale, shape, efficiency through three innovations.<n> TRIUNENet state-of-heart results with only 5.86 M parameters and no external pre-training, outperforming all baselines across all key tumor metrics.
  arXiv  Detail & Related papers  (2026-09-08T06:24:24Z)
• Multi-Stage Mamba-Based Architecture for Fast and Scalable Superconducting Qubit Readout [2.7180447022087932]
  In superconducting quantum processors, readout operations are both error-prone and high-latency.<n>We present a multi-stage qubit state discriminator based on the Mamba model.<n>Our model achieves a geometric mean readout fidelity of 0.906, outperforming the best-reported state-of-the-art method.
  arXiv  Detail & Related papers  (2026-07-13T15:12:07Z)
• Closed-Form Spectral Regularization for Multi-Task Model Merging [96.82449201305234]
  Model merging combines several independently fine-tuned experts into a single multi-task model without any training data.<n>State-of-the-art merging methods formulate merging as a layer-wise interference problem.<n>We propose SWUDI, a closed-form method that combines a soft exponential filter, which matches the gradient-flow trajectory of iterative descent.
  arXiv  Detail & Related papers  (2026-06-05T14:00:47Z)
• Normalized Architectures are Natively 4-Bit [49.13186675123547]
  nGPT, an architecture that constrains weights and hidden representations to the unit hypersphere, is inherently more robust to low-precision arithmetic.<n>We validate this approach on both a 1.2B dense model and hybrid (Mamba-Transformer) MoE models of up to 3B/30B parameters.
  arXiv  Detail & Related papers  (2026-05-07T11:54:07Z)
• Rethinking Attention Output Projection: Structured Hadamard Transforms for Efficient Transformers [0.0]
  We propose replacing the dense output projection in multi-head attention with a fixed, parameter-free Walsh Hadamard Transform.<n>Across different model sizes, we demonstrate that this structured substitution maintains comparable or slightly superior downstream task performance.<n>We observe that structured Hadamard-based models exhibit a steeper validation loss curve relative to training FLOPs, suggesting more favorable compute utilization during training.
  arXiv  Detail & Related papers  (2026-03-09T13:05:08Z)
• JTok: On Token Embedding as another Axis of Scaling Law via Joint Token Self-modulation [46.64215658042213]
  We introduce Joint-Token (JTok) and Mixture of Joint-Token (JTok-M), which augment Transformer layers with modulation vectors retrieved from auxiliary embedding tables.<n>These vectors modulate the backbone via lightweight, element-wise operations, incurring negligible FLOPs overhead.<n>Our approach consistently reduces validation loss and significantly improves downstream task performance.
  arXiv  Detail & Related papers  (2026-01-31T16:15:18Z)
• DRBD-Mamba for Robust and Efficient Brain Tumor Segmentation with Analytical Insights [54.87947751720332]
  Accurate brain tumor segmentation is significant for clinical diagnosis and treatment.<n>Mamba-based State Space Models have demonstrated promising performance.<n>We propose a dual-resolution bi-directional Mamba that captures multi-scale long-range dependencies with minimal computational overhead.
  arXiv  Detail & Related papers  (2025-10-16T07:31:21Z)
• SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models [63.118592279833656]
  Post-training quantization (PTQ) is an effective technique for compressing large language models (LLMs)<n>We propose SliM-LLM, a salience-driven mixed-precision quantization framework that allocates bit-widths at the group-wise.<n> Experiments show that SliM-LLM achieves superior performance across various LLMs at low bit-widths.
  arXiv  Detail & Related papers  (2024-05-23T16:21:48Z)
• AsymFormer: Asymmetrical Cross-Modal Representation Learning for Mobile Platform Real-Time RGB-D Semantic Segmentation [4.54656749337919]
  AsymFormer is a novel network that improves real-time semantic segmentation accuracy using RGB-D multi-modal information. AsymFormer uses an asymmetrical backbone for multimodal feature extraction, reducing redundant parameters. AsymFormer can strike a balance between high accuracy and efficiency.
  arXiv  Detail & Related papers  (2023-09-25T11:57:16Z)
• UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation [93.88170217725805]
  We propose a 3D medical image segmentation approach, named UNETR++, that offers both high-quality segmentation masks as well as efficiency in terms of parameters, compute cost, and inference speed. The core of our design is the introduction of a novel efficient paired attention (EPA) block that efficiently learns spatial and channel-wise discriminative features. Our evaluations on five benchmarks, Synapse, BTCV, ACDC, BRaTs, and Decathlon-Lung, reveal the effectiveness of our contributions in terms of both efficiency and accuracy.
  arXiv  Detail & Related papers  (2022-12-08T18:59:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.