LoMix: Learnable Weighted Multi-Scale Logits Mixing for Medical Image Segmentation

• URL: http://arxiv.org/abs/2510.22995v1
• Date: Mon, 27 Oct 2025 04:25:57 GMT
• Title: LoMix: Learnable Weighted Multi-Scale Logits Mixing for Medical Image Segmentation
• Abstract summary: LoMix is a differentiable plug-and-play module that generates new mixed-scale outputs.<n>It learns how exactly each of them should guide the training process.<n>Every original or mutant map is given a softplus loss weight that is co-optimized with network parameters.
• Score: 27.68818512667956
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: U-shaped networks output logits at multiple spatial scales, each capturing a different blend of coarse context and fine detail. Yet, training still treats these logits in isolation - either supervising only the final, highest-resolution logits or applying deep supervision with identical loss weights at every scale - without exploring mixed-scale combinations. Consequently, the decoder output misses the complementary cues that arise only when coarse and fine predictions are fused. To address this issue, we introduce LoMix (Logits Mixing), a NAS-inspired, differentiable plug-and-play module that generates new mixed-scale outputs and learns how exactly each of them should guide the training process. More precisely, LoMix mixes the multi-scale decoder logits with four lightweight fusion operators: addition, multiplication, concatenation, and attention-based weighted fusion, yielding a rich set of synthetic mutant maps. Every original or mutant map is given a softplus loss weight that is co-optimized with network parameters, mimicking a one-step architecture search that automatically discovers the most useful scales, mixtures, and operators. Plugging LoMix into recent U-shaped architectures (i.e., PVT-V2-B2 backbone with EMCAD decoder) on Synapse 8-organ dataset improves DICE by +4.2% over single-output supervision, +2.2% over deep supervision, and +1.5% over equally weighted additive fusion, all with zero inference overhead. When training data are scarce (e.g., one or two labeled scans), the advantage grows to +9.23%, underscoring LoMix's data efficiency. Across four benchmarks and diverse U-shaped networks, LoMiX improves DICE by up to +13.5% over single-output supervision, confirming that learnable weighted mixed-scale fusion generalizes broadly while remaining data efficient, fully interpretable, and overhead-free at inference. Our code is available at https://github.com/SLDGroup/LoMix.

Related papers

• Mixture-of-Mamba: Enhancing Multi-Modal State-Space Models with Modality-Aware Sparsity [56.0251572416922]
  State Space Models (SSMs) have emerged as efficient alternatives to Transformers for sequential modeling.<n>We propose a novel SSM architecture that introduces modality-aware sparsity through modality-specific parameterization of the Mamba block.<n>We evaluate Mixture-of-Mamba across three multi-modal pretraining settings.
  arXiv  Detail & Related papers  (2025-01-27T18:35:05Z)
• MM-Mixing: Multi-Modal Mixing Alignment for 3D Understanding [64.65145700121442]
  We introduce MM-Mixing, a multi-modal mixing alignment framework for 3D understanding. Our proposed two-stage training pipeline combines feature-level and input-level mixing to optimize the 3D encoder. We demonstrate that MM-Mixing significantly improves baseline performance across various learning scenarios.
  arXiv  Detail & Related papers  (2024-05-28T18:44:15Z)
• PowMix: A Versatile Regularizer for Multimodal Sentiment Analysis [71.8946280170493]
  This paper introduces PowMix, a versatile embedding space regularizer that builds upon the strengths of unimodal mixing-based regularization approaches. PowMix is integrated before the fusion stage of multimodal architectures and facilitates intra-modal mixing, such as mixing text with text, to act as a regularizer.
  arXiv  Detail & Related papers  (2023-12-19T17:01:58Z)
• TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognition [63.93802691275012]
  We propose a lightweight Dual Dynamic Token Mixer (D-Mixer) to simultaneously learn global and local dynamics.<n>We use D-Mixer as the basic building block to design TransXNet, a novel hybrid CNN-Transformer vision backbone network.<n>In the ImageNet-1K classification, TransXNet-T surpasses Swin-T by 0.3% in top-1 accuracy while requiring less than half of the computational cost.
  arXiv  Detail & Related papers  (2023-10-30T09:35:56Z)
• LUMix: Improving Mixup by Better Modelling Label Uncertainty [33.56660038646426]
  Deep networks can be better generalized when trained with noisy samples and regularization techniques. Previous Mixup-based methods linearly combine images and labels to generate additional training data. We propose LUMix, which models such uncertainty by adding label perturbation during training.
  arXiv  Detail & Related papers  (2022-11-29T00:47:55Z)
• MIXER: Multiattribute, Multiway Fusion of Uncertain Pairwise Affinities [45.00178262932004]
  We present a multiway fusion algorithm capable of directly processing uncertain pairwise affinities. In contrast to existing works that require initial pairwise associations, our MIXER algorithm improves accuracy by leveraging the additional information provided by pairwise affinities.
  arXiv  Detail & Related papers  (2022-10-15T19:19:57Z)
• CoSMix: Compositional Semantic Mix for Domain Adaptation in 3D LiDAR Segmentation [62.259239847977014]
  We propose a new approach of sample mixing for point cloud UDA, namely Compositional Semantic Mix (CoSMix) CoSMix consists of a two-branch symmetric network that can process labelled synthetic data (source) and real-world unlabelled point clouds (target) concurrently. We evaluate CoSMix on two large-scale datasets, showing that it outperforms state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2022-07-20T09:33:42Z)
• MixMo: Mixing Multiple Inputs for Multiple Outputs via Deep Subnetworks [97.08677678499075]
  We introduce MixMo, a new framework for learning multi-input multi-output deepworks. We show that binary mixing in features - particularly with patches from CutMix - enhances results by makingworks stronger and more diverse. In addition to being easy to implement and adding no cost at inference, our models outperform much costlier data augmented deep ensembles.
  arXiv  Detail & Related papers  (2021-03-10T15:31:02Z)
• StackMix: A complementary Mix algorithm [26.99020412895196]
  StackMix: Each input is presented as a concatenation of two images, and the label is the mean of the two one-hot labels. We show significant gains across a variety of benchmarks are achieved by combining StackMix with existing Mix augmentation.
  arXiv  Detail & Related papers  (2020-11-25T10:15:24Z)
• FMix: Enhancing Mixed Sample Data Augmentation [5.820517596386667]
  Mixed Sample Data Augmentation (MSDA) has received increasing attention in recent years. We show that MixUp distorts learned functions in a way that CutMix does not. We propose FMix, an MSDA that uses random binary masks obtained by applying a threshold to low frequency images.
  arXiv  Detail & Related papers  (2020-02-27T11:46:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.