TK-Mamba: Marrying KAN with Mamba for Text-Driven 3D Medical Image Segmentation

• URL: http://arxiv.org/abs/2505.18525v1
• Date: Sat, 24 May 2025 05:41:55 GMT
• Title: TK-Mamba: Marrying KAN with Mamba for Text-Driven 3D Medical Image Segmentation
• Authors: Haoyu Yang, Yuxiang Cai, Jintao Chen, Xuhong Zhang, Wenhui Lei, Xiaoming Shi, Jianwei Yin, Yankai Jiang,
• Abstract summary: 3D medical image segmentation is vital for clinical diagnosis and treatment.<n>Traditional single-modality networks, such as CNNs and Transformers, are often limited by computational inefficiency and constrained contextual modeling.<n>We introduce a novel multimodal framework that leverages Mamba and Kolmogorov-Arnold Networks (KAN) as an efficient backbone for long-sequence modeling.
• Score: 22.62310549476759
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D medical image segmentation is vital for clinical diagnosis and treatment but is challenged by high-dimensional data and complex spatial dependencies. Traditional single-modality networks, such as CNNs and Transformers, are often limited by computational inefficiency and constrained contextual modeling in 3D settings. We introduce a novel multimodal framework that leverages Mamba and Kolmogorov-Arnold Networks (KAN) as an efficient backbone for long-sequence modeling. Our approach features three key innovations: First, an EGSC (Enhanced Gated Spatial Convolution) module captures spatial information when unfolding 3D images into 1D sequences. Second, we extend Group-Rational KAN (GR-KAN), a Kolmogorov-Arnold Networks variant with rational basis functions, into 3D-Group-Rational KAN (3D-GR-KAN) for 3D medical imaging - its first application in this domain - enabling superior feature representation tailored to volumetric data. Third, a dual-branch text-driven strategy leverages CLIP's text embeddings: one branch swaps one-hot labels for semantic vectors to preserve inter-organ semantic relationships, while the other aligns images with detailed organ descriptions to enhance semantic alignment. Experiments on the Medical Segmentation Decathlon (MSD) and KiTS23 datasets show our method achieving state-of-the-art performance, surpassing existing approaches in accuracy and efficiency. This work highlights the power of combining advanced sequence modeling, extended network architectures, and vision-language synergy to push forward 3D medical image segmentation, delivering a scalable solution for clinical use. The source code is openly available at https://github.com/yhy-whu/TK-Mamba.

Related papers

• Mamba Based Feature Extraction And Adaptive Multilevel Feature Fusion For 3D Tumor Segmentation From Multi-modal Medical Image [8.999013226631893]
  Multi-modal 3D medical image segmentation aims to accurately identify tumor regions across different modalities.<n>Traditional convolutional neural network (CNN)-based methods struggle with capturing global features.<n>Transformers-based methods, despite effectively capturing global context, encounter high computational costs in 3D medical image segmentation.
  arXiv  Detail & Related papers  (2025-04-30T03:29:55Z)
• A Novel Convolutional-Free Method for 3D Medical Imaging Segmentation [0.0]
  Convolutional neural networks (CNNs) have dominated the field, achieving significant success in 3D medical image segmentation.<n>Recent transformer-based models, such as TransUNet and nnFormer, have demonstrated promise in addressing these limitations.<n>This paper introduces a novel, fully convolutional-free model based on transformer architecture and self-attention mechanisms.
  arXiv  Detail & Related papers  (2025-02-08T00:52:45Z)
• Generative Enhancement for 3D Medical Images [74.17066529847546]
  We propose GEM-3D, a novel generative approach to the synthesis of 3D medical images. Our method begins with a 2D slice, noted as the informed slice to serve the patient prior, and propagates the generation process using a 3D segmentation mask. By decomposing the 3D medical images into masks and patient prior information, GEM-3D offers a flexible yet effective solution for generating versatile 3D images.
  arXiv  Detail & Related papers  (2024-03-19T15:57:04Z)
• Promise:Prompt-driven 3D Medical Image Segmentation Using Pretrained Image Foundation Models [13.08275555017179]
  We propose ProMISe, a prompt-driven 3D medical image segmentation model using only a single point prompt. We evaluate our model on two public datasets for colon and pancreas tumor segmentations.
  arXiv  Detail & Related papers  (2023-10-30T16:49:03Z)
• Spatiotemporal Modeling Encounters 3D Medical Image Analysis: Slice-Shift UNet with Multi-View Fusion [0.0]
  We propose a new 2D-based model dubbed Slice SHift UNet which encodes three-dimensional features at 2D CNN's complexity. More precisely multi-view features are collaboratively learned by performing 2D convolutions along the three planes of a volume. The effectiveness of our approach is validated in Multi-Modality Abdominal Multi-Organ axis (AMOS) and Multi-Atlas Labeling Beyond the Cranial Vault (BTCV) datasets.
  arXiv  Detail & Related papers  (2023-07-24T14:53:23Z)
• SeMLaPS: Real-time Semantic Mapping with Latent Prior Networks and Quasi-Planar Segmentation [53.83313235792596]
  We present a new methodology for real-time semantic mapping from RGB-D sequences. It combines a 2D neural network and a 3D network based on a SLAM system with 3D occupancy mapping. Our system achieves state-of-the-art semantic mapping quality within 2D-3D networks-based systems.
  arXiv  Detail & Related papers  (2023-06-28T22:36:44Z)
• Dynamic Linear Transformer for 3D Biomedical Image Segmentation [2.440109381823186]
  Transformer-based neural networks have surpassed promising performance on many biomedical image segmentation tasks. Main challenge for 3D transformer-based segmentation methods is the quadratic complexity introduced by the self-attention mechanism. We propose a novel transformer architecture for 3D medical image segmentation using an encoder-decoder style architecture with linear complexity.
  arXiv  Detail & Related papers  (2022-06-01T21:15:01Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation [95.51455777713092]
  Convolutional neural networks (CNNs) have been the de facto standard for nowadays 3D medical image segmentation. We propose a novel framework that efficiently bridges a bf Convolutional neural network and a bf Transformer bf (CoTr) for accurate 3D medical image segmentation.
  arXiv  Detail & Related papers  (2021-03-04T13:34:22Z)
• TSGCNet: Discriminative Geometric Feature Learning with Two-Stream GraphConvolutional Network for 3D Dental Model Segmentation [141.2690520327948]
  We propose a two-stream graph convolutional network (TSGCNet) to learn multi-view information from different geometric attributes. We evaluate our proposed TSGCNet on a real-patient dataset of dental models acquired by 3D intraoral scanners.
  arXiv  Detail & Related papers  (2020-12-26T08:02:56Z)
• Volumetric Medical Image Segmentation: A 3D Deep Coarse-to-fine Framework and Its Adversarial Examples [74.92488215859991]
  We propose a novel 3D-based coarse-to-fine framework to efficiently tackle these challenges. The proposed 3D-based framework outperforms their 2D counterparts by a large margin since it can leverage the rich spatial information along all three axes. We conduct experiments on three datasets, the NIH pancreas dataset, the JHMI pancreas dataset and the JHMI pathological cyst dataset.
  arXiv  Detail & Related papers  (2020-10-29T15:39:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.