BALR-SAM: Boundary-Aware Low-Rank Adaptation of SAM for Resource-Efficient Medical Image Segmentation

• URL: http://arxiv.org/abs/2509.24204v2
• Date: Fri, 31 Oct 2025 07:38:04 GMT
• Title: BALR-SAM: Boundary-Aware Low-Rank Adaptation of SAM for Resource-Efficient Medical Image Segmentation
• Authors: Zelin Liu, Sicheng Dong, Bocheng Li, Yixuan Yang, Jiacheng Ruan, Chenxu Zhou, Suncheng Xiang,
• Abstract summary: Vision foundation models like the Segment Anything Model (SAM) often struggle in medical image segmentation due to a lack of domain-specific adaptation.<n>We propose BALR-SAM, a boundary-aware low-rank adaptation framework that enhances SAM for medical imaging.<n>It combines three tailored components: (1) a Complementary Detail Enhancement Network (CDEN) using depthwise separable convolutions and multi-scale fusion to capture boundary-sensitive features essential for accurate segmentation; (2) low-rank adapters integrated into SAM's Vision Transformer blocks to optimize feature representation and attention for medical contexts, while simultaneously significantly reducing the parameter space; and
• Score: 11.634558989215392
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision foundation models like the Segment Anything Model (SAM), pretrained on large-scale natural image datasets, often struggle in medical image segmentation due to a lack of domain-specific adaptation. In clinical practice, fine-tuning such models efficiently for medical downstream tasks with minimal resource demands, while maintaining strong performance, is challenging. To address these issues, we propose BALR-SAM, a boundary-aware low-rank adaptation framework that enhances SAM for medical imaging. It combines three tailored components: (1) a Complementary Detail Enhancement Network (CDEN) using depthwise separable convolutions and multi-scale fusion to capture boundary-sensitive features essential for accurate segmentation; (2) low-rank adapters integrated into SAM's Vision Transformer blocks to optimize feature representation and attention for medical contexts, while simultaneously significantly reducing the parameter space; and (3) a low-rank tensor attention mechanism in the mask decoder, cutting memory usage by 75% and boosting inference speed. Experiments on standard medical segmentation datasets show that BALR-SAM, without requiring prompts, outperforms several state-of-the-art (SOTA) methods, including fully fine-tuned MedSAM, while updating just 1.8% (11.7M) of its parameters.

Related papers

• VesSAM: Efficient Multi-Prompting for Segmenting Complex Vessel [68.24765319399286]
  We present VesSAM, a powerful and efficient framework tailored for 2D vessel segmentation.<n>VesSAM integrates (1) a convolutional adapter to enhance local texture features, (2) a multi-prompt encoder that fuses anatomical prompts, and (3) a lightweight mask decoder to reduce jagged artifacts.<n>VesSAM consistently outperforms state-of-the-art PEFT-based SAM variants by over 10% Dice and 13% IoU.
  arXiv  Detail & Related papers  (2025-11-02T15:47:05Z)
• MedSeqFT: Sequential Fine-tuning Foundation Models for 3D Medical Image Segmentation [55.37355146924576]
  MedSeqFT is a sequential fine-tuning framework for medical image analysis.<n>It adapts pre-trained models to new tasks while refining their representational capacity.<n>It consistently outperforms state-of-the-art fine-tuning strategies.
  arXiv  Detail & Related papers  (2025-09-07T15:22:53Z)
• SAM-aware Test-time Adaptation for Universal Medical Image Segmentation [27.96281051256966]
  We propose SAM-aware Test-Time Adaptation (SAM-TTA), a pipeline that preserves the generalization of SAM while improving its segmentation performance in medical imaging via a test-time framework.<n>Our framework comprises (1) Self-adaptive Bezier Curve-based Transformation (SBCT), which adaptively converts single-channel medical images into three-channel SAM-compatible inputs while maintaining structural integrity, and (2) Dual-scale Uncertainty-driven Mean Teacher adaptation (DUMT), which employs consistency learning to align SAM's internal representations to medical semantics.
  arXiv  Detail & Related papers  (2025-06-05T16:38:16Z)
• Few-Shot Adaptation of Training-Free Foundation Model for 3D Medical Image Segmentation [8.78725593323412]
  Few-shot Adaptation of Training-frEe SAM (FATE-SAM) is a novel method designed to adapt the advanced Segment Anything Model 2 (SAM2) for 3D medical image segmentation.<n>FATE-SAM reassembles pre-trained modules of SAM2 to enable few-shot adaptation, leveraging a small number of support examples.<n>We evaluate FATE-SAM on multiple medical imaging datasets and compare it with supervised learning methods, zero-shot SAM approaches, and fine-tuned medical SAM methods.
  arXiv  Detail & Related papers  (2025-01-15T20:44:21Z)
• DB-SAM: Delving into High Quality Universal Medical Image Segmentation [100.63434169944853]
  We propose a dual-branch adapted SAM framework, named DB-SAM, to bridge the gap between natural and 2D/3D medical data. Our proposed DB-SAM achieves an absolute gain of 8.8%, compared to a recent medical SAM adapter in the literature.
  arXiv  Detail & Related papers  (2024-10-05T14:36:43Z)
• PAM: A Propagation-Based Model for Segmenting Any 3D Objects across Multi-Modal Medical Images [11.373941923130305]
  PAM (Propagating Anything Model) is a segmentation approach that uses a 2D prompt, like a bounding box or sketch, to create a complete 3D segmentation of medical image volumes. It significantly outperformed existing models like MedSAM and SegVol, with an average improvement of over 18.1% in dice similarity coefficient (DSC) across 44 medical datasets and various object types.
  arXiv  Detail & Related papers  (2024-08-25T13:42:47Z)
• Unleashing the Potential of SAM for Medical Adaptation via Hierarchical Decoding [15.401507589312702]
  This paper introduces H-SAM, a prompt-free adaptation of the Segment Anything Model (SAM) for efficient fine-tuning of medical images. In the initial stage, H-SAM employs SAM's original decoder to generate a prior probabilistic mask, guiding a more intricate decoding process. Our H-SAM demonstrates a 4.78% improvement in average Dice compared to existing prompt-free SAM variants.
  arXiv  Detail & Related papers  (2024-03-27T05:55:16Z)
• SAMIHS: Adaptation of Segment Anything Model for Intracranial Hemorrhage Segmentation [18.867207134086193]
  Intracranial hemorrhage segmentation is a crucial and challenging step in stroke diagnosis and surgical planning. We propose a SAM-based parameter-efficient fine-tuning method, called SAMIHS, for intracranial hemorrhage segmentation. Our experimental results on two public datasets demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2023-11-14T14:23:09Z)
• MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation [58.53672866662472]
  We introduce a modality-agnostic SAM adaptation framework, named as MA-SAM. Our method roots in the parameter-efficient fine-tuning strategy to update only a small portion of weight increments. By injecting a series of 3D adapters into the transformer blocks of the image encoder, our method enables the pre-trained 2D backbone to extract third-dimensional information from input data.
  arXiv  Detail & Related papers  (2023-09-16T02:41:53Z)
• 3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation [52.699139151447945]
  We propose a novel adaptation method for transferring the segment anything model (SAM) from 2D to 3D for promptable medical image segmentation. Our model can outperform domain state-of-the-art medical image segmentation models on 3 out of 4 tasks, specifically by 8.25%, 29.87%, and 10.11% for kidney tumor, pancreas tumor, colon cancer segmentation, and achieve similar performance for liver tumor segmentation.
  arXiv  Detail & Related papers  (2023-06-23T12:09:52Z)
• Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation [51.770805270588625]
  The Segment Anything Model (SAM) has recently gained popularity in the field of image segmentation. Recent studies and individual experiments have shown that SAM underperforms in medical image segmentation. We propose the Medical SAM Adapter (Med-SA), which incorporates domain-specific medical knowledge into the segmentation model.
  arXiv  Detail & Related papers  (2023-04-25T07:34:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.