BoxCell: Leveraging SAM for Cell Segmentation with Box Supervision

• URL: http://arxiv.org/abs/2311.17960v2
• Date: Tue, 04 Nov 2025 19:25:20 GMT
• Title: BoxCell: Leveraging SAM for Cell Segmentation with Box Supervision
• Authors: Aayush Kumar Tyagi, Vaibhav Mishra, Prathosh A. P., Mausam,
• Abstract summary: We study a weakly supervised setting, where only bounding box supervision is available, and present the use of Segment Anything (SAM) for this without any finetuning.<n>We propose BoxCell, a cell segmentation framework that utilizes SAM's capability to interpret bounding boxes as prompts, emphboth at train and test times.<n>We experiment on three publicly available cell segmentation datasets namely, CoNSep, MoNuSeg, and TNBC, and find that BoxCell significantly outperforms existing box supervised image segmentation models.
• Score: 22.761067372749398
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cell segmentation in histopathological images is vital for diagnosis, and treatment of several diseases. Annotating data is tedious, and requires medical expertise, making it difficult to employ supervised learning. Instead, we study a weakly supervised setting, where only bounding box supervision is available, and present the use of Segment Anything (SAM) for this without any finetuning, i.e., directly utilizing the pre-trained model. We propose BoxCell, a cell segmentation framework that utilizes SAM's capability to interpret bounding boxes as prompts, \emph{both} at train and test times. At train time, gold bounding boxes given to SAM produce (pseudo-)masks, which are used to train a standalone segmenter. At test time, BoxCell generates two segmentation masks: (1) generated by this standalone segmenter, and (2) a trained object detector outputs bounding boxes, which are given as prompts to SAM to produce another mask. Recognizing complementary strengths, we reconcile the two segmentation masks using a novel integer programming formulation with intensity and spatial constraints. We experiment on three publicly available cell segmentation datasets namely, CoNSep, MoNuSeg, and TNBC, and find that BoxCell significantly outperforms existing box supervised image segmentation models, obtaining 6-10 point Dice gains.

Related papers

• Evaluating SAM2 for Video Semantic Segmentation [60.157605818225186]
  The Anything Model 2 (SAM2) has proven to be a powerful foundation model for promptable visual object segmentation in both images and videos.<n>This paper explores the extension of SAM2 to dense Video Semantic (VSS)<n>Our experiments suggest that leveraging SAM2 enhances overall performance in VSS, primarily due to its precise predictions of object boundaries.
  arXiv  Detail & Related papers  (2025-12-01T15:15:16Z)
• DC-SAM: In-Context Segment Anything in Images and Videos via Dual Consistency [91.30252180093333]
  We propose the Dual Consistency SAM (DCSAM) method based on prompttuning to adapt SAM and SAM2 for in-context segmentation. Our key insights are to enhance the features of the SAM's prompt encoder in segmentation by providing high-quality visual prompts. Although the proposed DC-SAM is primarily designed for images, it can be seamlessly extended to the video domain with the support SAM2.
  arXiv  Detail & Related papers  (2025-04-16T13:41:59Z)
• BiSeg-SAM: Weakly-Supervised Post-Processing Framework for Boosting Binary Segmentation in Segment Anything Models [6.74659948545092]
  BiSeg-SAM is a weakly supervised prompting and boundary refinement network for the segmentation of polyps and skin lesions.<n>Our method demonstrates significant superiority over state-of-the-art (SOTA) methods when tested on five polyp datasets and one skin cancer dataset.
  arXiv  Detail & Related papers  (2025-04-02T08:04:37Z)
• Learnable Prompting SAM-induced Knowledge Distillation for Semi-supervised Medical Image Segmentation [47.789013598970925]
  We propose a learnable prompting SAM-induced Knowledge distillation framework (KnowSAM) for semi-supervised medical image segmentation. Our model outperforms the state-of-the-art semi-supervised segmentation approaches.
  arXiv  Detail & Related papers  (2024-12-18T11:19:23Z)
• SAM-MPA: Applying SAM to Few-shot Medical Image Segmentation using Mask Propagation and Auto-prompting [6.739803086387235]
  Medical image segmentation often faces the challenge of prohibitively expensive annotation costs. We propose leveraging the Segment Anything Model (SAM), pre-trained on over 1 billion masks. We develop SAM-MPA, an innovative SAM-based framework for few-shot medical image segmentation.
  arXiv  Detail & Related papers  (2024-11-26T12:12:12Z)
• DB-SAM: Delving into High Quality Universal Medical Image Segmentation [100.63434169944853]
  We propose a dual-branch adapted SAM framework, named DB-SAM, to bridge the gap between natural and 2D/3D medical data. Our proposed DB-SAM achieves an absolute gain of 8.8%, compared to a recent medical SAM adapter in the literature.
  arXiv  Detail & Related papers  (2024-10-05T14:36:43Z)
• Automating MedSAM by Learning Prompts with Weak Few-Shot Supervision [10.609371657347806]
  This work proposes to replace conditioning on input prompts with a lightweight module that directly learns a prompt embedding from the image embedding. Our approach is validated on MedSAM, a version of SAM fine-tuned for medical images.
  arXiv  Detail & Related papers  (2024-09-30T13:53:01Z)
• SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and Medical Image Segmentation [51.90445260276897]
  We prove that the Segment Anything Model 2 (SAM2) can be a strong encoder for U-shaped segmentation models. We propose a simple but effective framework, termed SAM2-UNet, for versatile image segmentation.
  arXiv  Detail & Related papers  (2024-08-16T17:55:38Z)
• SAM-CP: Marrying SAM with Composable Prompts for Versatile Segmentation [88.80792308991867]
  Segment Anything model (SAM) has shown ability to group image pixels into patches, but applying it to semantic-aware segmentation still faces major challenges. This paper presents SAM-CP, a simple approach that establishes two types of composable prompts beyond SAM and composes them for versatile segmentation. Experiments show that SAM-CP achieves semantic, instance, and panoptic segmentation in both open and closed domains.
  arXiv  Detail & Related papers  (2024-07-23T17:47:25Z)
• CycleSAM: One-Shot Surgical Scene Segmentation using Cycle-Consistent Feature Matching to Prompt SAM [2.9500242602590565]
  CycleSAM is an approach for one-shot surgical scene segmentation using the training image-mask pair at test-time. We employ a ResNet50 encoder pretrained on surgical images in a self-supervised fashion, thereby maintaining high label-efficiency.
  arXiv  Detail & Related papers  (2024-07-09T12:08:07Z)
• Segment Anything without Supervision [65.93211374889196]
  We present Unsupervised SAM (UnSAM) for promptable and automatic whole-image segmentation. UnSAM utilizes a divide-and-conquer strategy to "discover" the hierarchical structure of visual scenes. We show that supervised SAM can also benefit from our self-supervised labels.
  arXiv  Detail & Related papers  (2024-06-28T17:47:32Z)
• Moving Object Segmentation: All You Need Is SAM (and Flow) [82.78026782967959]
  We investigate two models for combining SAM with optical flow that harness the segmentation power of SAM with the ability of flow to discover and group moving objects. In the first model, we adapt SAM to take optical flow, rather than RGB, as an input. In the second, SAM takes RGB as an input, and flow is used as a segmentation prompt. These surprisingly simple methods, without any further modifications, outperform all previous approaches by a considerable margin in both single and multi-object benchmarks.
  arXiv  Detail & Related papers  (2024-04-18T17:59:53Z)
• Test-Time Adaptation with SaLIP: A Cascade of SAM and CLIP for Zero shot Medical Image Segmentation [10.444726122035133]
  We propose a simple unified framework, SaLIP, for organ segmentation. SAM is used for part based segmentation within the image, followed by CLIP to retrieve the mask corresponding to the region of interest. Finally, SAM is prompted by the retrieved ROI to segment a specific organ.
  arXiv  Detail & Related papers  (2024-04-09T14:56:34Z)
• WSI-SAM: Multi-resolution Segment Anything Model (SAM) for histopathology whole-slide images [8.179859593451285]
  We present WSI-SAM, enhancing Segment Anything Model (SAM) with precise object segmentation capabilities for histopathology images. To fully exploit pretrained knowledge while minimizing training overhead, we keep SAM frozen, introducing only minimal extra parameters. Our model outperforms SAM by 4.1 and 2.5 percent points on a ductal carcinoma in situ (DCIS) segmentation tasks and breast cancer metastasis segmentation task.
  arXiv  Detail & Related papers  (2024-03-14T10:30:43Z)
• BLO-SAM: Bi-level Optimization Based Overfitting-Preventing Finetuning of SAM [37.1263294647351]
  We introduce BLO-SAM, which finetunes the Segment Anything Model (SAM) based on bi-level optimization (BLO) BLO-SAM reduces the risk of overfitting by training the model's weight parameters and the prompt embedding on two separate subsets of the training dataset. Results demonstrate BLO-SAM's superior performance over various state-of-the-art image semantic segmentation methods.
  arXiv  Detail & Related papers  (2024-02-26T06:36:32Z)
• How to Efficiently Adapt Large Segmentation Model(SAM) to Medical Images [15.181219203629643]
  Segment Anything (SAM) exhibits impressive capabilities in zero-shot segmentation for natural images. However, when applied to medical images, SAM suffers from noticeable performance drop. In this work, we propose to freeze SAM encoder and finetune a lightweight task-specific prediction head.
  arXiv  Detail & Related papers  (2023-06-23T18:34:30Z)
• Personalize Segment Anything Model with One Shot [52.54453744941516]
  We propose a training-free Personalization approach for Segment Anything Model (SAM) Given only a single image with a reference mask, PerSAM first localizes the target concept by a location prior. PerSAM segments it within other images or videos via three techniques: target-guided attention, target-semantic prompting, and cascaded post-refinement.
  arXiv  Detail & Related papers  (2023-05-04T17:59:36Z)
• Robust Visual Tracking by Segmentation [103.87369380021441]
  Estimating the target extent poses a fundamental challenge in visual object tracking. We propose a segmentation-centric tracking pipeline that produces a highly accurate segmentation mask. Our tracker is able to better learn a target representation that clearly differentiates the target in the scene from background content.
  arXiv  Detail & Related papers  (2022-03-21T17:59:19Z)
• Box-Adapt: Domain-Adaptive Medical Image Segmentation using Bounding BoxSupervision [52.45336255472669]
  We propose a weakly supervised do-main adaptation setting for deep learning. Box-Adapt fully explores the fine-grained segmenta-tion mask in the source domain and the weak bounding box in the target domain. We demonstrate the effectiveness of our method in the liver segmentation task.
  arXiv  Detail & Related papers  (2021-08-19T01:51:04Z)
• Learning Class-Agnostic Pseudo Mask Generation for Box-Supervised Semantic Segmentation [156.9155100983315]
  We seek for a more accurate learning-based class-agnostic pseudo mask generator tailored to box-supervised semantic segmentation. Our method can further close the performance gap between box-supervised and fully-supervised models.
  arXiv  Detail & Related papers  (2021-03-09T14:54:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.