PicoSAM3: Real-Time In-Sensor Region-of-Interest Segmentation

• URL: http://arxiv.org/abs/2603.11917v1
• Date: Thu, 12 Mar 2026 13:31:43 GMT
• Title: PicoSAM3: Real-Time In-Sensor Region-of-Interest Segmentation
• Abstract summary: We introduce PicoSAM3, a lightweight promptable visual segmentation model optimized for edge and in-sensor execution.<n>PicoSAM3 has 1.3 M parameters and combines a dense CNN architecture with region of interest prompt encoding, Efficient Channel Attention, and knowledge distillation from SAM2 and SAM3.<n>On COCO and LVIS, PicoSAM3 achieves 65.45% and 64.01% mIoU, respectively, outperforming existing SAM-based and edge-oriented baselines.
• Score: 22.190837932060607
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-time, on-device segmentation is critical for latency-sensitive and privacy-aware applications such as smart glasses and Internet-of-Things devices. We introduce PicoSAM3, a lightweight promptable visual segmentation model optimized for edge and in-sensor execution, including deployment on the Sony IMX500 vision sensor. PicoSAM3 has 1.3 M parameters and combines a dense CNN architecture with region of interest prompt encoding, Efficient Channel Attention, and knowledge distillation from SAM2 and SAM3. On COCO and LVIS, PicoSAM3 achieves 65.45% and 64.01% mIoU, respectively, outperforming existing SAM-based and edge-oriented baselines at similar or lower complexity. The INT8 quantized model preserves accuracy with negligible degradation while enabling real-time in-sensor inference at 11.82 ms latency on the IMX500, fully complying with its memory and operator constraints. Ablation studies show that distillation from large SAM models yields up to +14.5% mIoU improvement over supervised training and demonstrate that high-quality, spatially flexible promptable segmentation is feasible directly at the sensor level.

Related papers

• LightAVSeg: Lightweight Audio-Visual Segmentation [67.18006751024687]
  LightAVSeg replaces heavy attention with a decoupled design for semantic filtering and spatial grounding.<n>Experiments demonstrate it achieves a new state-of-the-art among lightweight methods.<n>It reaches 50.4 mIoU on the MS3 benchmark and enables efficient inference on a mobile processor.
  arXiv  Detail & Related papers  (2026-05-09T08:47:54Z)
• Design and Validation of a Lightweight 1D CNN for Affective Touch Classification in Soft Plush Companions [59.49967357689445]
  Soft, sensorized companions offer a physically safe and emotionally intuitive interface for socially assistive technologies.<n>This study presents a complete open-source dilated real-based framework for the development and validation of deep learning models for affective touch recognition in soft interactive companions.
  arXiv  Detail & Related papers  (2026-04-16T10:20:05Z)
• TinyGLASS: Real-Time Self-Supervised In-Sensor Anomaly Detection [7.978059159334374]
  Anomaly detection plays a key role in industrial quality control, where defects must be identified despite the scarcity of labeled faulty samples.<n>Recent self-supervised approaches, such as GLASS, learn normal visual patterns using only defect-free data and have shown strong performance on industrial benchmarks.<n>This work introduces TinyGLASS, a lightweight adaptation of the GLASS framework designed for real-time edge and in-sensor anomaly detection.
  arXiv  Detail & Related papers  (2026-03-17T12:31:34Z)
• VesSAM: Efficient Multi-Prompting for Segmenting Complex Vessel [68.24765319399286]
  We present VesSAM, a powerful and efficient framework tailored for 2D vessel segmentation.<n>VesSAM integrates (1) a convolutional adapter to enhance local texture features, (2) a multi-prompt encoder that fuses anatomical prompts, and (3) a lightweight mask decoder to reduce jagged artifacts.<n>VesSAM consistently outperforms state-of-the-art PEFT-based SAM variants by over 10% Dice and 13% IoU.
  arXiv  Detail & Related papers  (2025-11-02T15:47:05Z)
• AD-SAM: Fine-Tuning the Segment Anything Vision Foundation Model for Autonomous Driving Perception [3.298091299319354]
  The Autonomous Driving Segment Anything Model (AD-SAM) is a fine-tuned vision foundation model for semantic segmentation in autonomous driving (AD)<n>AD-SAM extends the Segment Anything Model (SAM) with a dual-encoder and deformable decoder tailored to spatial and geometric complexity of road scenes.<n> Experiments show that AD-SAM surpasses SAM, Generalized SAM (G-SAM), and a deep learning baseline (DeepLabV3) in segmentation accuracy.
  arXiv  Detail & Related papers  (2025-10-30T23:30:33Z)
• PicoSAM2: Low-Latency Segmentation In-Sensor for Edge Vision Applications [10.20223636234956]
  PicoSAM2, a lightweight (1.3M parameters, 336M MACs) promptable segmentation model optimized for edge and in-sensor execution, including the Sony IMX500.<n>On COCO and LVIS, it achieves 51.9% and 44.9% mIoU, respectively.<n>The quantized model (1.22MB) runs at 14.3 ms on the IMX500-achieving 86 MACs/cycle, making it the only model meeting both memory and compute constraints for in-sensor deployment.
  arXiv  Detail & Related papers  (2025-06-23T16:16:02Z)
• HRSAM: Efficient Interactive Segmentation in High-Resolution Images [59.537068118473066]
  Segment Anything Model (SAM) has advanced interactive segmentation but is limited by the high computational cost on high-resolution images. We focus on visual length extrapolation and propose a lightweight model named HRSAM. The extrapolation enables HRSAM trained on low resolutions to generalize to high resolutions.
  arXiv  Detail & Related papers  (2024-07-02T09:51:56Z)
• SAM-Lightening: A Lightweight Segment Anything Model with Dilated Flash Attention to Achieve 30 times Acceleration [6.515075311704396]
  Segment Anything Model (SAM) has garnered significant attention in segmentation tasks due to their zero-shot generalization ability. We introduce SAM-Lightening, a variant of SAM, that features a re-engineered attention mechanism, termed Dilated Flash Attention. Experiments on COCO and LVIS reveal that SAM-Lightening significantly outperforms the state-of-the-art methods in both run-time efficiency and segmentation accuracy.
  arXiv  Detail & Related papers  (2024-03-14T09:07:34Z)
• Q-Segment: Segmenting Images In-Sensor for Vessel-Based Medical Diagnosis [13.018482089796159]
  We present "Q-Segment", a quantized real-time segmentation algorithm, and conduct a comprehensive evaluation on a low-power edge vision platform with the Sony IMX500. Q-Segment achieves ultra-low inference time in-sensor only 0.23 ms and power consumption of only 72mW. This research contributes valuable insights into edge-based image segmentation, laying the foundation for efficient algorithms tailored to low-power environments.
  arXiv  Detail & Related papers  (2023-12-15T15:01:41Z)
• EdgeSAM: Prompt-In-the-Loop Distillation for SAM [87.52687622659904]
  EdgeSAM is an accelerated variant of the Segment Anything Model (SAM)<n>Our approach involves distilling the original ViT-based SAM image encoder into a purely CNN-based architecture.<n>It is the first SAM variant that can run at over 30 FPS on an iPhone 14.
  arXiv  Detail & Related papers  (2023-12-11T18:59:52Z)
• Stable Segment Anything Model [79.9005670886038]
  The Segment Anything Model (SAM) achieves remarkable promptable segmentation given high-quality prompts. This paper presents the first comprehensive analysis on SAM's segmentation stability across a diverse spectrum of prompt qualities. Our solution, termed Stable-SAM, offers several advantages: 1) improved SAM's segmentation stability across a wide range of prompt qualities, while 2) retaining SAM's powerful promptable segmentation efficiency and generality.
  arXiv  Detail & Related papers  (2023-11-27T12:51:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.