SAM3 Self-Distillation for Fine-Grained GOOSE 2D Semantic Segmentation

• URL: http://arxiv.org/abs/2606.20130v1
• Date: Thu, 18 Jun 2026 11:54:07 GMT
• Title: SAM3 Self-Distillation for Fine-Grained GOOSE 2D Semantic Segmentation
• Abstract summary: We describe our 4th-place entry to the ICRA 2026 GOOSE 2D Fine-Grained Semantic Challenge.<n>Our model adapts the image of a recent visual foundation model, Segment Anything Model 3.
• Score: 3.487520041555239
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We describe our 4th-place entry to the ICRA 2026 GOOSE 2D Fine-Grained Semantic Segmentation Challenge, which reached a composite mean Intersection-over-Union (mIoU) of 69.73% on the official 1,815-image test set. Our model adapts the image encoder of a recent visual foundation model, Segment Anything Model 3 (SAM3), with a lightweight decoder. Beyond this, we contribute two techniques and one empirical finding: (i) a self-distillation scheme that re-uses SAM3 itself, prompted with ground-truth boxes, as a teacher on the classes where it outperforms our own model; (ii) an image-level multi-scale test-time augmentation scheme that restores multi-scale inference for a fixed-input-size model by rescaling the image rather than the model input; and (iii) the finding that an aggressive photometric distortion from a winning 2025 GOOSE 2D entry, transplanted onto our pipeline, is its single largest source of improvement.

Related papers

• SeFi-Image: A Text-to-Image Foundation Model with Semantic-First Diffusion [0.0]
  SeFi-Image is a text-to-image foundation model built upon semantic-first diffusion.<n>We instantiate SeFi-Image at three model scales, 1B, 2B, and 5B parameters.<n>SeFi-Image achieves strong performance on a wide range of benchmarks, including GenEval, DPG, LongTextBench, OneIG, and DMDG-2K.
  arXiv  Detail & Related papers  (2026-06-21T16:10:22Z)
• Diffusion Image Generation with Explicit Modeling of Data Manifold Geometry [64.61931485161833]
  Image generative models aim to sample data points from the underlying data manifold.<n>We propose the Data Manifold-aware Image diffusioN moDel (MIND)<n>MIND integrates discrete patch tokenization into the score function of a continuous diffusion model.
  arXiv  Detail & Related papers  (2026-05-25T08:43:14Z)
• D3S2: Diffusion-Guided Dataset Distillation for Semantic Segmentation [8.30626759264565]
  We identify three key challenges for segmentation DD: (i) long-tailed class imbalance, (ii) the need for strict pixel-wise alignment between images and dense labels, and (iii) the high computational cost of optimizing high-resolution data with complex models.<n>In Class-Balanced Mask Selection, we construct a representative mask set via a greedy strategy that prioritizes underrepresented classes.<n>In Diffusion-Guided Image Synthesis, we employ a pretrained layout-to-image diffusion model to generate images conditioned on the selected masks, naturally ensuring alignment.
  arXiv  Detail & Related papers  (2026-05-24T12:01:38Z)
• Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders [74.72147962028265]
  Representation Autoencoders (RAEs) have shown distinct advantages in diffusion modeling on ImageNet.<n>We investigate whether this framework can scale to large-scale, freeform text-to-image (T2I) generation.
  arXiv  Detail & Related papers  (2026-01-22T18:58:16Z)
• ENSAM: an efficient foundation model for interactive segmentation of 3D medical images [0.0]
  ENSAM is a promptable model for universal 3D medical image segmentation.<n> ENSAM is designed to achieve good performance under limited data and computational budgets.<n> ENSAM was evaluated on hidden test set with multimodal 3D medical images.
  arXiv  Detail & Related papers  (2025-09-19T11:20:22Z)
• Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Image Generation [52.261584726401686]
  We present a novel direction to build an image tokenizer directly on top of a frozen vision foundation model.<n>Based on these designs, our proposed image tokenizer, VFMTok, achieves substantial improvements in image reconstruction and generation quality.
  arXiv  Detail & Related papers  (2025-07-11T09:32:45Z)
• HNOSeg-XS: Extremely Small Hartley Neural Operator for Efficient and Resolution-Robust 3D Image Segmentation [3.990336239705776]
  We propose a resolution-robust HNOSeg-XS architecture for medical image segmentation.<n>It is resolution robust, fast, memory efficient, and extremely parameter efficient.<n>It was tested on the BraTS'23, KiTS'23, and MVSeg'23 datasets with a Tesla V100 GPU.
  arXiv  Detail & Related papers  (2025-07-10T22:33:19Z)
• Adventurer: Optimizing Vision Mamba Architecture Designs for Efficiency [41.87857129429512]
  We introduce the Adventurer series models where we treat images as sequences of patch tokens and employ uni-directional language models to learn visual representations.<n>This modeling paradigm allows us to process images in a recurrent formulation with linear complexity relative to the sequence length.<n>In detail, we introduce two simple designs that seamlessly integrate image inputs into the causal inference framework.
  arXiv  Detail & Related papers  (2024-10-10T04:14:52Z)
• GroupMamba: Efficient Group-Based Visual State Space Model [66.35608254724566]
  State-space models (SSMs) have recently shown promise in capturing long-range dependencies with subquadratic computational complexity.<n>However, purely SSM-based models face critical challenges related to stability and achieving state-of-the-art performance in computer vision tasks.<n>Our paper addresses the challenges of scaling SSM-based models for computer vision, particularly the instability and inefficiency of large model sizes.
  arXiv  Detail & Related papers  (2024-07-18T17:59:58Z)
• Randomize to Generalize: Domain Randomization for Runway FOD Detection [1.4249472316161877]
  Tiny Object Detection is challenging due to small size, low resolution, occlusion, background clutter, lighting conditions and small object-to-image ratio. We propose a novel two-stage methodology Synthetic Image Augmentation (SRIA) to enhance generalization capabilities of models encountering 2D datasets. We report that detection accuracy improved from an initial 41% to 92% for OOD test set.
  arXiv  Detail & Related papers  (2023-09-23T05:02:31Z)
• Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors [117.61449210940955]
  We propose an efficient abnormal event detection model based on a lightweight masked auto-encoder (AE) applied at the video frame level. We introduce an approach to weight tokens based on motion gradients, thus shifting the focus from the static background scene to the foreground objects. We generate synthetic abnormal events to augment the training videos, and task the masked AE model to jointly reconstruct the original frames.
  arXiv  Detail & Related papers  (2023-06-21T06:18:05Z)
• HoloDiffusion: Training a 3D Diffusion Model using 2D Images [71.1144397510333]
  We introduce a new diffusion setup that can be trained, end-to-end, with only posed 2D images for supervision. We show that our diffusion models are scalable, train robustly, and are competitive in terms of sample quality and fidelity to existing approaches for 3D generative modeling.
  arXiv  Detail & Related papers  (2023-03-29T07:35:56Z)
• StraIT: Non-autoregressive Generation with Stratified Image Transformer [63.158996766036736]
  Stratified Image Transformer(StraIT) is a pure non-autoregressive(NAR) generative model. Our experiments demonstrate that StraIT significantly improves NAR generation and out-performs existing DMs and AR methods.
  arXiv  Detail & Related papers  (2023-03-01T18:59:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.