Zero-Shot Object Re-Identification in Egocentric Kitchen Videos via Multi-Stage SAM3 Feature Fusion

• URL: http://arxiv.org/abs/2605.26383v1
• Date: Mon, 25 May 2026 23:01:41 GMT
• Title: Zero-Shot Object Re-Identification in Egocentric Kitchen Videos via Multi-Stage SAM3 Feature Fusion
• Abstract summary: We study zero-shot object ReID on the EPIC-Kitchens benchmark.<n>The goal is to match active food and kitchen-tool instances across frames using only pre-trained visual features.
• Score: 59.32099847726022
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Object re-identification (ReID) in egocentric kitchen videos is challenging due to rapid viewpoint changes, frequent occlusions, cluttered scenes, and large intra-class appearance variations. Objects may leave and re-enter the field of view, and the large diversity of instances with limited annotations makes supervised ReID difficult to scale, motivating zero-shot approaches. We study zero-shot object ReID on the EPIC-Kitchens benchmark, where the goal is to match active food and kitchen-tool instances across frames using only pre-trained visual features. We first evaluate five state-of-the-art feature extractors, including Vision-Language Models (VLMs) - CLIP, DINOv2, DreamSim, I-JEPA, and SAM3 - and show that zero-shot methods fail, with the best baseline achieving only 45.3% mAP. We then propose an Enhanced SAM3 ReID Pipeline, a zero-shot multi-stage method built around SAM3 segmentation as the core component. Stage 1 uses SAM3 to suppress background clutter. Stage 2 fuses embeddings from SAM3, DINOv2, and CLIP into a single L2-normalized descriptor. Stage 3 augments cosine similarity with mask-shape IoU for geometric consistency, and Stage 4 applies k-reciprocal re-ranking. The full pipeline improves performance by 7.5% mAP to 52.8%.

Related papers

• Scene-SAM3D: Multi-View Scene Asset Generation Without Fine-Tuning [51.11861596631423]
  We introduce Scene-SAM3D, a training-free framework that extends SAM3D from single-view object generation to multi-view scene asset generation.<n>Scene-SAM3D selects a compact set of complementary views, reducing observation redundancy while providing additional evidence for regions occluded in individual views.<n>It performs step-efficient latent velocity fusion to integrate multi-view evidence and suppress cross-view conflicts in canonical space.
  arXiv  Detail & Related papers  (2026-07-18T12:51:17Z)
• The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentation [65.24213788883016]
  This report presents our winning solution to the 5th PVUW MeViS-Text Challenge.<n>We build a fully training-free pipeline that combines strong multimodal large language models with SAM3.<n>Our method ranks first on the PVUW 2026 MeViS-Text test set, achieving a Final score of 0.909064 and a J&F score of 0.7897026.
  arXiv  Detail & Related papers  (2026-04-01T02:42:30Z)
• Re-Prompting SAM 3 via Object Retrieval: 3rd of the 5th PVUW MOSE Track [51.15679610275694]
  We develop an automatic re-prompting framework to improve robustness under target disappearance and reappearance.<n>Our solution achieves a J&F of 51.17% on the test set, ranking 3rd in the MOSEv2 track.
  arXiv  Detail & Related papers  (2026-03-24T23:45:15Z)
• SAM-pose2seg: Pose-Guided Human Instance Segmentation in Crowds [15.318646611581741]
  We adapt Segment Anything (SAM) for pose-guided segmentation with minimal encoder modifications.<n>We incorporate pose keypoints with high visibility into the iterative correction process.<n>During inference, we simplify prompting by selecting only the three keypoints with the highest visibility.
  arXiv  Detail & Related papers  (2026-01-13T21:12:03Z)
• SegEarth-OV3: Exploring SAM 3 for Open-Vocabulary Semantic Segmentation in Remote Sensing Images [51.42466259821335]
  We present a preliminary exploration of applying SAM 3 to the remote sensing OVSS task without any training.<n>First, we implement a mask fusion strategy that combines the outputs from SAM 3's semantic segmentation head and the Transformer decoder.<n>Second, we utilize the presence score from the presence head to filter out categories that do not exist in the scene.
  arXiv  Detail & Related papers  (2025-12-09T15:42:28Z)
• VesSAM: Efficient Multi-Prompting for Segmenting Complex Vessel [68.24765319399286]
  We present VesSAM, a powerful and efficient framework tailored for 2D vessel segmentation.<n>VesSAM integrates (1) a convolutional adapter to enhance local texture features, (2) a multi-prompt encoder that fuses anatomical prompts, and (3) a lightweight mask decoder to reduce jagged artifacts.<n>VesSAM consistently outperforms state-of-the-art PEFT-based SAM variants by over 10% Dice and 13% IoU.
  arXiv  Detail & Related papers  (2025-11-02T15:47:05Z)
• WSI-SAM: Multi-resolution Segment Anything Model (SAM) for histopathology whole-slide images [8.179859593451285]
  We present WSI-SAM, enhancing Segment Anything Model (SAM) with precise object segmentation capabilities for histopathology images. To fully exploit pretrained knowledge while minimizing training overhead, we keep SAM frozen, introducing only minimal extra parameters. Our model outperforms SAM by 4.1 and 2.5 percent points on a ductal carcinoma in situ (DCIS) segmentation tasks and breast cancer metastasis segmentation task.
  arXiv  Detail & Related papers  (2024-03-14T10:30:43Z)
• SAM-Deblur: Let Segment Anything Boost Image Deblurring [21.964258084389243]
  We propose a framework SAM-Deblur, integrating prior knowledge from the Segment Anything Model (SAM) into the deblurring task. Experimental results on the RealBlurJ, ReloBlur, and REDS datasets reveal that incorporating our methods improves GoPro-trained NAFNet's PSNR by 0.05, 0.96, and 7.03, respectively.
  arXiv  Detail & Related papers  (2023-09-05T14:33:56Z)
• Personalize Segment Anything Model with One Shot [52.54453744941516]
  We propose a training-free Personalization approach for Segment Anything Model (SAM) Given only a single image with a reference mask, PerSAM first localizes the target concept by a location prior. PerSAM segments it within other images or videos via three techniques: target-guided attention, target-semantic prompting, and cascaded post-refinement.
  arXiv  Detail & Related papers  (2023-05-04T17:59:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.