Unlocking Diffusion Hierarchies: Adaptive Timestep Selection for Zero-Shot Segmentation

• URL: http://arxiv.org/abs/2606.15590v1
• Date: Sun, 14 Jun 2026 04:21:21 GMT
• Title: Unlocking Diffusion Hierarchies: Adaptive Timestep Selection for Zero-Shot Segmentation
• Abstract summary: Contextual Similarity Maps fuse high-resolution attention maps with rich U-Net encoder features.<n>We identify an emergent hierarchical semantic progression within the denoising process of various diffusion models.<n>Our method consistently outperforms existing zero-shot segmentation baselines.
• Score: 1.2074552857379275
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Zero-shot segmentation has recently shown notable improvement by leveraging the rich visual priors in large-scale text-to-image diffusion models, such as Stable Diffusion. However, current diffusion-based methods often face limitations due to the trade-off between spatial resolution and contextual information, as well as their reliance on a single static timestep for feature extraction. To overcome these challenges, our work introduces two key advancements. First, our Contextual Similarity Maps fuse high-resolution attention maps with rich U-Net encoder features, providing both fine-grained and robust per-pixel representations. Second, we identify an emergent hierarchical semantic progression within the denoising process of various diffusion models: representations transition from part-level abstractions at earlier timesteps to object-level abstractions at later stages. Leveraging this insight, we introduce a mechanism to adaptively select the optimal timestep for each pixel. Extensive experiments demonstrate that our method consistently outperforms existing zero-shot segmentation baselines, validating the efficacy of combining contextual features with dynamic, hierarchical timestep selection.

Related papers

• Rarity-Aware Discrete Diffusion with Spatially Consistent Decoding for Photo-Realistic Image Super-Resolution [80.28905986756685]
  DiMOO-SR is a rarity-aware multimodal discrete diffusion framework for photo-realistic image Super-Resolution.<n>Experiments on widely used real-world SR benchmarks demonstrate that DiMOO-SR achieves competitive perceptual quality with only a few parallel decoding steps.
  arXiv  Detail & Related papers  (2026-07-20T07:01:01Z)
• From Diffusion to Rectified Flow: Rethinking Text-Based Segmentation [6.365160557699766]
  Text-based image segmentation aims to delineate object boundaries within an image from text prompts.<n>Recent studies have shown that diffusion models can provide rich multimodal semantic features.<n>We propose RLFSeg, a novel framework that leverages Rectified Flow to learn direct mapping from the image to the segmentation mask within the latent space.
  arXiv  Detail & Related papers  (2026-05-06T07:40:45Z)
• CIAR: Interval-based Collaborative Decoding for Image Generation Acceleration [47.52310736101766]
  Auto-regressive (AR) models have recently made notable progress in image generation, achieving performance comparable to diffusion-based approaches.<n>We address this via a cloud-device collaboration framework textbfCIAR, which utilizes on-device self-verification to handle two key properties of visual synthesis.<n>Our solution centers on an on-device token uncertainty quantifier, which adopts continuous probability intervals to accelerate processing and make it feasible for large visual vocabularies.
  arXiv  Detail & Related papers  (2026-03-26T14:04:50Z)
• OptiCorNet: Optimizing Sequence-Based Context Correlation for Visual Place Recognition [2.3093110834423616]
  This paper presents OptiCorNet, a novel sequence modeling framework.<n>It unifies spatial feature extraction and temporal differencing into a differentiable, end-to-end trainable module.<n>Our approach outperforms state-of-the-art baselines under challenging seasonal and viewpoint variations.
  arXiv  Detail & Related papers  (2025-07-19T04:29:43Z)
• Flow Intelligence: Robust Feature Matching via Temporal Signature Correlation [12.239059174851654]
  Flow Intelligence is a paradigm-shifting approach that focuses on temporal motion patterns exclusively.<n>Our method extracts motion signatures from pixel blocks across consecutive frames and extract temporal motion signatures between videos.<n>By leveraging motion rather than appearance, Flow Intelligence enables robust, real-time video feature matching in diverse environments.
  arXiv  Detail & Related papers  (2025-04-16T10:25:20Z)
• Mask Approximation Net: A Novel Diffusion Model Approach for Remote Sensing Change Captioning [14.994719390886774]
  Remote sensing image change description represents an innovative multimodal task within the realm of remote sensing processing.<n>This paper proposes a paradigm that shift towards data distribution learning using diffusion models, reinforced by frequency-domain noise filtering.<n>We validate the effectiveness of our proposed method across several datasets for remote sensing change detection and description.
  arXiv  Detail & Related papers  (2024-12-26T11:35:57Z)
• High-Precision Dichotomous Image Segmentation via Probing Diffusion Capacity [69.32473738284374]
  Diffusion models have revolutionized text-to-image synthesis by delivering exceptional quality, fine detail resolution, and strong contextual awareness.<n>We propose DiffDIS, a diffusion-driven segmentation model that taps into the potential of the pre-trained U-Net within diffusion models.<n>Experiments on the DIS5K dataset demonstrate the superiority of DiffDIS, achieving state-of-the-art results through a streamlined inference process.
  arXiv  Detail & Related papers  (2024-10-14T02:49:23Z)
• Binarized Diffusion Model for Image Super-Resolution [61.963833405167875]
  Binarization, an ultra-compression algorithm, offers the potential for effectively accelerating advanced diffusion models (DMs) Existing binarization methods result in significant performance degradation. We introduce a novel binarized diffusion model, BI-DiffSR, for image SR.
  arXiv  Detail & Related papers  (2024-06-09T10:30:25Z)
• Look Back and Forth: Video Super-Resolution with Explicit Temporal Difference Modeling [105.69197687940505]
  We propose to explore the role of explicit temporal difference modeling in both LR and HR space. To further enhance the super-resolution result, not only spatial residual features are extracted, but the difference between consecutive frames in high-frequency domain is also computed.
  arXiv  Detail & Related papers  (2022-04-14T17:07:33Z)
• Learning Discriminative Shrinkage Deep Networks for Image Deconvolution [122.79108159874426]
  We propose an effective non-blind deconvolution approach by learning discriminative shrinkage functions to implicitly model these terms. Experimental results show that the proposed method performs favorably against the state-of-the-art ones in terms of efficiency and accuracy.
  arXiv  Detail & Related papers  (2021-11-27T12:12:57Z)
• TimeLens: Event-based Video Frame Interpolation [54.28139783383213]
  We introduce Time Lens, a novel indicates equal contribution method that leverages the advantages of both synthesis-based and flow-based approaches. We show an up to 5.21 dB improvement in terms of PSNR over state-of-the-art frame-based and event-based methods.
  arXiv  Detail & Related papers  (2021-06-14T10:33:47Z)
• Coarse-Fine Networks for Temporal Activity Detection in Videos [45.03545172714305]
  We introduce 'Co-Fine Networks', a two-stream architecture which benefits from different abstractions of temporal resolution to learn better video representations for long-term motion. We show that our method can outperform the state-of-the-arts for action detection in public datasets with a significantly reduced compute and memory footprint.
  arXiv  Detail & Related papers  (2021-03-01T20:48:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.