SSAM: Self-Supervised Association Modeling for Test-Time Adaption

• URL: http://arxiv.org/abs/2506.00513v1
• Date: Sat, 31 May 2025 11:13:07 GMT
• Title: SSAM: Self-Supervised Association Modeling for Test-Time Adaption
• Abstract summary: We propose SSAM (Self-Supervised Association Modeling), a new TTA framework that enables dynamic encoder refinement through dual-phase association learning.<n>Our method operates via two synergistic components: 1) Soft Prototype Estimation (SPE), which estimates probabilistic category associations to guide feature space reorganization, and 2) Prototype-anchored Image Reconstruction (PIR), enforcing encoder stability through cluster-conditional image feature reconstruction.
• Score: 42.00379819876794
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Test-time adaption (TTA) has witnessed important progress in recent years, the prevailing methods typically first encode the image and the text and design strategies to model the association between them. Meanwhile, the image encoder is usually frozen due to the absence of explicit supervision in TTA scenarios. We identify a critical limitation in this paradigm: While test-time images often exhibit distribution shifts from training data, existing methods persistently freeze the image encoder due to the absence of explicit supervision during adaptation. This practice overlooks the image encoder's crucial role in bridging distribution shift between training and test. To address this challenge, we propose SSAM (Self-Supervised Association Modeling), a new TTA framework that enables dynamic encoder refinement through dual-phase association learning. Our method operates via two synergistic components: 1) Soft Prototype Estimation (SPE), which estimates probabilistic category associations to guide feature space reorganization, and 2) Prototype-anchored Image Reconstruction (PIR), enforcing encoder stability through cluster-conditional image feature reconstruction. Comprehensive experiments across diverse baseline methods and benchmarks demonstrate that SSAM can surpass state-of-the-art TTA baselines by a clear margin while maintaining computational efficiency. The framework's architecture-agnostic design and minimal hyperparameter dependence further enhance its practical applicability.

Related papers

• Rarity-Aware Discrete Diffusion with Spatially Consistent Decoding for Photo-Realistic Image Super-Resolution [80.28905986756685]
  DiMOO-SR is a rarity-aware multimodal discrete diffusion framework for photo-realistic image Super-Resolution.<n>Experiments on widely used real-world SR benchmarks demonstrate that DiMOO-SR achieves competitive perceptual quality with only a few parallel decoding steps.
  arXiv  Detail & Related papers  (2026-07-20T07:01:01Z)
• HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval [32.85064790572752]
  Composed Image Retrieval (CIR) is a flexible image retrieval paradigm that enables users to accurately locate the target image.<n>This paper proposes a cHrono-synergiA roBust progressIve learning framework for composed image reTrieval (HABIT)<n>Experiments conducted on two standard CIR datasets demonstrate that HABIT significantly outperforms most methods under various noise ratios.
  arXiv  Detail & Related papers  (2026-04-20T10:02:18Z)
• Bootstrapping Video Semantic Segmentation Model via Distillation-assisted Test-Time Adaptation [58.40817037271021]
  DiTTA (Distillation-assisted Test-Time Adaptation) is a novel framework that converts an ISS model into a temporally-aware VSS model without annotated videos.<n>We demonstrate DiTTA's effectiveness, achieving competitive or superior performance relative to fully-supervised VSS methods.
  arXiv  Detail & Related papers  (2026-04-13T03:47:08Z)
• Tuning Real-World Image Restoration at Inference: A Test-Time Scaling Paradigm for Flow Matching Models [26.002982722748726]
  ResFlow-Tuner is an image restoration framework based on the state-of-the-art flow matching model, FLUX.1-dev.<n>We propose a training-free test-time scaling paradigm tailored for image restoration.
  arXiv  Detail & Related papers  (2026-03-23T14:33:43Z)
• ITO: Images and Texts as One via Synergizing Multiple Alignment and Training-Time Fusion [14.791563751107502]
  ITO is a framework addressing the limitation through two synergistic mechanisms.<n>We show that ITO consistently outperforms strong baselines across classification, retrieval, and multimodal benchmarks.<n>Our analysis reveals that while multiple alignment drives discriminative power, training-time fusion acts as a critical structural regularizer.
  arXiv  Detail & Related papers  (2026-03-03T09:08:53Z)
• Fast, faithful and photorealistic diffusion-based image super-resolution with enhanced Flow Map models [8.773008452897791]
  FlowMapSR is a novel diffusion-based framework for image super-resolution explicitly designed for efficient inference.<n>We show that FlowMapSR achieves a better balance between reconstruction faithfulness and photorealism than recent state-of-the-art methods for both x4 and x8 upscaling.
  arXiv  Detail & Related papers  (2026-01-23T11:25:04Z)
• Rethinking Autoregressive Models for Lossless Image Compression via Hierarchical Parallelism and Progressive Adaptation [75.58269386927076]
  Autoregressive (AR) models are often dismissed as impractical due to prohibitive computational cost.<n>This work re-thinks this paradigm, introducing a framework built on hierarchical parallelism and progressive adaptation.<n> Experiments on diverse datasets (natural, satellite, medical) validate that our method achieves new state-of-the-art compression.
  arXiv  Detail & Related papers  (2025-11-14T06:27:58Z)
• Vision Foundation Models as Effective Visual Tokenizers for Autoregressive Image Generation [66.73899356886652]
  We build an image tokenizer directly atop pre-trained vision foundation models.<n>Our proposed image tokenizer, VFMTok, achieves substantial improvements in image reconstruction and generation quality.<n>It further boosts autoregressive (AR) generation -- achieving a gFID of 2.07 on ImageNet benchmarks.
  arXiv  Detail & Related papers  (2025-07-11T09:32:45Z)
• One-Way Ticket:Time-Independent Unified Encoder for Distilling Text-to-Image Diffusion Models [65.96186414865747]
  Text-to-Image (T2I) diffusion models face a trade-off between inference speed and image quality.<n>We introduce the first Time-independent Unified TiUE for the student model UNet architecture.<n>Using a one-pass scheme, TiUE shares encoder features across multiple decoder time steps, enabling parallel sampling.
  arXiv  Detail & Related papers  (2025-05-28T04:23:22Z)
• Efficient Knowledge Distillation of SAM for Medical Image Segmentation [0.04672991859386895]
  The Segment Anything Model (SAM) has set a new standard in interactive image segmentation, offering robust performance across various tasks.<n>We propose a novel knowledge distillation approach, KD SAM, which incorporates both encoder and decoder optimization through a combination of Mean Squared Error (MSE) and Perceptual Loss.<n> KD SAM effectively balances segmentation accuracy and computational efficiency, making it well-suited for real-time medical image segmentation applications in resource-constrained environments.
  arXiv  Detail & Related papers  (2025-01-28T06:33:30Z)
• Efficient One-Step Diffusion Refinement for Snapshot Compressive Imaging [8.819370643243012]
  Coded Aperture Snapshot Spectral Imaging (CASSI) is a crucial technique for capturing three-dimensional multispectral images (MSIs) Current state-of-the-art methods, predominantly end-to-end, face limitations in reconstructing high-frequency details. This paper introduces a novel one-step Diffusion Probabilistic Model within a self-supervised adaptation framework for Snapshot Compressive Imaging.
  arXiv  Detail & Related papers  (2024-09-11T17:02:10Z)
• PMT: Progressive Mean Teacher via Exploring Temporal Consistency for Semi-Supervised Medical Image Segmentation [51.509573838103854]
  We propose a semi-supervised learning framework, termed Progressive Mean Teachers (PMT), for medical image segmentation. Our PMT generates high-fidelity pseudo labels by learning robust and diverse features in the training process. Experimental results on two datasets with different modalities, i.e., CT and MRI, demonstrate that our method outperforms the state-of-the-art medical image segmentation approaches.
  arXiv  Detail & Related papers  (2024-09-08T15:02:25Z)
• Binarized Diffusion Model for Image Super-Resolution [61.963833405167875]
  Binarization, an ultra-compression algorithm, offers the potential for effectively accelerating advanced diffusion models (DMs) Existing binarization methods result in significant performance degradation. We introduce a novel binarized diffusion model, BI-DiffSR, for image SR.
  arXiv  Detail & Related papers  (2024-06-09T10:30:25Z)
• Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversion [35.21106030549071]
  Diffusion Probabilistic Models (DPMs) are dominant force in text-to-image generation tasks. We propose an alternative view of state-of-the-art DPMs as a way of inverting advanced Vision-Language Models (VLMs) By directly optimizing images with the supervision of discriminative VLMs, the proposed method can potentially achieve a better text-image alignment.
  arXiv  Detail & Related papers  (2024-02-26T05:08:40Z)
• Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond [62.730497582218284]
  We develop a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data.
  arXiv  Detail & Related papers  (2020-04-30T03:23:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.