Memory-Supported Synergistic Adaptation for Training-Free Test-Time Medical Image Segmentation

• URL: http://arxiv.org/abs/2607.17693v1
• Date: Mon, 20 Jul 2026 08:41:47 GMT
• Title: Memory-Supported Synergistic Adaptation for Training-Free Test-Time Medical Image Segmentation
• Abstract summary: Test-time adaptation (TTA) aims to mitigate distribution shifts by adapting models with unlabeled target data at inference time.<n>We propose Memory-Supported Synergistic Adaptation (MSSA), a novel training-free TTA framework for medical image segmentation.
• Score: 45.4173998742666
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Test-time adaptation (TTA) aims to mitigate distribution shifts by adapting models with unlabeled target data at inference time. While TTA with vision-language models (VLMs) has shown promising results in classification, extending it to medical image segmentation remains challenging. In this setting, the adaptation gains from optimizing on VLM-generated predictions are often outweighed by the degradation to the VLM's strong pretrained features caused by noisy, update-driven learning, resulting in limited and unstable improvements. We therefore propose Memory-Supported Synergistic Adaptation (MSSA), a novel training-free TTA framework for medical image segmentation. Without updating model parameters, MSSA dynamically selects reliable image-text predictions to construct an online memory, uses them as text-guided semantic priors, and couples them with cross-image structural alignment for robust adaptation. Specifically, MSSA consists of (i) a noise-aware memory construction module that filters and stabilizes cross-modal predictions, and (ii) a relevance-driven prototype alignment module that aligns the target sample with structurally consistent memory samples and their reliable predictions to improve adaptation. Extensive experiments on multiple medical segmentation benchmarks demonstrate that MSSA consistently improves VLM-based segmentation models and outperforms existing fine-tuning-based TTA methods by a clear margin, with gains of up to 12.2% DSC and 11.7% mIoU. Project page: https://lingrayy.github.io/MSSA/ .

Related papers

• MLLM-Guided Semantic Correction for Text-to-Video Generation [66.7723669725808]
  We introduce a training-free, interpretable mid-generation correction framework that integrates multimodal large language model feedback.<n>We propose two key modules: a Semantic Assessment Supervisor that generates intermediate preview frames for semantic evaluations and deviation diagnostics, and a Semantic Modification Assistant that corrects semantic drift during inference.<n>Our method improves semantic alignment, visual fidelity, and temporal consistency without modifying model parameters.
  arXiv  Detail & Related papers  (2026-08-17T12:54:10Z)
• Concept Alignment Contrast and Long-Short Prompt Memory for Test-Time Adaptation of SAM3 in Medical Image Segmentation [14.712933368175008]
  Test-Time Adaptation (TTA) is essential for improving the testing performance by updating the model on the fly without annotations.<n>Existing vision-language TTA methods are mainly driven by image-level uncertainty minimization.<n>We propose Concept Alignment Contrast and LongShort Prompt Memory for Test-Time Adaptation (CM-TTA) of SAM3 for medical images.
  arXiv  Detail & Related papers  (2026-06-22T07:43:19Z)
• Bootstrapping Video Semantic Segmentation Model via Distillation-assisted Test-Time Adaptation [58.40817037271021]
  DiTTA (Distillation-assisted Test-Time Adaptation) is a novel framework that converts an ISS model into a temporally-aware VSS model without annotated videos.<n>We demonstrate DiTTA's effectiveness, achieving competitive or superior performance relative to fully-supervised VSS methods.
  arXiv  Detail & Related papers  (2026-04-13T03:47:08Z)
• Dual-Teacher Distillation with Subnetwork Rectification for Black-Box Domain Adaptation [69.7036779439312]
  Black box domain adaptation represents a highly practical yet extremely challenging setting.<n>We propose a dual teacher distillation with subnetwork rectification (DDSR) model that exploits specific knowledge embedded in black box source models.<n>Our approach demonstrates consistent improvements over state of the art methods, including those using source data or models.
  arXiv  Detail & Related papers  (2026-03-24T07:54:19Z)
• SPEGC: Continual Test-Time Adaptation via Semantic-Prompt-Enhanced Graph Clustering for Medical Image Segmentation [12.0982298854338]
  Continual Test-Time Adaptation (CTTA) aims to enable pre-trained models to adapt to continuously changing unlabeled domains.<n>We propose a CTTA via Semantic-Prompt-Enhanced Graph Clustering (SPEGC) for medical image segmentation.
  arXiv  Detail & Related papers  (2026-03-12T03:22:43Z)
• EMO-TTA: Improving Test-Time Adaptation of Audio-Language Models for Speech Emotion Recognition [3.1649536621597973]
  Speech emotion recognition (SER) with audio-language models (ALMs) remains vulnerable to distribution shifts at test time.<n>We propose Emo-TTA, a lightweight, training-free adaptation framework that incrementally updates class-conditional statistics.<n>Emo-TTA operates on individual test samples without modifying model weights.
  arXiv  Detail & Related papers  (2025-09-29T20:52:01Z)
• Domain Adaptive Skin Lesion Classification via Conformal Ensemble of Vision Transformers [3.7305040207339286]
  This paper proposes a novel framework termed Conformal Ensemble of Vision Transformers (CE-ViTs)<n>It is designed to enhance image classification performance by prioritizing domain adaptation and model robustness, while accounting for uncertainty.<n>The framework achieves a high coverage rate of 90.38%, representing an improvement of 9.95% compared to the HAM10000 model.
  arXiv  Detail & Related papers  (2025-05-21T20:28:43Z)
• Test-Time Model Adaptation with Only Forward Passes [68.11784295706995]
  Test-time adaptation has proven effective in adapting a given trained model to unseen test samples with potential distribution shifts. We propose a test-time Forward-Optimization Adaptation (FOA) method. FOA runs on quantized 8-bit ViT, outperforms gradient-based TENT on full-precision 32-bit ViT, and achieves an up to 24-fold memory reduction on ImageNet-C.
  arXiv  Detail & Related papers  (2024-04-02T05:34:33Z)
• A Novel Benchmark for Few-Shot Semantic Segmentation in the Era of Foundation Models [7.428199805959228]
  Few-shot semantic segmentation (FSS) is a crucial challenge in computer vision.<n>With the emergence of vision foundation models (VFM) as generalist feature extractors, we seek to explore the adaptation of these models for FSS.<n>We propose a novel realistic benchmark with a simple and straightforward adaptation process tailored for this task.
  arXiv  Detail & Related papers  (2024-01-20T19:50:51Z)
• Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation [67.18144414660681]
  We propose a Fast-Slow Test-Time Adaptation (FSTTA) approach for online Vision-and-Language Navigation (VLN) Our method obtains impressive performance gains on four popular benchmarks.
  arXiv  Detail & Related papers  (2023-11-22T07:47:39Z)
• Generalized Face Forgery Detection via Adaptive Learning for Pre-trained Vision Transformer [54.32283739486781]
  We present a textbfForgery-aware textbfAdaptive textbfVision textbfTransformer (FA-ViT) under the adaptive learning paradigm. FA-ViT achieves 93.83% and 78.32% AUC scores on Celeb-DF and DFDC datasets in the cross-dataset evaluation.
  arXiv  Detail & Related papers  (2023-09-20T06:51:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.