GoTTA be Diverse: Rethinking Memory Policies for Test-Time Adaptation

• URL: http://arxiv.org/abs/2605.19890v1
• Date: Tue, 19 May 2026 14:18:46 GMT
• Title: GoTTA be Diverse: Rethinking Memory Policies for Test-Time Adaptation
• Abstract summary: Test-time adaptation (TTA) enables a pre-trained model to adapt online to an unlabeled test stream under distribution shift.<n>While most TTA research focuses on the adaptation objective, practical streams also depend critically on the memory used to select which test samples drive adaptation.<n>We introduce Guided Observational Test-Time Adaptation (GOTTA), a family of diversity-aware memory policies that combine class-balanced allocation with feature-space diversity.
• Score: 54.70334474895361
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Test-time adaptation (TTA) enables a pre-trained model to adapt online to an unlabeled test stream under distribution shift. While most TTA research focuses on the adaptation objective, practical streams also depend critically on the memory used to select which test samples drive adaptation. Existing memory mechanisms are usually evaluated as components of specific TTA algorithms, making it difficult to isolate which memory design choices matter and when they matter. In this work, we provide a systematic benchmark that decouples memory from the adaptation algorithm and evaluates memory policies under unified conditions across i.i.d., non-i.i.d., continual, and practical test streams. Our study shows that effective memory management requires more than retaining recent or class-balanced samples. In particular, intra-class diversity is a key factor for avoiding redundant buffers and maintaining representative adaptation signals under temporally correlated and label-skewed streams. Motivated by this finding, we introduce Guided Observational Test-Time Adaptation (GOTTA), a family of diversity-aware memory policies that combine class-balanced allocation with feature-space diversity. GOTTA memories act as drop-in replacements for existing buffers and can be paired with different TTA objectives. Across corruption benchmarks and video-stream settings, diversity-aware memory improves adaptation most clearly under constrained memory budgets and challenging non-i.i.d. streams, while remaining competitive as memory capacity increases. These results highlight memory management as a first-class component of robust test-time adaptation and identify diversity as a central principle for practical TTA.

Related papers

• Dual-Grained Agent Memory and Shapley Context Attribution for Multimodal Agentic Learner [55.80248281761638]
  multimodal large language models (MLLMs) deliver impressive perception yet still falter on scientific and mathematical reasoning.<n>We propose DG-Mem, a dual-grained agentic memory framework that augments a frozen MLLM with a non- externally stored memory built once from training-time rollouts and consulted read-only at test time.
  arXiv  Detail & Related papers  (2026-08-24T13:56:01Z)
• MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning [17.668597279485407]
  MILES is a framework that dynamically expands step-wise memory and applies correctness-optimized memory composition under realistic test-time constraints.<n> MILES consistently matches or outperforms prior methods while achieving superior accuracy-efficiency tradeoffs.
  arXiv  Detail & Related papers  (2026-07-08T03:51:37Z)
• Is One Score Enough? Rethinking the Evaluation of Sequentially Evolving LLM Memory [50.857546269660276]
  We introduce SeqMem-Eval, a diagnostic evaluation framework for sequentially evolving large language models (LLMs) memory.<n>Rather than focusing only on final performance, SeqMem-Eval evaluates how memory states evolve, generalize, consolidate experience, and retain useful information during sequential inference.
  arXiv  Detail & Related papers  (2026-05-14T20:15:22Z)
• Test-Time Adaptive Object Detection with Foundation Model [39.96498227103011]
  We propose the first foundation model-powered test-time adaptive object detection method.<n>We design a Multi-modal Prompt-based Mean-Teacher framework for vision-language detector-driven test-time adaptation.<n>To guarantee high-quality pseudo-labels in every test batch, we maintain an Instance Dynamic Memory.
  arXiv  Detail & Related papers  (2025-10-29T05:19:38Z)
• Dynamic Dual Buffer with Divide-and-Conquer Strategy for Online Continual Learning [10.599650191041217]
  Online Continual Learning (OCL) presents a complex learning environment in which new data arrives in a batch-to-batch online format.<n>We introduce an innovative memory framework that incorporates a short-term memory system to retain dynamic information and a long-term memory system to archive enduring knowledge.
  arXiv  Detail & Related papers  (2025-05-23T16:57:04Z)
• SURGEON: Memory-Adaptive Fully Test-Time Adaptation via Dynamic Activation Sparsity [30.260783715373382]
  Test-time adaptation (TTA) has emerged to improve the performance of deep models by adapting them to unlabeled target data online.<n>Yet, the significant memory cost, particularly in resource-constrained terminals, impedes the effective deployment of most backward-propagation-based TTA methods.<n>To tackle memory constraints, we introduce SURGEON, a method that substantially reduces memory cost while preserving comparable accuracy improvements.
  arXiv  Detail & Related papers  (2025-03-26T09:27:09Z)
• Resilient Practical Test-Time Adaptation: Soft Batch Normalization Alignment and Entropy-driven Memory Bank [24.096250529224914]
  We propose a practical test-time adaptation (ResiTTA) method focused on parameter resilience and data quality. We use an entropy-driven memory bank that accounts for timeliness, the persistence of over-confident samples, and sample uncertainty for high-quality data in adaptation. We empirically validate ResiTTA across various benchmark datasets, demonstrating state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-01-26T03:24:55Z)
• AdaNPC: Exploring Non-Parametric Classifier for Test-Time Adaptation [64.9230895853942]
  Domain generalization can be arbitrarily hard without exploiting target domain information. Test-time adaptive (TTA) methods are proposed to address this issue. In this work, we adopt Non-Parametric to perform the test-time Adaptation (AdaNPC)
  arXiv  Detail & Related papers  (2023-04-25T04:23:13Z)
• EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularization [71.70414291057332]
  TTA may primarily be conducted on edge devices with limited memory. Long-term adaptation often leads to catastrophic forgetting and error accumulation. We present lightweight meta networks that can adapt the frozen original networks to the target domain.
  arXiv  Detail & Related papers  (2023-03-03T13:05:30Z)
• Robust Continual Test-time Adaptation: Instance-aware BN and Prediction-balanced Memory [58.72445309519892]
  We present a new test-time adaptation scheme that is robust against non-i.i.d. test data streams. Our novelty is mainly two-fold: (a) Instance-Aware Batch Normalization (IABN) that corrects normalization for out-of-distribution samples, and (b) Prediction-balanced Reservoir Sampling (PBRS) that simulates i.i.d. data stream from non-i.i.d. stream in a class-balanced manner.
  arXiv  Detail & Related papers  (2022-08-10T03:05:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.