ReC-TTT: Contrastive Feature Reconstruction for Test-Time Training

• URL: http://arxiv.org/abs/2411.17869v1
• Date: Tue, 26 Nov 2024 20:38:02 GMT
• Title: ReC-TTT: Contrastive Feature Reconstruction for Test-Time Training
• Authors: Marco Colussi, Sergio Mascetti, Jose Dolz, Christian Desrosiers,
• Abstract summary: We propose a test-time training technique that can adapt a deep learning model to new unseen domains. ReC-TTT uses cross-reconstruction as an auxiliary task between a frozen encoder and two trainable encoders. We show that ReC-TTT achieves better results than other state-of-the-art techniques in most domain shift classification challenges.
• Score: 15.572896213775438
• License:
• Abstract: The remarkable progress in deep learning (DL) showcases outstanding results in various computer vision tasks. However, adaptation to real-time variations in data distributions remains an important challenge. Test-Time Training (TTT) was proposed as an effective solution to this issue, which increases the generalization ability of trained models by adding an auxiliary task at train time and then using its loss at test time to adapt the model. Inspired by the recent achievements of contrastive representation learning in unsupervised tasks, we propose ReC-TTT, a test-time training technique that can adapt a DL model to new unseen domains by generating discriminative views of the input data. ReC-TTT uses cross-reconstruction as an auxiliary task between a frozen encoder and two trainable encoders, taking advantage of a single shared decoder. This enables, at test time, to adapt the encoders to extract features that will be correctly reconstructed by the decoder that, in this phase, is frozen on the source domain. Experimental results show that ReC-TTT achieves better results than other state-of-the-art techniques in most domain shift classification challenges.

Related papers

• Enhancing Test Time Adaptation with Few-shot Guidance [35.13317598777832]
  Deep neural networks often encounter significant performance drops while facing with domain shifts between training (source) and test (target) data. Test Time Adaptation (TTA) methods have been proposed to adapt pre-trained source model to handle out-of-distribution streaming target data. We develop Few-Shot Test Time Adaptation (FS-TTA), a novel and practical setting that utilizes a few-shot support set on top of TTA.
  arXiv  Detail & Related papers  (2024-09-02T15:50:48Z)
• NC-TTT: A Noise Contrastive Approach for Test-Time Training [19.0284321951354]
  Noise-Contrastive Test-Time Training (NC-TTT) is a novel unsupervised TTT technique based on the discrimination of noisy feature maps. By learning to classify noisy views of projected feature maps, and then adapting the model accordingly on new domains, classification performance can be recovered by an important margin.
  arXiv  Detail & Related papers  (2024-04-12T10:54:11Z)
• ClusT3: Information Invariant Test-Time Training [19.461441044484427]
  Test-time training (TTT) methods have been developed in an attempt to mitigate these vulnerabilities. We propose a novel unsupervised TTT technique based on the Mutual of Mutual Information between multi-scale feature maps and a discrete latent representation. Experimental results demonstrate competitive classification performance on different popular test-time adaptation benchmarks.
  arXiv  Detail & Related papers  (2023-10-18T21:43:37Z)
• A Study of Forward-Forward Algorithm for Self-Supervised Learning [65.268245109828]
  We study the performance of forward-forward vs. backpropagation for self-supervised representation learning. Our main finding is that while the forward-forward algorithm performs comparably to backpropagation during (self-supervised) training, the transfer performance is significantly lagging behind in all the studied settings.
  arXiv  Detail & Related papers  (2023-09-21T10:14:53Z)
• Improved Test-Time Adaptation for Domain Generalization [48.239665441875374]
  Test-time training (TTT) adapts the learned model with test data. This work addresses two main factors: selecting an appropriate auxiliary TTT task for updating and identifying reliable parameters to update during the test phase. We introduce additional adaptive parameters for the trained model, and we suggest only updating the adaptive parameters during the test phase.
  arXiv  Detail & Related papers  (2023-04-10T10:12:38Z)
• TeST: Test-time Self-Training under Distribution Shift [99.68465267994783]
  Test-Time Self-Training (TeST) is a technique that takes as input a model trained on some source data and a novel data distribution at test time. We find that models adapted using TeST significantly improve over baseline test-time adaptation algorithms.
  arXiv  Detail & Related papers  (2022-09-23T07:47:33Z)
• CAFA: Class-Aware Feature Alignment for Test-Time Adaptation [50.26963784271912]
  Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. We propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously encourages a model to learn target representations in a class-discriminative manner.
  arXiv  Detail & Related papers  (2022-06-01T03:02:07Z)
• Test-time Adaptation with Slot-Centric Models [63.981055778098444]
  Slot-TTA is a semi-supervised scene decomposition model that at test time is adapted per scene through gradient descent on reconstruction or cross-view synthesis objectives. We show substantial out-of-distribution performance improvements against state-of-the-art supervised feed-forward detectors, and alternative test-time adaptation methods.
  arXiv  Detail & Related papers  (2022-03-21T17:59:50Z)
• Learning Invariant Representations across Domains and Tasks [81.30046935430791]
  We propose a novel Task Adaptation Network (TAN) to solve this unsupervised task transfer problem. In addition to learning transferable features via domain-adversarial training, we propose a novel task semantic adaptor that uses the learning-to-learn strategy to adapt the task semantics. TAN significantly increases the recall and F1 score by 5.0% and 7.8% compared to recently strong baselines.
  arXiv  Detail & Related papers  (2021-03-03T11:18:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.