Test-Time Adaptation with Online Personalized Energy-Based Cache for Fine-Grained Video Expression Recognition

• URL: http://arxiv.org/abs/2608.06467v1
• Date: Thu, 06 Aug 2026 18:02:20 GMT
• Title: Test-Time Adaptation with Online Personalized Energy-Based Cache for Fine-Grained Video Expression Recognition
• Abstract summary: Facial expression recognition (FER) in videos is challenging because models must identify subtle, temporally evolving affective states that vary across individuals.<n>Existing test-time adaptation (TTA) methods commonly update model parameters during inference, increasing computational cost and latency.<n>We introduce Energy-Based Cache Personalization (EB-CaP), a subject-based online TTA method for video FER that generates class-specific prototypes personalized to each target video.
• Score: 59.83490704563065
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Facial expression recognition (FER) in videos is challenging because models must identify subtle, temporally evolving affective states that vary across individuals. Although vision-language models provide transferable visual-semantic representations, models trained on subject-independent data often degrade under subject-specific distribution shifts at inference time. Existing test-time adaptation (TTA) methods commonly update model parameters during inference, increasing computational cost and latency. Cache-based methods avoid parameter updates, but they usually require enough target samples to form reliable class prototypes, which is difficult early in adaptation and for rarely observed classes. We introduce Energy-Based Cache Personalization (EB-CaP), a subject-based online TTA method for video FER that generates class-specific prototypes personalized to each target video. EB-CaP uses a lightweight energy-based model to sample prototypes from the current unlabeled video and populate a personalized cache online, without accumulating large amounts of target data or storing diverse source prototypes. Its energy function relies only on pretrained CLIP: similarities between the target video embedding and class text embeddings guide prototype sampling. In parallel, positive and negative caches store reliable and uncertain target embeddings. An adaptive entropy gate controls cache updates according to the evolving confidence distribution, while a diversity gate limits redundant samples. Final predictions combine cache-derived scores with the current CLIP scores. Experiments on BioVid, StressID, and BAH show that EB-CaP outperforms state-of-the-art TTA methods while maintaining low computational and memory overhead. Code is available at https://github.com/MasoumehSharafi/EB-CaP.

Related papers

• Prototype-Based Test-Time Adaptation of Vision-Language Models [58.28899599773308]
  Prototype-Based Test-Time Adaptation (PTA)<n>PTA uses a set of class-specific knowledge prototypes to accumulate knowledge from test samples.<n>PTA achieves extremely high efficiency while achieving state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-23T07:20:56Z)
• Test-Time Adaptation via Cache Personalization for Facial Expression Recognition in Videos [59.83490704563065]
  This paper introduces Cache Personalization (TTA-CaP), a cache-based TTA method that enables cost-effective personalization of vision-language models for video FER.<n>Our experiments show TTA-CaP can outperform state-of-the-art TTA methods under subject-specific and environmental shifts.
  arXiv  Detail & Related papers  (2026-03-22T16:31:25Z)
• Bayesian Test-time Adaptation for Object Recognition and Detection with Vision-language Models [86.53246292425699]
  We present BCA+, a training-free framework for TTA for both object recognition and detection.<n>We formulate adaptation as a Bayesian inference problem, where final predictions are generated by fusing the initial VLM output with a cache-based prediction.<n>BCA+ achieves state-of-the-art performance on both recognition and detection benchmarks.
  arXiv  Detail & Related papers  (2025-10-03T06:27:33Z)
• Multi-Cache Enhanced Prototype Learning for Test-Time Generalization of Vision-Language Models [7.542207462593201]
  In zero-shot setting, test-time adaptation adjusts pre-trained models using unlabeled data from the test phase to enhance performance on unknown test distributions.<n>This study identifies a positive correlation between cache-enhanced performance and intra-class compactness.<n>We propose a Multi-Cache enhanced Prototype-based Test-Time Adaptation (MCP) featuring three caches.
  arXiv  Detail & Related papers  (2025-08-02T06:43:43Z)
• DOTA: Distributional Test-Time Adaptation of Vision-Language Models [69.41389326333771]
  Vision-language foundation models can be unreliable when significant distribution gaps exist between training and test data.<n>We propose DOTA (DistributiOnal Test-time Adaptation), a simple yet effective method addressing this limitation.<n>This distribution-centric approach enables the model to continually learn and adapt to the deployment environment.
  arXiv  Detail & Related papers  (2024-09-28T15:03:28Z)
• Test-Time Model Adaptation with Only Forward Passes [68.11784295706995]
  Test-time adaptation has proven effective in adapting a given trained model to unseen test samples with potential distribution shifts. We propose a test-time Forward-Optimization Adaptation (FOA) method. FOA runs on quantized 8-bit ViT, outperforms gradient-based TENT on full-precision 32-bit ViT, and achieves an up to 24-fold memory reduction on ImageNet-C.
  arXiv  Detail & Related papers  (2024-04-02T05:34:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.