TTSD-FAR: Test-Time Self-Distillation with Fisher-Anchored Restoration for Missing-Modality Emotion Recognition in LVLMs

• URL: http://arxiv.org/abs/2608.18386v1
• Date: Tue, 18 Aug 2026 23:40:28 GMT
• Title: TTSD-FAR: Test-Time Self-Distillation with Fisher-Anchored Restoration for Missing-Modality Emotion Recognition in LVLMs
• Abstract summary: Large video-language models (LVLMs) have shown remarkable performance on multimodal tasks like multimodal emotion recognition (ER) in the wild.<n>However, real-world deployment remains challenging: modalities may be missing or noisy at test time.<n>We propose a solution in which a frozen teacher guides an adaptive low-rank student via self-distillation.<n>Experiments on MELD, DFEW, and BAH under 0%-50% missing modalities show that this unified adaptation-restoration design consistently outperforms entropy-based adaptation, RAG, and perplexity-based generation.
• Score: 83.78708832901194
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large video-language models (LVLMs) have shown remarkable performance on multimodal tasks like multimodal emotion recognition (ER) in the wild. ER is inherently multimodal, requiring a joint understanding of facial expressions, vocalizations, language, biosignals, and gestures. However, real-world deployment remains challenging: modalities may be missing or noisy at test time. Partial observations can be viewed as a distribution shift relative to the complete-modality distribution. SOTA TTA methods based on entropy minimization or perplexity reduction do not transfer to autoregressive LVLMs, while retrieval augmented generation (RAG) degrades when the observed modality is weak. Because no ground-truth supervision exists to verify individual updates, adaptation across this stream risks accumulating drift and degrading once the model departs from a reliable solution. An effective solution must therefore adapt to arbitrary missing-modality patterns and remain effective during continual adaptation. We address both jointly with Test-Time Self-Distillation (TTSD), a parameter-efficient framework in which a frozen teacher, trained on complete modalities, guides an adaptive low-rank student via self-distillation, updating only a negligible number of parameters. Stability is built into this same loop through Fisher-Anchored Restoration (FAR), which monitors Fisher information stability to detect convergence versus drift and restores the student toward the teacher's anchor when distributional shifts are identified. Our experiments on MELD, DFEW, and BAH under 0%-50% missing modalities show that this unified adaptation-restoration design consistently outperforms entropy-based adaptation, RAG, and perplexity-based generation over long adaptation horizons, where baselines without restoration progressively degrade while TTSD-FAR remains consistent.

Related papers

• RDDMPI: Residual Denoising Diffusion Model for Probabilistic Multivariate Time Series Imputation [2.589687124983865]
  Multi-time series imputation aims to recover missing values in temporal data composed of multiple variables.<n>This problem is central to real-world applications such as healthcare monitoring, traffic networks, and energy systems.<n>Recent diffusion-based approaches have shown strong potential for probabilistic imputation by learning to generate missing values through iterative denoising.
  arXiv  Detail & Related papers  (2026-09-10T14:53:38Z)
• ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning [74.19730275669227]
  ARISE-RL is a novel full-cycle self-evolution framework for training open-ended agents.<n>We present Reward-Gated Self-Evolution Distillation (RG-SED), which distills a memory-augmented variant of the same policy back into itself.<n>We also present ECR-Bench, an expert-calibrated rubric benchmark suite covering single-tool deep research and multi-tool travel planning.
  arXiv  Detail & Related papers  (2026-09-01T10:54:13Z)
• Local Margin Restoration for Test-Time Adaptation of Vision-Language Models [15.097704252897818]
  Vision-language models (VLMs) such as CLIP exhibit remarkable zero-shot capabilities, yet their performance frequently degrades sharply under unexpected test-time distribution shifts.<n>We propose Local Margin Restoration (LMR), a lightweight, one-step Test-Time Adaptation framework.<n>Experiments on CIFAR-C, ImageNet-C, and ImageNet variants demonstrate that LMR consistently outperforms state-of-the-art TTA baselines.
  arXiv  Detail & Related papers  (2026-08-03T13:38:11Z)
• Where to Refine, When to Stop: Rethinking Redundancy via Latent Discrepancy for Efficient Visual Autoregressive Generation [50.139984798361375]
  We propose a training-free framework that removes redundancy via latent discrepancy.<n> LD-Pruning substantially reduces inference while maintaining high generation quality, achieving up to 2.35x speedup on Infinity-8B.
  arXiv  Detail & Related papers  (2026-05-29T19:34:39Z)
• MoASE++: Mixture of Activation Sparsity Experts with Domain-Adaptive On-policy Distillation for Continual Test Time Adaptation [65.068801413044]
  Continual test-time adaptation adapts a source-pretrained model to non-stationary, unlabeled target streams.<n>We introduce MoASE, a mixture-of-experts that disentangles domain-agnostic structure from domain-specific texture.
  arXiv  Detail & Related papers  (2026-05-18T01:52:12Z)
• Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillation [71.16738415436458]
  We introduce Reflection-Enhanced Self-Distillation (RESD), a framework that transforms raw failure feedback into an active source of corrective supervision.<n>RESD interprets failed trajectories by generating retrospective reflections to diagnose local errors, and curates a persistent global playbook.<n> Empirical evaluations on multiple continual learning tasks demonstrate that RESD substantially outperforms standard self-distillation baselines.
  arXiv  Detail & Related papers  (2026-05-12T20:46:05Z)
• Adaptive recurrent flow map operator learning for reaction diffusion dynamics [0.9137554315375919]
  We develop an operator learner with adaptive recurrent training (DDOL-ART) using a robust recurrent strategy with lightweight validation milestones.<n>DDOL-ART learns one-step operators that remain stable under long rollouts and generalize zero-shot to strong shifts.<n>It is several-fold faster than a physics-based numerical-loss operator learner (NLOL) under matched settings.
  arXiv  Detail & Related papers  (2026-02-10T07:33:13Z)
• Analyzing and Improving Diffusion Models for Time-Series Data Imputation: A Proximal Recursion Perspective [45.713195454899875]
  Diffusion models (DMs) have shown promise for Time-Series Data Imputation.<n>DMs' performance remains inconsistent in complex scenarios.<n>We propose a novel framework called SPIRIT (Semi-Proximal Transport Regularized time-series Imputation)
  arXiv  Detail & Related papers  (2026-02-01T12:11:57Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• Holistic Continual Learning under Concept Drift with Adaptive Memory Realignment [6.0897744845912865]
  We introduce a holistic framework for continual learning under concept drift.<n>We propose Adaptive Memory Realignment (AMR), a lightweight alternative that equips rehearsal-based learners with a drift-aware adaptation mechanism.<n>AMR consistently counters concept drift, maintaining high accuracy with minimal overhead.
  arXiv  Detail & Related papers  (2025-07-03T04:41:20Z)
• Restoration Score Distillation: From Corrupted Diffusion Pretraining to One-Step High-Quality Generation [82.39763984380625]
  We propose textitRestoration Score Distillation (RSD), a principled generalization of Denoising Score Distillation (DSD)<n>RSD accommodates a broader range of corruption types, such as blurred, incomplete, or low-resolution images.<n>It consistently surpasses its teacher model across diverse restoration tasks on both natural and scientific datasets.
  arXiv  Detail & Related papers  (2025-05-19T17:21:03Z)
• Uncertainty-Aware Deep Attention Recurrent Neural Network for Heterogeneous Time Series Imputation [0.25112747242081457]
  Missingness is ubiquitous in multivariate time series and poses an obstacle to reliable downstream analysis. We propose DEep Attention Recurrent Imputation (Imputation), which jointly estimates missing values and their associated uncertainty. Experiments show that I surpasses the SOTA in diverse imputation tasks using real-world datasets.
  arXiv  Detail & Related papers  (2024-01-04T13:21:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.