EgoAdapt: A Multi-Scene Egocentric Adaptation Method for CVPR 2026 HD-EPIC VQA Challenge

• URL: http://arxiv.org/abs/2605.24500v1
• Date: Sat, 23 May 2026 10:16:51 GMT
• Title: EgoAdapt: A Multi-Scene Egocentric Adaptation Method for CVPR 2026 HD-EPIC VQA Challenge
• Abstract summary: HD-EPIC evaluates whether a vision-language model can reason over realistic first-person kitchen videos.<n> benchmark contains 26K multiple-choice questions across seven macro-categories.<n>EgoAdapt introduces three inference-time components: (1) category-conditioned routing with per-category prompts, frame budgets, and sampling rates; (2) option scoring that evaluates all candidate answers with letter-token likelihoods and generation agreement instead of relying on direct generation; and (3) test-time consistency adaptation that aggregates predictions across option permutations and verification-style prompts for ambiguous cases.
• Score: 69.56534058291463
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This technical report presents our solution, EgoAdapt (Egocentric Adaptation via Category, Calibration, and Consistency), to the CVPR 2026 HD-EPIC VQA challenge. HD-EPIC evaluates whether a vision-language model can reason over realistic first-person kitchen videos, where the evidence for an answer may be a short hand-object interaction, a long recipe trajectory, a spatial relation to a fixture, or a subtle gaze cue. The benchmark contains 26K multiple-choice questions across seven macro-categories: recipe, ingredient, nutrition, fine-grained action, 3D perception, object motion, and gaze. We observe that the main difficulty is not only model capacity, but also the mismatch between a single generic inference recipe and the heterogeneous temporal, spatial, and semantic structure of the benchmark. Our method, EgoAdapt, introduces three inference-time components: (1) category-conditioned routing with per-category prompts, frame budgets, and sampling rates; (2) calibrated option scoring that evaluates all candidate answers with letter-token likelihoods and generation agreement instead of relying only on direct generation; and (3) test-time consistency adaptation that aggregates predictions across option permutations and verification-style prompts for ambiguous cases. This design substantially improves over the available HD-EPIC baselines.

Related papers

• Counterfactual Contrastive Analysis [6.190850256222902]
  We propose a classifier-free approach for visual counterfactual generation based on Contrastive Analysis (CA)<n>We disentangle the generative factors that are common across the two datasets from those that are salient to each dataset, and generate counterfactual images by swapping only the salient factors.<n>We evaluate our method on three medical imaging datasets and demonstrate superior counterfactual generation quality compared to existing approaches.
  arXiv  Detail & Related papers  (2026-08-19T15:25:21Z)
• Not All Attention Is Equal: A Quantitative Survey of the EEI Trade-off [8.068122032539332]
  This survey covers four connected threads: their formulation for sequence-to-sequence tasks, adaptation to computer vision, efficiency innovations that address the quadratic bottleneck, and advances in interpretability.<n>We define three criteria: efficiency, expressiveness, and interpretability, and compare twenty-one methods using an EEI scoring framework.<n>We conclude by framing attention research as an expansion of the efficiency-expressiveness-interpretability frontier and identifying future directions.
  arXiv  Detail & Related papers  (2026-08-16T00:37:38Z)
• MQAdapter: Multi-Modal Quantum Adapter for Coarse-to-Fine VLM Fine-tuning [14.855850083028988]
  We propose a novel coarse-to-fine VLM fine-tuning approach for few-shot learning that leverages quantum computation.<n>MQAdapter first retrieves the Top-K category candidates most similar to the input image and uses them as semantic anchors.<n>It then employs a cross-modal quantum learning mechanism to refine visual features under the guidance of these anchors.
  arXiv  Detail & Related papers  (2026-07-14T06:46:24Z)
• CMAG: Concept-Scaffolded Retrieval for Marketplace Avatar Generation [0.5131152350448099]
  Metaverse platforms rely on creator-driven marketplaces where avatars are assembled from discrete, taxonomy-labeled 3D assets.<n>We propose textbfCMAG, a concept-scaffolded retrieval and verified composition framework for marketplace generation.
  arXiv  Detail & Related papers  (2026-05-18T17:21:43Z)
• LoViF 2026 The First Challenge on Holistic Quality Assessment for 4D World Model (PhyScore) [130.91833247554004]
  The LoViF 2026 PhyScore challenge is a competition on holistic quality assessment of world-model-generated videos.<n>The benchmark dataset contains 1,554 videos generated by seven representative world generative models.<n>This report summarizes the challenge design and provides method-level insights from submitted solutions.
  arXiv  Detail & Related papers  (2026-05-06T17:52:39Z)
• Continual Action Quality Assessment via Adaptive Manifold-Aligned Graph Regularization [53.82400605816587]
  Action Quality Assessment (AQA) quantifies human actions in videos, supporting applications in sports scoring, rehabilitation, and skill evaluation.<n>A major challenge lies in the non-stationary nature of quality distributions in real-world scenarios.<n>We introduce Continual AQA (CAQA), which equips with Continual Learning capabilities to handle evolving distributions.
  arXiv  Detail & Related papers  (2025-10-08T10:09:47Z)
• HERO-VQL: Hierarchical, Egocentric and Robust Visual Query Localization [13.581537394737923]
  We introduce HERO-VQL, a novel method inspired by human cognitive process in object recognition.<n>Top-down Attention Guidance refines the attention mechanism by leveraging the class token for high-level context.<n>EgoAug enhances query diversity by replacing the query with a randomly selected corresponding object from groundtruth annotations.
  arXiv  Detail & Related papers  (2025-08-30T06:50:49Z)
• Towards Variable and Coordinated Holistic Co-Speech Motion Generation [21.244719987338243]
  This paper addresses the problem of generating lifelike holistic co-speech motions for 3D avatars. We aim to achieve both with ProbTalk, a unified probabilistic framework designed to jointly model facial, hand, and body movements in speech.
  arXiv  Detail & Related papers  (2024-03-30T13:41:57Z)
• Hybrid Relation Guided Set Matching for Few-shot Action Recognition [51.3308583226322]
  We propose a novel Hybrid Relation guided Set Matching (HyRSM) approach that incorporates two key components. The purpose of the hybrid relation module is to learn task-specific embeddings by fully exploiting associated relations within and cross videos in an episode. We evaluate HyRSM on six challenging benchmarks, and the experimental results show its superiority over the state-of-the-art methods by a convincing margin.
  arXiv  Detail & Related papers  (2022-04-28T11:43:41Z)
• Egocentric Action Recognition by Video Attention and Temporal Context [83.57475598382146]
  We present the submission of Samsung AI Centre Cambridge to the CVPR 2020 EPIC-Kitchens Action Recognition Challenge. In this challenge, action recognition is posed as the problem of simultaneously predicting a single verb' and noun' class label given an input trimmed video clip. Our solution achieves strong performance on the challenge metrics without using object-specific reasoning nor extra training data.
  arXiv  Detail & Related papers  (2020-07-03T18:00:32Z)
• C3VQG: Category Consistent Cyclic Visual Question Generation [51.339348810676896]
  Visual Question Generation (VQG) is the task of generating natural questions based on an image. In this paper, we try to exploit the different visual cues and concepts in an image to generate questions using a variational autoencoder (VAE) without ground-truth answers. Our approach solves two major shortcomings of existing VQG systems: (i) minimize the level of supervision and (ii) replace generic questions with category relevant generations.
  arXiv  Detail & Related papers  (2020-05-15T20:25:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.