Moving Beyond More Views: Redundancy-Aware Ego-Exo Fusion for Proficiency Estimation

• URL: http://arxiv.org/abs/2608.25736v1
• Date: Wed, 26 Aug 2026 12:47:42 GMT
• Title: Moving Beyond More Views: Redundancy-Aware Ego-Exo Fusion for Proficiency Estimation
• Abstract summary: EgoExo estimation aims to integrate fine-grained motion cues from egocentric (1st-person) views with spatial context from multiple exocentric (3rd-person) views.<n>Our analysis identifies two key causes: (1) Multiview redundancy - From the data perspective, certain views provide limited or noisy information, diluting discriminative cues; (2) Overfitting - From the feature perspective, conventional fusion increases representational complexity, causing the model to memorise view-specific patterns rather than learn generalisable representations.
• Score: 12.151605559533337
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: EgoExo proficiency estimation aims to assess action quality by integrating fine-grained motion cues from egocentric (1st-person) views with spatial context from multiple exocentric (3rd-person) views. Simply adding more exocentric views degrades EgoExo performance, as redundant or noisy perspectives dilute useful motion cues. Our analysis identifies two key causes: (1) Multiview redundancy - From the data perspective, certain views provide limited or noisy information, diluting discriminative cues; (2) Overfitting - From the feature perspective, conventional fusion increases representational complexity, causing the model to memorise view-specific patterns rather than learn generalisable representations. To address these issues, we propose two complementary modules: AdaMVS, which adaptively identifies and fuses the most informative view tokens under weak supervision from the data perspective, and VIB-GB, which combines Gradient Blending and Variational Information Bottleneck regularisation from the feature perspective to compress redundant signals and suppress overfitting during training. Experiments on EgoExo-4D and EgoExo-Fitness demonstrate that our method learns both which view to look at and how to fuse them, achieving new state-of-the-art results. Our source code is available at https://github.com/dx199771/AdaMVS

Related papers

• VIEW2SPACE: Studying Multi-View Visual Reasoning from Sparse Observations [47.94531550391802]
  Multi-view visual reasoning is essential for intelligent systems that must understand complex environments from sparse and discrete viewpoints.<n>In real-world scenarios, reasoning across views requires integrating partial observations without explicit guidance.<n>We leverage physically grounded simulation to construct diverse, high-fidelity 3D scenes with precise per-view metadata.
  arXiv  Detail & Related papers  (2026-03-17T13:36:30Z)
• Robust Egocentric Referring Video Object Segmentation via Dual-Modal Causal Intervention [58.05340906967343]
  Egocentric Referring Video Object (Ego-RVOS) aims to segment the specific object actively involved in a human action, as described by a language query, within first-person videos.<n>Existing methods often struggle, learning spurious correlations from skewed object-action pairings in datasets.<n>We introduce Causal-REferring (CERES), a plug-in causal framework that adapts strong, pre-trained RVOSs to the egocentric domain.
  arXiv  Detail & Related papers  (2025-12-30T16:22:14Z)
• EgoExo-Con: Exploring View-Invariant Video Temporal Understanding [66.25513481642845]
  Can Video-LLMs achieve consistent temporal understanding when videos capture the same event from different viewpoints?<n>EgoExo-Con (Consistency) is a benchmark of comprehensively synchronized egocentric and exocentric video pairs with human-refined queries in natural language.<n>We propose View-GRPO, a novel reinforcement learning framework that effectively strengthens view-specific temporal reasoning.
  arXiv  Detail & Related papers  (2025-10-30T03:53:22Z)
• EgoWorld: Translating Exocentric View to Egocentric View using Rich Exocentric Observations [4.252119151012245]
  We introduce EgoWorld, a novel framework that reconstructs an egocentric view from rich exocentric observations.<n>Our approach reconstructs a point cloud from estimated exocentric depth maps, reprojects it into the egocentric perspective, and then applies diffusion-based inpainting to produce dense, semantically coherent egocentric images.<n>EgoWorld achieves state-of-the-art performance and demonstrates robust generalization to new objects, actions, scenes, and subjects.
  arXiv  Detail & Related papers  (2025-06-22T04:21:48Z)
• Bootstrap Your Own Views: Masked Ego-Exo Modeling for Fine-grained View-invariant Video Representations [47.04855334955006]
  We propose a novel masked ego-exo modeling that promotes both causal temporal dynamics and cross-view alignment.<n>We highlight the importance of capturing the compositional nature of human actions as a basis for robust cross-view understanding.
  arXiv  Detail & Related papers  (2025-03-25T14:33:32Z)
• ObjectRelator: Enabling Cross-View Object Relation Understanding Across Ego-Centric and Exo-Centric Perspectives [109.11714588441511]
  The Ego-Exo object correspondence task aims to understand object relations across ego-exo perspectives through segmentation.<n> PSALM, a recently proposed segmentation method, stands out as a notable exception with its demonstrated zero-shot ability on this task.<n>We propose ObjectRelator, a novel approach featuring two key modules: Multimodal Condition Fusion and SSL-based Cross-View Object Alignment.
  arXiv  Detail & Related papers  (2024-11-28T12:01:03Z)
• Ego3DPose: Capturing 3D Cues from Binocular Egocentric Views [9.476008200056082]
  Ego3DPose is a highly accurate binocular egocentric 3D pose reconstruction system. We propose a two-path network architecture with a path that estimates pose per limb independently with its binocular heatmaps. We propose a new perspective-aware representation using trigonometry, enabling the network to estimate the 3D orientation of limbs.
  arXiv  Detail & Related papers  (2023-09-21T10:34:35Z)
• Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignment [71.16699226211504]
  We propose to learn fine-grained action features that are invariant to the viewpoints by aligning egocentric and exocentric videos in time. To this end, we propose AE2, a self-supervised embedding approach with two key designs. For evaluation, we establish a benchmark for fine-grained video understanding in the ego-exo context.
  arXiv  Detail & Related papers  (2023-06-08T19:54:08Z)
• Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos [92.38049744463149]
  We introduce an approach for pre-training egocentric video models using large-scale third-person video datasets. Our idea is to discover latent signals in third-person video that are predictive of key egocentric-specific properties. Our experiments show that our Ego-Exo framework can be seamlessly integrated into standard video models.
  arXiv  Detail & Related papers  (2021-04-16T06:10:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.