SkillMoV: Mixture-of-View Routing with Prototype-Conditioned Gating for Unified Multi-View Proficiency Estimation

• URL: http://arxiv.org/abs/2606.17615v1
• Date: Tue, 16 Jun 2026 07:19:20 GMT
• Title: SkillMoV: Mixture-of-View Routing with Prototype-Conditioned Gating for Unified Multi-View Proficiency Estimation
• Abstract summary: We introduce SkillMoV, a unified framework for multi-scenario proficiency estimation from synchronized video.<n>At its core, SkillMoV introduces a Mixture-of-View Projector (MoVP), which adapts the mixture-of-experts to camera-specific view features.<n>We evaluate SkillMoV on EgoExo4D across six skill domains and three separately trained view configurations.
• Score: 1.3893859937118993
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Estimating human proficiency from video is a key challenge for automated skill assessment, with applications in sports coaching, music pedagogy, surgical training, and workplace learning. Existing approaches often focus on individual scenarios or rely on shared multi-view aggregation, limiting their ability to adapt to heterogeneous camera viewpoints and activity domains. We introduce SkillMoV, a unified, parameter-efficient framework for multi-scenario proficiency estimation from synchronized multi-view video. At its core, SkillMoV introduces a Mixture-of-View Projector (MoVP), which adapts the mixture-of-experts paradigm to camera-specific view features. MoVP is composed of four stages: (i) a Mixture-of-View soft router with twelve expert MLPs that learns view-dependent expert preferences without camera-identity supervision; (ii) cross-view attention to align synchronized cameras; (iii) learnable prototype anchoring to condition the representation on class-level reference vectors; and (iv) a prototype-conditioned gated projection that produces the final skill embedding. We evaluate SkillMoV on EgoExo4D across six skill domains and three separately trained view configurations: Ego, Exos, and Ego+Exos. SkillMoV reaches 50.17% overall accuracy in the Exos setting with a single model trained jointly across all scenarios, surpassing the strongest reported Exos result among the compared methods by 3.57 percentage points. In Ego+Exos, SkillMoV remains close to the best reported result in that setting (47.63% versus 48.20%). Ablations on the selected Exos configuration validate each component: MoV routing contributes +6.61 pp over attentive aggregation, cross-view attention +4.92 pp, prototype anchoring +4.07 pp, and stochastic view dropout +3.90 pp. Through LoRA adaptation, SkillMoV trains only 23.32% of its parameters and adds limited measured overhead relative to a LoRA-only baseline.

Related papers

• Multi-Dimensional Quality Assessment for AI-Generated Human-Centric Videos: Dataset and Model [95.02470963378896]
  HVEval+ is the largest holistic quality assessment dataset for AI-generated human-centric videos.<n>MoE-Rater is a Mixture-of-Experts (MoE)-inspired and multimodal large language model (MLLM)-based all-in-one method.
  arXiv  Detail & Related papers  (2026-07-18T10:04:18Z)
• SkillSpotter: Pose-Aware Multi-View Skilled Action Detection and Grading in Ego-Exo Videos [27.95264551649598]
  We introduce SkillSpotter, a pose-aware multi-view architecture that jointly detects and grades skilled actions through three task-specific modules.<n>SkillSpotter improves class-specific mAP from 12.40 to 21.82 (+76%) and balanced accuracy from 55.99% to 60.40% over the best baseline.
  arXiv  Detail & Related papers  (2026-06-30T04:43:03Z)
• UniviewVLA: A Unified Multiview Vision-Language-Action Model with World Modeling [33.6086006552446]
  We propose UniviewVLA, a unified multiview Vision-Language-Action model with world modeling.<n>By leveraging generated multiview future views, UniviewVLA reveals occluded cues and models future scene evolution.<n>UniviewVLA achieves 95.8% on LIBERO and 4.60 on CALVIN ABCD to D, both standard occlusion-free benchmarks.
  arXiv  Detail & Related papers  (2026-06-19T14:53:40Z)
• Mesh-Aware Epipolar Matching for Multi-View Multi-Person 3D Pose Estimation in Basketball [10.727074525703147]
  Mesh-Aware Epipolar Matching (MAEM) is a training-free framework for multi-view multi-person 3D pose estimation.<n> Experiments on two public multi-view basketball datasets demonstrate that MAEM consistently outperforms existing training-free association baselines.
  arXiv  Detail & Related papers  (2026-05-28T13:59:30Z)
• Zero-Shot Object Re-Identification in Egocentric Kitchen Videos via Multi-Stage SAM3 Feature Fusion [59.32099847726022]
  We study zero-shot object ReID on the EPIC-Kitchens benchmark.<n>The goal is to match active food and kitchen-tool instances across frames using only pre-trained visual features.
  arXiv  Detail & Related papers  (2026-05-25T23:01:41Z)
• CalibFree: Self-Supervised View Feature Separation for Calibration-Free Multi-Camera Multi-Object Tracking [50.354685761416484]
  CalibFree is a self-supervised representation learning framework that does not need any calibration or manual labeling for the MCMOT task.<n> Experiments on the MMP-MvMHAT dataset show a 3% improvement in overall accuracy and a 7.5% increase in the average F1 score over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2026-05-10T01:04:12Z)
• SkillFormer: Unified Multi-View Video Understanding for Proficiency Estimation [1.3893859937118993]
  SkillFormer is a parameter-efficient architecture for unified multi-view proficiency estimation.<n>It fuses view-specific features using multi-head cross-attention, learnable gating, and adaptive self-calibration.<n>It achieves state-of-the-art accuracy in multi-view settings while demonstrating remarkable computational efficiency.
  arXiv  Detail & Related papers  (2025-05-13T15:27:24Z)
• CoMP: Continual Multimodal Pre-training for Vision Foundation Models [72.3323674291719]
  We continually pre-train prevailing Vision Foundation Models (VFMs) in a multimodal manner.<n>We introduce CoMP, a carefully designed multimodal pre-training pipeline.<n>Leading VFMs like DINOv2, SigLIP and AIMv2 achieve remarkable improvements in multimodal understanding tasks.
  arXiv  Detail & Related papers  (2025-03-24T17:52:47Z)
• MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts [95.26323548734692]
  MoMa is a modality-aware mixture-of-experts architecture for pre-training mixed-modal, early-fusion language models. Under a 1-trillion-token training budget, the MoMa 1.4B model, featuring 4 text experts and 4 image experts, achieves impressive FLOPs savings.
  arXiv  Detail & Related papers  (2024-07-31T17:46:51Z)
• EgoPoseFormer: A Simple Baseline for Stereo Egocentric 3D Human Pose Estimation [15.590340765703893]
  We present EgoPoseFormer, a transformer-based model for stereo egocentric human pose estimation. Our approach overcomes the main challenge of overcoming joint invisibility caused by self-occlusion or a limited field of view (FOV) of head-mounted cameras. We evaluate our method on the stereo UnrealEgo dataset and show it significantly outperforms previous approaches.
  arXiv  Detail & Related papers  (2024-03-26T20:02:48Z)
• Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching [63.88319217738223]
  We present Matcher, a novel perception paradigm that utilizes off-the-shelf vision foundation models to address various perception tasks. Matcher demonstrates impressive generalization performance across various segmentation tasks, all without training. Our results further showcase the open-world generality and flexibility of Matcher when applied to images in the wild.
  arXiv  Detail & Related papers  (2023-05-22T17:59:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.