ProfVLM: A Lightweight Video-Language Model for Multi-View Proficiency Estimation

• URL: http://arxiv.org/abs/2509.26278v1
• Date: Tue, 30 Sep 2025 14:00:41 GMT
• Title: ProfVLM: A Lightweight Video-Language Model for Multi-View Proficiency Estimation
• Authors: Edoardo Bianchi, Jacopo Staiano, Antonio Liotta,
• Abstract summary: We present ProfVLM, a compact vision-language model that reformulates this task as generative reasoning.<n>It jointly predicts skill level and generates expert-like feedback from egocentric and exocentric videos.
• Score: 3.115853870709636
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing approaches to skill proficiency estimation often rely on black-box video classifiers, ignoring multi-view context and lacking explainability. We present ProfVLM, a compact vision-language model that reformulates this task as generative reasoning: it jointly predicts skill level and generates expert-like feedback from egocentric and exocentric videos. Central to our method is an AttentiveGatedProjector that dynamically fuses multi-view features, projected from a frozen TimeSformer backbone into a language model tuned for feedback generation. Trained on EgoExo4D with expert commentaries, ProfVLM surpasses state-of-the-art methods while using up to 20x fewer parameters and reducing training time by up to 60%. Our approach not only achieves superior accuracy across diverse activities, but also outputs natural language critiques aligned with performance, offering transparent reasoning. These results highlight generative vision-language modeling as a powerful new direction for skill assessment.

Related papers

• Towards Understanding Multimodal Fine-Tuning: Spatial Features [25.349396112139214]
  Vision-Language Models (VLMs) achieve strong performance on a wide range of tasks by pairing a vision encoder with a pre-trained language model.<n>We present the first mechanistic analysis of VLM adaptation using stage-wise model diffing.
  arXiv  Detail & Related papers  (2026-02-06T18:48:18Z)
• VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning [49.35834435935727]
  VideoZoomer is a novel agentic framework that enables MLLMs to control their visual focus during reasoning.<n>Our 7B model delivers diverse and complex reasoning patterns, yielding strong performance across a broad set of long video understanding and reasoning benchmarks.<n>These emergent capabilities allow it to consistently surpass existing open-source models and even rival proprietary systems on challenging tasks.
  arXiv  Detail & Related papers  (2025-12-26T11:43:21Z)
• Learning Skill-Attributes for Transferable Assessment in Video [56.813876909367856]
  Skill assessment from video entails rating the quality of a person's physical performance and explaining what could be done better.<n>Our CrossTrainer approach discovers skill-attributes, such as balance, control, and hand positioning.<n>By abstracting out the shared behaviors indicative of human skill, the proposed video representation generalizes substantially better than an array of existing techniques.
  arXiv  Detail & Related papers  (2025-11-17T23:53:06Z)
• ExpertAF: Expert Actionable Feedback from Video [81.46431188306397]
  We introduce a novel method to generate actionable feedback from video of a person doing a physical activity, such as basketball or soccer.<n>Our method takes a video demonstration and its accompanying 3D body pose and generates expert commentary describing what the person is doing well and what they could improve.<n>We show how to leverage Ego-Exo4D's [29] videos of skilled activity and expert commentary together with a strong language model to create a weakly-supervised training dataset for this task.
  arXiv  Detail & Related papers  (2024-08-01T16:13:07Z)
• Towards Multi-Task Multi-Modal Models: A Video Generative Perspective [5.495245220300184]
  This thesis chronicles our endeavor to build multi-task models for generating videos and other modalities under diverse conditions. We unveil a novel approach to mapping bidirectionally between visual observation and interpretable lexical terms. Our scalable visual token representation proves beneficial across generation, compression, and understanding tasks.
  arXiv  Detail & Related papers  (2024-05-26T23:56:45Z)
• Veagle: Advancements in Multimodal Representation Learning [0.0]
  This paper introduces a novel approach to enhance the multimodal capabilities of existing models. Our proposed model Veagle, incorporates a unique mechanism inspired by the successes and insights of previous works. Our results indicate a improvement of 5-6 % in performance, with Veagle outperforming existing models by a notable margin.
  arXiv  Detail & Related papers  (2024-01-18T12:45:25Z)
• Video-GroundingDINO: Towards Open-Vocabulary Spatio-Temporal Video Grounding [108.79026216923984]
  Video grounding aims to localize a-temporal section in a video corresponding to an input text query. This paper addresses a critical limitation in current video grounding methodologies by introducing an Open-Vocabulary Spatio-Temporal Video Grounding task.
  arXiv  Detail & Related papers  (2023-12-31T13:53:37Z)
• LifelongMemory: Leveraging LLMs for Answering Queries in Long-form Egocentric Videos [15.127197238628396]
  LifelongMemory is a new framework for accessing long-form egocentric videographic memory through natural language question answering and retrieval. Our approach achieves state-of-the-art performance on the benchmark for question answering and is highly competitive on the natural language query (NLQ) challenge of Ego4D.
  arXiv  Detail & Related papers  (2023-12-07T19:19:25Z)
• Video-Teller: Enhancing Cross-Modal Generation with Fusion and Decoupling [79.49128866877922]
  Video-Teller is a video-language foundation model that leverages multi-modal fusion and fine-grained modality alignment. Video-Teller boosts the training efficiency by utilizing frozen pretrained vision and language modules. It capitalizes on the robust linguistic capabilities of large language models, enabling the generation of both concise and elaborate video descriptions.
  arXiv  Detail & Related papers  (2023-10-08T03:35:27Z)
• Object Relational Graph with Teacher-Recommended Learning for Video Captioning [92.48299156867664]
  We propose a complete video captioning system including both a novel model and an effective training strategy. Specifically, we propose an object relational graph (ORG) based encoder, which captures more detailed interaction features to enrich visual representation. Meanwhile, we design a teacher-recommended learning (TRL) method to make full use of the successful external language model (ELM) to integrate the abundant linguistic knowledge into the caption model.
  arXiv  Detail & Related papers  (2020-02-26T15:34:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.