Decision-Aware Uncertainty Evaluation of Vision-Language Model-Based Early Action Anticipation for Human-Robot Interaction

• URL: http://arxiv.org/abs/2603.10061v2
• Date: Thu, 12 Mar 2026 01:00:28 GMT
• Title: Decision-Aware Uncertainty Evaluation of Vision-Language Model-Based Early Action Anticipation for Human-Robot Interaction
• Abstract summary: Overconfident early predictions can lead to unsafe or disruptive interaction.<n>Recent vision-language model-based approaches have been proposed for short-term action recognition.<n>We present the first systematic evaluation of uncertainty in vision-language model-based short-term action recognition.
• Score: 5.479569303819538
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robots in shared workspaces must interpret human actions from partial, ambiguous observations, where overconfident early predictions can lead to unsafe or disruptive interaction. This challenge is amplified in egocentric views, where viewpoint changes and occlusions increase perceptual noise and ambiguity. As a result, downstream human-robot interaction modules require not only an action hypothesis but also a trustworthy estimate of confidence under partial observation. Recent vision-language model-based approaches have been proposed for short-term action recognition due to their open-vocabulary and context-aware reasoning, but their uncertainty reliability in the temporal-prefix regime is largely uncharacterized. We present the first systematic evaluation of uncertainty in vision-language model-based short-term action recognition for human-robot interaction. We introduce a temporal-prefix evaluation protocol and metrics for calibration and selective prediction. We also characterize miscalibration patterns and failure modes under partial observations. Our study provides the missing reliability evidence needed to use vision-language model predictions in confidence-gated human-robot interaction modules.

Related papers

• Aligning LLM Uncertainty with Human Disagreement in Subjectivity Analysis [57.78407973423517]
  We advocate uncertainty-aware subjectivity analysis, where models are expected to make predictions while expressing uncertainty that reflects human disagreement.<n>To operationalize this perspective, we propose a two-phase Disagreement Perception and Uncertainty Alignment framework.<n> Experiments on three subjectivity analysis tasks show that DPUA preserves task performance while better aligning model uncertainty with human disagreement.
  arXiv  Detail & Related papers  (2026-05-11T11:52:58Z)
• Rethinking Video Human-Object Interaction: Set Prediction over Time for Unified Detection and Anticipation [18.693454975393703]
  Video-based human-object interaction (HOI) understanding requires both detecting ongoing interactions and anticipating their future evolution.<n>We introduce DETAnt-HOI and HOI-DA, a pair-centric framework that jointly performs subject-object localization, present HOI detection, and future anticipation.<n>Experiments show consistent improvements in both detection and anticipation, with larger gains at longer horizons.
  arXiv  Detail & Related papers  (2026-04-12T01:07:43Z)
• Joint Prediction of Human Motions and Actions in Human-Robot Collaboration [81.20389836121502]
  We introduce textsfMA-HERP, a hierarchical framework for the emphjoint estimation and prediction of human movements and actions.<n>We present a preliminary experimental evaluation based on musculoskeletal simulations of reaching movements, showing accurate motion prediction and robust action inference under noise.
  arXiv  Detail & Related papers  (2026-04-03T14:40:07Z)
• SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models [86.19617358080016]
  Social Omni is a benchmark that operationalizes the evaluation of conversational interactivity across three core dimensions.<n>Social Omni features 2,000 perception samples and a quality-controlled diagnostic set of 209 interaction-generation instances.<n>Our analysis reveals a pronounced decoupling between a model's perceptual accuracy and its ability to generate contextually appropriate interruptions.
  arXiv  Detail & Related papers  (2026-03-17T17:58:44Z)
• Before the Outrage: Challenges and Advances in Predicting Online Antisocial Behavior [1.223779595809275]
  Antisocial behavior on social media poses growing challenges for platform safety and societal wellbeing.<n>Prior work has primarily focused on detecting harmful content after it appears, but predictive approaches aim to forecast future harmful behaviors.<n>Despite increasing interest, the field remains fragmented, lacking a unified taxonomy or clear synthesis of existing methods.
  arXiv  Detail & Related papers  (2025-07-28T08:27:58Z)
• Reasoner Outperforms: Generative Stance Detection with Rationalization for Social Media [12.479554210753664]
  This study adopts a generative approach, where stance predictions include explicit, interpretable rationales.<n>We find that incorporating reasoning into stance detection enables the smaller model (FlanT5) to outperform GPT-3.5's zero-shot performance.
  arXiv  Detail & Related papers  (2024-12-13T16:34:39Z)
• UAHOI: Uncertainty-aware Robust Interaction Learning for HOI Detection [18.25576487115016]
  This paper focuses on Human-Object Interaction (HOI) detection. It addresses the challenge of identifying and understanding the interactions between humans and objects within a given image or video frame. We propose a novel approach textscUAHOI, Uncertainty-aware Robust Human-Object Interaction Learning.
  arXiv  Detail & Related papers  (2024-08-14T10:06:39Z)
• Toward Reliable Human Pose Forecasting with Uncertainty [51.628234388046195]
  We develop an open-source library for human pose forecasting, including multiple models, supporting several datasets. We devise two types of uncertainty in the problem to increase performance and convey better trust.
  arXiv  Detail & Related papers  (2023-04-13T17:56:08Z)
• Empirical Estimates on Hand Manipulation are Recoverable: A Step Towards Individualized and Explainable Robotic Support in Everyday Activities [80.37857025201036]
  Key challenge for robotic systems is to figure out the behavior of another agent. Processing correct inferences is especially challenging when (confounding) factors are not controlled experimentally. We propose equipping robots with the necessary tools to conduct observational studies on people.
  arXiv  Detail & Related papers  (2022-01-27T22:15:56Z)
• Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations [97.91630330328815]
  We conduct a crowdsourcing study, where participants interact with deception detection models that have been trained to distinguish between genuine and fake hotel reviews. We observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
  arXiv  Detail & Related papers  (2021-12-17T18:29:56Z)
• Probabilistic Human Motion Prediction via A Bayesian Neural Network [71.16277790708529]
  We propose a probabilistic model for human motion prediction in this paper. Our model could generate several future motions when given an observed motion sequence. We extensively validate our approach on a large scale benchmark dataset Human3.6m.
  arXiv  Detail & Related papers  (2021-07-14T09:05:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.