Joint Prediction of Human Motions and Actions in Human-Robot Collaboration

• URL: http://arxiv.org/abs/2604.03065v1
• Date: Fri, 03 Apr 2026 14:40:07 GMT
• Title: Joint Prediction of Human Motions and Actions in Human-Robot Collaboration
• Abstract summary: We introduce textsfMA-HERP, a hierarchical framework for the emphjoint estimation and prediction of human movements and actions.<n>We present a preliminary experimental evaluation based on musculoskeletal simulations of reaching movements, showing accurate motion prediction and robust action inference under noise.
• Score: 81.20389836121502
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fluent human--robot collaboration requires robots to continuously estimate human behaviour and anticipate future intentions. This entails reasoning jointly about \emph{continuous movements} and \emph{discrete actions}, which are still largely modelled in isolation. In this paper, we introduce \textsf{MA-HERP}, a hierarchical and recursive probabilistic framework for the \emph{joint estimation and prediction} of human movements and actions. The model combines: (i) a hierarchical representation in which movements compose into actions through admissible Allen interval relations, (ii) a unified probabilistic factorisation coupling continuous dynamics, discrete labels, and durations, and (iii) a recursive inference scheme inspired by Bayesian filtering, alternating top-down action prediction with bottom-up sensory evidence. We present a preliminary experimental evaluation based on neural models trained on musculoskeletal simulations of reaching movements, showing accurate motion prediction, robust action inference under noise, and computational performance compatible with on-line human--robot collaboration.

Related papers

• Pose-Anchored Optical Flow for Low-Latency Human Action Anticipation in Human-Robot Teaming [0.4282159812965447]
  PoseOFF is a pose-anchored optical flow representation that captures local motion information around human joints.<n>By conditioning motion feature extraction on human pose, PoseOFF encodes localised motion dynamics at semantically meaningful body locations.<n>Our results show that PoseOFF enables models to achieve comparable or improved performance while observing less of the action sequence.
  arXiv  Detail & Related papers  (2026-08-26T08:07:19Z)
• From Brain Models to Executable Digital Twins: Execution Semantics and Neuro-Neuromorphic Systems [51.82266520875928]
  This survey introduces physically constrained executability as a unifying perspective for comparing approaches at the level of execution.<n>I propose a taxonomy of execution regimes ranging from isolated offline models to coordinated co-simulation, to continuously executing digital twins sustained by online data.<n>The executability concept clarifies why accuracy alone is insufficient, and motivates an agenda centered on semantic interoperability, hybrid-time correctness, evaluation protocols, scalable assimilation, and safe closed-loop validation.
  arXiv  Detail & Related papers  (2026-04-15T07:34:46Z)
• HUMOF: Human Motion Forecasting in Interactive Social Scenes [29.621970821619424]
  Complex scenes present significant challenges for predicting human behaviour due to the abundance of interaction information.<n>We propose an effective method for human motion forecasting in interactive scenes.<n>Our method achieves state-of-the-art performance across four public datasets.
  arXiv  Detail & Related papers  (2025-06-04T09:21:54Z)
• Continual Learning from Simulated Interactions via Multitask Prospective Rehearsal for Bionic Limb Behavior Modeling [0.7922558880545526]
  We introduce a model for human behavior in the context of bionic prosthesis control.<n>We propose a multitasking, continually adaptive model that anticipates and refines movements over time.<n>We validate our model through experiments on real-world human gait datasets, including transtibial amputees.
  arXiv  Detail & Related papers  (2024-05-02T09:22:54Z)
• Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation [50.01551945190676]
  Social robot navigation can be helpful in various contexts of daily life but requires safe human-robot interactions and efficient trajectory planning. We propose a systematic relational reasoning approach with explicit inference of the underlying dynamically evolving relational structures. We demonstrate its effectiveness for multi-agent trajectory prediction and social robot navigation.
  arXiv  Detail & Related papers  (2024-01-22T18:58:22Z)
• Enhanced Human-Robot Collaboration using Constrained Probabilistic Human-Motion Prediction [5.501477817904299]
  We propose a novel human motion prediction framework that incorporates human joint constraints and scene constraints. It is tested on a human arm kinematic model and implemented on a human-robot collaborative setup with a UR5 robot arm.
  arXiv  Detail & Related papers  (2023-10-05T05:12:14Z)
• Dyadic Human Motion Prediction [119.3376964777803]
  We introduce a motion prediction framework that explicitly reasons about the interactions of two observed subjects. Specifically, we achieve this by introducing a pairwise attention mechanism that models the mutual dependencies in the motion history of the two subjects. This allows us to preserve the long-term motion dynamics in a more realistic way and more robustly predict unusual and fast-paced movements.
  arXiv  Detail & Related papers  (2021-12-01T10:30:40Z)
• Non-local Graph Convolutional Network for joint Activity Recognition and Motion Prediction [2.580765958706854]
  3D skeleton-based motion prediction and activity recognition are two interwoven tasks in human behaviour analysis. We propose a new way to combine the advantages of both graph convolutional neural networks and recurrent neural networks for joint human motion prediction and activity recognition.
  arXiv  Detail & Related papers  (2021-08-03T14:07:10Z)
• RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting [34.54878390622877]
  We propose a generic motion forecasting framework with dynamic key information selection and ranking based on a hybrid attention mechanism. The framework is instantiated to handle multi-agent trajectory prediction and human motion forecasting tasks. We validate the framework on both synthetic simulations and motion forecasting benchmarks in different domains.
  arXiv  Detail & Related papers  (2021-08-03T06:30:30Z)
• Probabilistic Human Motion Prediction via A Bayesian Neural Network [71.16277790708529]
  We propose a probabilistic model for human motion prediction in this paper. Our model could generate several future motions when given an observed motion sequence. We extensively validate our approach on a large scale benchmark dataset Human3.6m.
  arXiv  Detail & Related papers  (2021-07-14T09:05:33Z)
• Learning to Abstract and Predict Human Actions [60.85905430007731]
  We model the hierarchical structure of human activities in videos and demonstrate the power of such structure in action prediction. We propose Hierarchical-Refresher-Anticipator, a multi-level neural machine that can learn the structure of human activities by observing a partial hierarchy of events and roll-out such structure into a future prediction in multiple levels of abstraction.
  arXiv  Detail & Related papers  (2020-08-20T23:57:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.