Emotion-Conditioned Short-Horizon Human Pose Forecasting with a Lightweight Predictive World Model

• URL: http://arxiv.org/abs/2604.23532v1
• Date: Sun, 26 Apr 2026 04:56:45 GMT
• Title: Emotion-Conditioned Short-Horizon Human Pose Forecasting with a Lightweight Predictive World Model
• Abstract summary: Short-term human pose prediction plays a crucial role in interactive systems, assistive robots, and emotion-aware human-computer interaction.<n>This paper investigates whether facial expression-derived emotion embeddings can provide auxiliary conditional signals for short-term pose prediction.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Short-term human pose prediction plays a crucial role in interactive systems, assistive robots, and emotion-aware human-computer interaction[1-3]. While current trajectory prediction models primarily rely on geometric motion cues, they often overlook the underlying emotional signals influencing human motion dynamics[4-5]. This paper investigates whether facial expression-derived emotion embeddings can provide auxiliary conditional signals for short-term pose prediction. To further evaluate multimodal conditionation in a recursive prediction setting, we propose a lightweight autoregressive predictive world model that performs 15-step rolling pose prediction. This framework combines pose keypoints with emotion embeddings through a learnable gating mechanism and performs autoregressive unfolding prediction using a recurrent sequence model based on a two-layer LSTM architecture. Experiments were conducted on two small-scale pose-emotion video datasets: controlled motion sequences with minimal facial expression changes and, natural emotion-driven motion sequences with considerable facial expression changes. The results show that simple multimodal fusion does not consistently improve prediction accuracy, while normalized gating fusion significantly enhances the performance of emotion-driven motion sequences. Furthermore, counterfactual perturbation experiments demonstrate that the predicted trajectory exhibits measurable sensitivity to changes in multimodal input, suggesting that facial expression embeddings act as auxiliary conditional signals rather than redundant features. In summary, these results indicate that incorporating facial expression-derived emotion embeddings into emotion-conditional short-term pose prediction based on a lightweight predictive world model architecture is a feasible approach.

Related papers

• Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving [78.83165764484562]
  Drive-HWM is a hierarchical slow--fast world modeling framework that organizes future representation prediction and action generation at complementary temporal scales.<n>Next-frame prediction encourages the fast model to capture imminent scene evolution, while one-step action generation allows decisions to be continuously updated as new observations arrive.
  arXiv  Detail & Related papers  (2026-09-03T09:21:19Z)
• Pose-Anchored Optical Flow for Low-Latency Human Action Anticipation in Human-Robot Teaming [0.4282159812965447]
  PoseOFF is a pose-anchored optical flow representation that captures local motion information around human joints.<n>By conditioning motion feature extraction on human pose, PoseOFF encodes localised motion dynamics at semantically meaningful body locations.<n>Our results show that PoseOFF enables models to achieve comparable or improved performance while observing less of the action sequence.
  arXiv  Detail & Related papers  (2026-08-26T08:07:19Z)
• Learning When to Listen: Gated Affect Fusion for Human Motion Prediction [0.0]
  We present a systematic study investigating the utility and temporal limitations of affect-conditioned forecasting in-the-wild.<n>We introduce the Gated Affect Transformer (GAT) to dynamically regulate cross-modal information flow.<n>Our findings provide empirical evidence that facial affect should be regarded as a complementary behavioral cue.
  arXiv  Detail & Related papers  (2026-07-01T00:47:02Z)
• AffectVerse: Emotional World Models for Multimodal Affective Computing [56.144242722718985]
  We propose AffectVerse, an action-free representation-level module for short-horizon latent affective prediction.<n>EWM contains three modules: 1) Cross-Modal Temporal Imagination predicts future video/audio representations from past tokens with multi-step rollout.<n>EWM compresses imagined tokens into modality-aware belief tokens.<n>These results suggest predictive belief-state modeling is a practical alternative for affective computing.
  arXiv  Detail & Related papers  (2026-05-19T15:05:00Z)
• Ego-centric Predictive Model Conditioned on Hand Trajectories [52.531681772560724]
  In egocentric scenarios, anticipating both the next action and its visual outcome is essential for understanding human-object interactions.<n>We propose a unified two-stage predictive framework that jointly models action and visual future in egocentric scenarios.<n>Our approach is the first unified model designed to handle both egocentric human activity understanding and robotic manipulation tasks.
  arXiv  Detail & Related papers  (2025-08-27T13:09:55Z)
• Continual Learning from Simulated Interactions via Multitask Prospective Rehearsal for Bionic Limb Behavior Modeling [0.7922558880545526]
  We introduce a model for human behavior in the context of bionic prosthesis control.<n>We propose a multitasking, continually adaptive model that anticipates and refines movements over time.<n>We validate our model through experiments on real-world human gait datasets, including transtibial amputees.
  arXiv  Detail & Related papers  (2024-05-02T09:22:54Z)
• Physics-constrained Attack against Convolution-based Human Motion Prediction [10.57307572170918]
  We propose a new adversarial attack method that generates the worst-case perturbation by maximizing the human motion predictor's prediction error with physical constraints. Specifically, we introduce a novel adaptable scheme that facilitates the attack to suit the scale of the target pose and two physical constraints to enhance the naturalness of the adversarial example.
  arXiv  Detail & Related papers  (2023-06-21T03:01:45Z)
• SPOTR: Spatio-temporal Pose Transformers for Human Motion Prediction [12.248428883804763]
  3D human motion prediction is a research area computation of high significance and a challenge in computer vision. Traditionally, autogregressive models have been used to predict human motion. We present a non-autoregressive model for human motion prediction.
  arXiv  Detail & Related papers  (2023-03-11T01:44:29Z)
• Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction [63.62263239934777]
  We conduct an indepth study on various pose representations with a focus on their effects on the motion prediction task. We propose a novel RNN architecture termed AHMR (Attentive Hierarchical Motion Recurrent network) for motion prediction. Our approach outperforms the state-of-the-art methods in short-term prediction and achieves much enhanced long-term prediction proficiency.
  arXiv  Detail & Related papers  (2021-12-30T10:45:22Z)
• Generating Smooth Pose Sequences for Diverse Human Motion Prediction [90.45823619796674]
  We introduce a unified deep generative network for both diverse and controllable motion prediction. Our experiments on two standard benchmark datasets, Human3.6M and HumanEva-I, demonstrate that our approach outperforms the state-of-the-art baselines in terms of both sample diversity and accuracy.
  arXiv  Detail & Related papers  (2021-08-19T00:58:00Z)
• Long Term Motion Prediction Using Keyposes [122.22758311506588]
  We argue that, to achieve long term forecasting, predicting human pose at every time instant is unnecessary. We call such poses "keyposes", and approximate complex motions by linearly interpolating between subsequent keyposes. We show that learning the sequence of such keyposes allows us to predict very long term motion, up to 5 seconds in the future.
  arXiv  Detail & Related papers  (2020-12-08T20:45:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.