From Actions to Kinesics: Extracting Human Psychological States through Bodily Movements

• URL: http://arxiv.org/abs/2510.04844v1
• Date: Mon, 06 Oct 2025 14:31:53 GMT
• Title: From Actions to Kinesics: Extracting Human Psychological States through Bodily Movements
• Authors: Cheyu Lin, Katherine A. Flanigan,
• Abstract summary: We present a kinesics recognition framework that infers the communicative functions of human activity from 3D skeleton joint data.<n>Our results on the Dyadic User EngagemenT dataset demonstrate that this method enables scalable, accurate, and human-centered modeling of behavior.
• Score: 1.2676356746752893
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Understanding the dynamic relationship between humans and the built environment is a key challenge in disciplines ranging from environmental psychology to reinforcement learning (RL). A central obstacle in modeling these interactions is the inability to capture human psychological states in a way that is both generalizable and privacy preserving. Traditional methods rely on theoretical models or questionnaires, which are limited in scope, static, and labor intensive. We present a kinesics recognition framework that infers the communicative functions of human activity -- known as kinesics -- directly from 3D skeleton joint data. Combining a spatial-temporal graph convolutional network (ST-GCN) with a convolutional neural network (CNN), the framework leverages transfer learning to bypass the need for manually defined mappings between physical actions and psychological categories. The approach preserves user anonymity while uncovering latent structures in bodily movements that reflect cognitive and emotional states. Our results on the Dyadic User EngagemenT (DUET) dataset demonstrate that this method enables scalable, accurate, and human-centered modeling of behavior, offering a new pathway for enhancing RL-driven simulations of human-environment interaction.

Related papers

• Learning Whole-Body Human-Humanoid Interaction from Human-Human Demonstrations [63.80827184637476]
  We introduce D-STAR, a hierarchical policy that disentangles when to act from where to act.<n>We validate our framework through extensive and rigorous simulations.
  arXiv  Detail & Related papers  (2026-01-14T14:37:06Z)
• BrainATCL: Adaptive Temporal Brain Connectivity Learning for Functional Link Prediction and Age Estimation [0.33748750222488655]
  We propose BrainATCL, an unsupervised, nonparametric framework for adaptive temporal brain connectivity learning.<n>Our method dynamically adjusts the lookback window for each snapshot based on the rate of newly added edges.<n>Graph sequences are encoded using a GINE-Mamba2 backbone to learn spatial-temporal representations of dynamic functional connectivity in resting-state fMRI data.
  arXiv  Detail & Related papers  (2025-08-09T21:18:25Z)
• Towards Immersive Human-X Interaction: A Real-Time Framework for Physically Plausible Motion Synthesis [51.95817740348585]
  Human-X is a novel framework designed to enable immersive and physically plausible human interactions across diverse entities.<n>Our method jointly predicts actions and reactions in real-time using an auto-regressive reaction diffusion planner.<n>Our framework is validated in real-world applications, including virtual reality interface for human-robot interaction.
  arXiv  Detail & Related papers  (2025-08-04T06:35:48Z)
• Visual-Geometric Collaborative Guidance for Affordance Learning [63.038406948791454]
  We propose a visual-geometric collaborative guided affordance learning network that incorporates visual and geometric cues. Our method outperforms the representative models regarding objective metrics and visual quality.
  arXiv  Detail & Related papers  (2024-10-15T07:35:51Z)
• Graph-Based Representation Learning of Neuronal Dynamics and Behavior [2.3859858429583665]
  We introduce the Temporal Attention-enhanced Variational Graph Recurrent Neural Network (TAVRNN), a novel framework that models time-varying neuronal connectivity.<n>TAVRNN learns latent dynamics at the single-unit level while maintaining interpretable population-level representations.<n>We validate TAVRNN on three diverse datasets: (1) electrophysiological data from a freely behaving rat, (2) primate somatosensory cortex recordings during a reaching task, and (3) biological neurons in the DishBrain platform interacting with a virtual game environment.
  arXiv  Detail & Related papers  (2024-10-01T13:19:51Z)
• Interpretable Spatio-Temporal Embedding for Brain Structural-Effective Network with Ordinary Differential Equation [56.34634121544929]
  In this study, we first construct the brain-effective network via the dynamic causal model. We then introduce an interpretable graph learning framework termed Spatio-Temporal Embedding ODE (STE-ODE) This framework incorporates specifically designed directed node embedding layers, aiming at capturing the dynamic interplay between structural and effective networks.
  arXiv  Detail & Related papers  (2024-05-21T20:37:07Z)
• DSAM: A Deep Learning Framework for Analyzing Temporal and Spatial Dynamics in Brain Networks [4.041732967881764]
  Most rs-fMRI studies compute a single static functional connectivity matrix across brain regions of interest. These approaches are at risk of oversimplifying brain dynamics and lack proper consideration of the goal at hand. We propose a novel interpretable deep learning framework that learns goal-specific functional connectivity matrix directly from time series.
  arXiv  Detail & Related papers  (2024-05-19T23:35:06Z)
• CG-HOI: Contact-Guided 3D Human-Object Interaction Generation [29.3564427724612]
  We propose CG-HOI, the first method to generate dynamic 3D human-object interactions (HOIs) from text. We model the motion of both human and object in an interdependent fashion, as semantically rich human motion rarely happens in isolation. We show that our joint contact-based human-object interaction approach generates realistic and physically plausible sequences.
  arXiv  Detail & Related papers  (2023-11-27T18:59:10Z)
• Persistent-Transient Duality: A Multi-mechanism Approach for Modeling Human-Object Interaction [58.67761673662716]
  Humans are highly adaptable, swiftly switching between different modes to handle different tasks, situations and contexts. In Human-object interaction (HOI) activities, these modes can be attributed to two mechanisms: (1) the large-scale consistent plan for the whole activity and (2) the small-scale children interactive actions that start and end along the timeline. This work proposes to model two concurrent mechanisms that jointly control human motion.
  arXiv  Detail & Related papers  (2023-07-24T12:21:33Z)
• Language Knowledge-Assisted Representation Learning for Skeleton-Based Action Recognition [71.35205097460124]
  How humans understand and recognize the actions of others is a complex neuroscientific problem. LA-GCN proposes a graph convolution network using large-scale language models (LLM) knowledge assistance.
  arXiv  Detail & Related papers  (2023-05-21T08:29:16Z)
• The world seems different in a social context: a neural network analysis of human experimental data [57.729312306803955]
  We show that it is possible to replicate human behavioral data in both individual and social task settings by modifying the precision of prior and sensory signals. An analysis of the neural activation traces of the trained networks provides evidence that information is coded in fundamentally different ways in the network in the individual and in the social conditions.
  arXiv  Detail & Related papers  (2022-03-03T17:19:12Z)
• Human Action Recognition and Assessment via Deep Neural Network Self-Organization [0.0]
  This chapter introduces a set of hierarchical models for the learning and recognition of actions from depth maps and RGB images. A particularity of these models is the use of growing self-organizing networks that quickly adapt to non-stationary distributions.
  arXiv  Detail & Related papers  (2020-01-04T15:58:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.