SocioGesture: Real-Time and Adaptive Social Gesture Perception for Human-Robot Interaction

• URL: http://arxiv.org/abs/2609.04545v1
• Date: Thu, 03 Sep 2026 23:03:43 GMT
• Title: SocioGesture: Real-Time and Adaptive Social Gesture Perception for Human-Robot Interaction
• Abstract summary: We present SocioGesture, a real-time adaptive social gesture perception system for human-robot interaction (HRI)<n>SocioGesture uses a compact confidence-aware body-hand skeleton representation and a lightweight dual-stream model that fuses body motion with hand articulation for low-latency onboard recognition.<n>On a social gesture dataset collected in mixed indoor-outdoor HRI scenarios, SocioGesture achieves strong held-out-subject recognition.<n>During deployment, uncertain interaction segments are saved for offline labeling and adaptation, enabling SocioGesture to expand its gesture vocabulary.
• Score: 4.0450087039946006
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Robots interacting with people must recognize not only explicit commands, but also social cues such as invitations, refusals, and unavailability. In real deployments, these cues must be inferred from noisy onboard perception under partial occlusion, changing viewpoints, and strict latency constraints. We present SocioGesture, a real-time adaptive social gesture perception system for human-robot interaction (HRI). SocioGesture uses a compact confidence-aware body-hand skeleton representation and a lightweight dual-stream model that fuses body motion with hand articulation for low-latency onboard recognition. To improve deployment robustness, we train the model with occlusion-aware skeleton corruption, exposing it to missing hands, occluded arms, and temporally unstable keypoints without increasing the inference cost. On a social gesture dataset collected in mixed indoor-outdoor HRI scenarios, SocioGesture achieves strong held-out-subject recognition, substantially improves robustness under structured joint occlusion, and runs in real time on a robot-mounted edge device. During deployment, uncertain interaction segments are saved for offline labeling and adaptation, enabling SocioGesture to expand its gesture vocabulary while preserving performance in the original classes. These results demonstrate a practical path toward robust, efficient, and adaptive social perception for interactive robots.

Related papers

• RGB-D Video Generation for Improving Human-to-Robot Object Handover Prediction [51.76737991200891]
  Human-to-robot (H2R) object handover is a fundamental capability for human-robot collaboration, yet progress is hindered by the scarcity of large-scale, human-centric datasets.<n>We introduce Hand2Bot, an RGB-D video dataset that provides rich contextual information such as body posture and facial expressions.<n>We propose PassGen, a generative pipeline that synthesizes realistic handover sequences while ensuring hand-object consistency.
  arXiv  Detail & Related papers  (2026-08-13T09:55:59Z)
• Anatomy of Uncertainty: Expressive Descriptors of Robotic Manipulator Motion for Non-verbal Communication in Human-Robot Collaboration [0.6576864734526405]
  This work introduces a mathematical framework for expressing perceptual uncertainty through robotic manipulator motion.<n>Results establish a perceptually grounded basis for encoding robot uncertainty in motion and support future autonomous trajectory generation.
  arXiv  Detail & Related papers  (2026-07-15T10:48:44Z)
• Trajectory Learning with Graph Representations for Social Robot Navigation [1.0732935873226022]
  We propose an imitation learning framework that leverages social dynamics for socially compliant navigation.<n>We present a navigation module that captures temporal dynamics and mitigates error accumulations.<n>Our framework outperforms established data-driven baselines on simulation and a real-world dataset across diverse social metrics.
  arXiv  Detail & Related papers  (2026-06-21T23:57:54Z)
• Face versus Body Tracking for Human-Robot Interaction: An Egocentric Dataset [7.3852311170912905]
  State-of-the-art computer vision models are heavily optimized for surveillance or autonomous driving.<n>A social robot faces distinct egocentric challenges, such as humans bouncing, obstructing each other, or leaving the frame.<n>We introduce a novel, custom-annotated egocentric dataset collected via the Furhat robot to capture complex social dynamics.
  arXiv  Detail & Related papers  (2026-06-02T14:15:17Z)
• SocialMirror: Reconstructing 3D Human Interaction Behaviors from Monocular Videos with Semantic and Geometric Guidance [49.69016078147708]
  Accurately reconstructing human behavior in close-interaction scenarios is crucial for enabling realistic virtual interactions in augmented reality.<n>We propose SocialMirror, a diffusion-based framework that integrates semantic and geometric cues to effectively address these issues.<n>SocialMirror achieves state-of-the-art performance in reconstructing interactive human meshes, demonstrating strong generalization across unseen datasets and in-the-wild scenarios.
  arXiv  Detail & Related papers  (2026-04-15T07:41:52Z)
• Real-Time Synchronized Interaction Framework for Emotion-Aware Humanoid Robots [0.5156484100374058]
  We present a real-time framework for NAO robots that synchronizes speech prosody with full-body gestures.<n>By enabling seamless sensorimotor coordination, this framework advances the deployment of context-aware social robots.
  arXiv  Detail & Related papers  (2026-01-24T03:53:09Z)
• Part-Aware Bottom-Up Group Reasoning for Fine-Grained Social Interaction Detection [82.70752567211251]
  We propose a part-aware bottom-up group reasoning framework for fine-grained social interaction detection.<n>The proposed method infers social groups and their interactions using body part features and their interpersonal relations.<n>Our model first detects individuals and enhances their features using part-aware cues, and then infers group configuration by associating individuals via similarity-based reasoning.
  arXiv  Detail & Related papers  (2025-11-05T17:33:03Z)
• Towards Immersive Human-X Interaction: A Real-Time Framework for Physically Plausible Motion Synthesis [51.95817740348585]
  Human-X is a novel framework designed to enable immersive and physically plausible human interactions across diverse entities.<n>Our method jointly predicts actions and reactions in real-time using an auto-regressive reaction diffusion planner.<n>Our framework is validated in real-world applications, including virtual reality interface for human-robot interaction.
  arXiv  Detail & Related papers  (2025-08-04T06:35:48Z)
• Data-driven emotional body language generation for social robotics [58.88028813371423]
  In social robotics, endowing humanoid robots with the ability to generate bodily expressions of affect can improve human-robot interaction and collaboration. We implement a deep learning data-driven framework that learns from a few hand-designed robotic bodily expressions. The evaluation study found that the anthropomorphism and animacy of the generated expressions are not perceived differently from the hand-designed ones.
  arXiv  Detail & Related papers  (2022-05-02T09:21:39Z)
• Cognitive architecture aided by working-memory for self-supervised multi-modal humans recognition [54.749127627191655]
  The ability to recognize human partners is an important social skill to build personalized and long-term human-robot interactions. Deep learning networks have achieved state-of-the-art results and demonstrated to be suitable tools to address such a task. One solution is to make robots learn from their first-hand sensory data with self-supervision.
  arXiv  Detail & Related papers  (2021-03-16T13:50:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.