OmniRobotHome: A Multi-Camera Platform for Real-Time Multiadic Human-Robot Interaction

• URL: http://arxiv.org/abs/2604.28197v1
• Date: Thu, 30 Apr 2026 17:59:58 GMT
• Title: OmniRobotHome: A Multi-Camera Platform for Real-Time Multiadic Human-Robot Interaction
• Abstract summary: We present OmniRobotHome, the first room-scale residential platform that unifies real-time 3D human and object perception with coordinated multi-robot actuation.<n>We focus on two central problems: safety in shared human-robot environments and human-anticipatory robotic assistance.
• Score: 21.78933851010173
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human-robot collaboration has been studied primarily in dyadic or sequential settings. However, real homes require multiadic collaboration, where multiple humans and robots share a workspace, acting concurrently on interleaved subtasks with tight spatial and temporal coupling. This regime remains underexplored because close-proximity interaction between humans, robots, and objects creates persistent occlusion and rapid state changes, making reliable real-time 3D tracking the central bottleneck. No existing platform provides the real-time, occlusion-robust, room-scale perception needed to make this regime experimentally tractable. We present OmniRobotHome, the first room-scale residential platform that unifies wide-area real-time 3D human and object perception with coordinated multi-robot actuation in a shared world frame. The system instruments a natural home environment with 48 hardware-synchronized RGB cameras for markerless, occlusion-robust tracking of multiple humans and objects, temporally aligned with two Franka arms that act on live scene state. Continuous capture within this consistent frame further supports long-horizon human behavior modeling from accumulated trajectories. The platform makes the multiadic collaboration regime experimentally tractable. We focus on two central problems: safety in shared human-robot environments and human-anticipatory robotic assistance, and show that real-time perception and accumulated behavior memory each yield measurable gains in both.

Related papers

• SPOT: Spatial Perception-Oriented Long-Horizon Humanoid Teleoperation [35.4182413141688]
  SPOT combines a robot-mounted fisheye camera, a wide-field stereoscopic display, viewpoint-decoupled free-looking, and visual stabilization.<n>We evaluate SPOT on perception-critical humanoid data-collection tasks spanning drop recovery, peripheral retrieval, large-workspace bimanual manipulation, fine alignment, and dynamic interaction.
  arXiv  Detail & Related papers  (2026-09-07T19:46:56Z)
• AnyWorld: Factorized Egocentric World Models for Cross-Embodiment Generalization [88.25615187835321]
  We propose AnyWorld, a cross-embodiment world modeling framework.<n>Our model factorizes an interaction into action, camera, and embodiment.<n>We train the model with large-scale human interaction pretraining.
  arXiv  Detail & Related papers  (2026-08-29T12:55:15Z)
• GRAIL: Generating Humanoid Loco-Manipulation from 3D Assets and Video Priors [113.71148915419246]
  We present GRAIL, a digital generation pipeline that composes 3D assets, simulator-ready scenes, and priors from video foundation models (VFMs) to synthesize interactions without rebuilding physical environments or teleoperating the robot.<n>GRAIL starts from fully specified 3D configurations in which object geometry, camera parameters, metric scale, environment depth, and a robot-proportioned character are known before video generation and reused during reconstruction.<n>We retarget the recovered motions to a humanoid robot and train complementary task-general morphology trackers.<n>GRAIL produces over 20,000 sequences spanning pick-up, object manipulation, sitting
  arXiv  Detail & Related papers  (2026-06-03T17:57:45Z)
• Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaboration [19.192199380267393]
  We introduce TouchSafeBench, a benchmark for evaluating collision grounding in vision-language models (VLMs)<n>TouchSafeBench contains 2,940 simulated indoor co-presence episodes across social navigation and social rearrangement.<n>We study two deployment-facing tasks: classifying the current safety state and warning about imminent collision before contact.
  arXiv  Detail & Related papers  (2026-05-29T12:04:38Z)
• Morphology-Consistent Humanoid Interaction through Robot-Centric Video Synthesis [25.249184346335557]
  Dream2Act is a robot-centric framework enabling zero-shot interaction through generative video.<n> operating strictly within the robot-native space, Dream2Act avoids errors and eliminates task-specific policy training.
  arXiv  Detail & Related papers  (2026-03-20T07:27:09Z)
• EmbodMocap: In-the-Wild 4D Human-Scene Reconstruction for Embodied Agents [85.77432303199176]
  We propose EmbodMocap, a portable and affordable data collection pipeline using two moving iPhones.<n>Our key idea is to jointly calibrate dual RGB-D sequences to reconstruct both humans and scenes.<n>Based on the collected data, we empower three embodied AI tasks: monocular human-scene-reconstruction, where we fine-tune feedforward models that output metric-scale, world-space aligned humans and scenes; physics-based character animation, where we prove our data could be used to scale human-object interaction skills and scene-aware motion tracking; and robot motion control, where we train a humanoid robot via
  arXiv  Detail & Related papers  (2026-02-26T16:53:41Z)
• HOSIG: Full-Body Human-Object-Scene Interaction Generation with Hierarchical Scene Perception [57.37135310143126]
  HO SIG is a novel framework for synthesizing full-body interactions through hierarchical scene perception.<n>Our framework supports unlimited motion length through autoregressive generation and requires minimal manual intervention.<n>This work bridges the critical gap between scene-aware navigation and dexterous object manipulation.
  arXiv  Detail & Related papers  (2025-06-02T12:08:08Z)
• VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation [53.63540587160549]
  VidBot is a framework enabling zero-shot robotic manipulation using learned 3D affordance from in-the-wild monocular RGB-only human videos.<n> VidBot paves the way for leveraging everyday human videos to make robot learning more scalable.
  arXiv  Detail & Related papers  (2025-03-10T10:04:58Z)
• Robot Interaction Behavior Generation based on Social Motion Forecasting for Human-Robot Interaction [9.806227900768926]
  We propose to model social motion forecasting in a shared human-robot representation space. ECHO operates in the aforementioned shared space to predict the future motions of the agents encountered in social scenarios. We evaluate our model in multi-person and human-robot motion forecasting tasks and obtain state-of-the-art performance by a large margin.
  arXiv  Detail & Related papers  (2024-02-07T11:37:14Z)
• Revisit Human-Scene Interaction via Space Occupancy [55.67657438543008]
  Human-scene Interaction (HSI) generation is a challenging task and crucial for various downstream tasks. In this work, we argue that interaction with a scene is essentially interacting with the space occupancy of the scene from an abstract physical perspective. By treating pure motion sequences as records of humans interacting with invisible scene occupancy, we can aggregate motion-only data into a large-scale paired human-occupancy interaction database.
  arXiv  Detail & Related papers  (2023-12-05T12:03:00Z)
• Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots [119.55240471433302]
  Habitat 3.0 is a simulation platform for studying collaborative human-robot tasks in home environments. It addresses challenges in modeling complex deformable bodies and diversity in appearance and motion. Human-in-the-loop infrastructure enables real human interaction with simulated robots via mouse/keyboard or a VR interface.
  arXiv  Detail & Related papers  (2023-10-19T17:29:17Z)
• Ditto in the House: Building Articulation Models of Indoor Scenes through Interactive Perception [31.009703947432026]
  This work explores building articulation models of indoor scenes through a robot's purposeful interactions. We introduce an interactive perception approach to this task. We demonstrate the effectiveness of our approach in both simulation and real-world scenes.
  arXiv  Detail & Related papers  (2023-02-02T18:22:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.