Embodied Human-Robot Interaction via Acoustics: A MARL Approach with AcoustoBots for Spatial Data Physicalization

• URL: http://arxiv.org/abs/2607.06563v1
• Date: Tue, 07 Jul 2026 17:59:44 GMT
• Title: Embodied Human-Robot Interaction via Acoustics: A MARL Approach with AcoustoBots for Spatial Data Physicalization
• Abstract summary: AcoustoBots is a mobile acoustophoretic data-physicalization platform.<n>TurtleBot3 robots carry upward-facing 8 x 8 ultrasonic phased arrays.<n>Each array levitates a particle whose height encodes a local urban scalar value, such as population density, noise, or traffic.
• Score: 32.62028759076795
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Traditional data physicalization is often static and disconnected from real environments, limiting its ability to convey embodied spatial dynamics and engage users. To address this limitation, we present AcoustoBots, a mobile acoustophoretic data-physicalization platform in which TurtleBot3 robots carry upward-facing 8 x 8 ultrasonic phased arrays. Each array levitates a particle whose height (1-10 cm) encodes a local urban scalar value, such as population density, noise, or traffic. A MARL (Multi-Agent Reinforcement Learning) policy based on the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm, with centralized training and decentralized execution, selects collision-aware navigation actions, while a high-rate Gerchberg-Saxton-Phased Array of Transducers (GS-PAT) acoustic controller maintains trap stability and updates array phases to achieve the commanded height during motion. This creates a closed perception-display-action loop. We evaluate single-robot city-to-city traversal and dual-robot cooperative coverage on a 4 m x 3 m scaled UK map using PhaseSpace-based localization for repeatable multi-robot trials. Results show stable in-motion levitation and consistent, location-dependent height rendering, with task success rates of 90% and 80% for the single and dual-robot regimes, respectively, over 10 trials per regime, and low collision counts. These findings support acoustophoretic levitation as a simple, glanceable, robot-mediated communication cue for embodied human-robot interaction in spatial analytics.

Related papers

• Distributed Dexterous Manipulation with Spatially Conditioned Multi-Agent Transformers [0.7310043452300737]
  This paper introduces a framework based on spatially conditioned Multi-Agent Transformers (MATs) to efficiently learn robust control policies for a Distributed Dexterous Manipulation system.<n>Our three core contributions are: (i) an MAT with adaptive layer norm for compute efficiency, (ii) spatial contrastive embeddings to ground transformer embeddings in the spatial configuration of the robots, and (iii) an MAT-based behavior cloning method fine-tuned using Soft Actor Critic.
  arXiv  Detail & Related papers  (2026-09-07T02:01:12Z)
• RoboGesture: Real-Time Semantic-aligned Co-Speech Gestures Generation for Humanoid Interaction [33.04319504310729]
  RoboGesture is a robot-centric framework that co-designs data, modeling, and control to power a complete interactive human-humanoid system.<n>We first establish the RoboGesture dataset featuring over 300 gesture categories and develop an automated pipeline to synthesize large-scale audio-motion pairs.<n>Experiments on a Unitree G1 humanoid demonstrate that RoboGesture generates safer, more rhythmic, and more semantically appropriate responses compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-08-27T02:13:46Z)
• RGB-D Video Generation for Improving Human-to-Robot Object Handover Prediction [51.76737991200891]
  Human-to-robot (H2R) object handover is a fundamental capability for human-robot collaboration, yet progress is hindered by the scarcity of large-scale, human-centric datasets.<n>We introduce Hand2Bot, an RGB-D video dataset that provides rich contextual information such as body posture and facial expressions.<n>We propose PassGen, a generative pipeline that synthesizes realistic handover sequences while ensuring hand-object consistency.
  arXiv  Detail & Related papers  (2026-08-13T09:55:59Z)
• HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone [19.886495773199023]
  We present HiFi-UMI, a portable UMI data-production system co-designed for trajectory accuracy, inter-gripper relative pose, and field of view.<n>We demonstrate zero-robot post-training: a policy post-trained solely on HiFi-UMI demonstrations deploys directly on a real robot.<n>We open-source HiFi-UMI-2K, 2,000 hours of micro-synchronized, ultra-wide-FoV demonstrations as a large-scale, high-fidelity resource for the robot-learning community.
  arXiv  Detail & Related papers  (2026-07-28T15:52:02Z)
• KinematicRL: A Sim-to-Real Reinforcement Learning Framework For Social Navigation With Kinodynamic Feasibility [27.73146983499444]
  Deep Reinforcement Learning (DRL) has shown promise for social navigation, yet its real-world deployment remains hindered by a persistent sim-to-real gap.<n>This work presents a unified framework that addresses these limitations to produce dynamically feasible navigation policies.<n> Experiments in real-world environments demonstrate that, when combined with the proposed tracking pipeline, KinematicRL can be deployed on a real differential drive robot with minimal modifications.
  arXiv  Detail & Related papers  (2026-06-10T13:07:03Z)
• Decentralized LLM-Driven Coordination of Acoustic Robots for Contactless Object Manipulation [5.280275012917872]
  This paper presents a framework for natural language-driven coordination of acoustic robots for contactless object manipulation.<n>It is implemented on two TurtleBot3-based acoustic robots, each equipped with an ultrasonic phased array.<n>Results show that natural language commands can be transformed into distributed robot actions for contactless manipulation.
  arXiv  Detail & Related papers  (2026-05-28T05:27:13Z)
• Simulation-Informed Diffusion for Decentralized Multi-robot Motion Planning [56.240199425429445]
  Multi-robot motion planning requires each robot to generate collision-free trajectories from local observations.<n>Most existing planners generate trajectories from a static snapshot of the local observation.<n>This paper introduces Simulation-Informed Diffusion (SID), a decentralized framework built on constraint-aware diffusion models.
  arXiv  Detail & Related papers  (2026-05-26T21:17:53Z)
• A Gesture-Based Visual Learning Model for Acoustophoretic Interactions using a Swarm of AcoustoBots [8.171680721155761]
  AcoustoBots are mobile acoustophoretic robots capable of delivering haptics, directional audio, and acoustic levitation.<n>Existing implementations rely on scripted commands and lack an intuitive interface for real-time human control.<n>This work presents a gesture-based visual learning framework for contactless human-swarm interaction with a multimodal AcoustoBot platform.
  arXiv  Detail & Related papers  (2026-04-21T16:32:38Z)
• Dual-Agent Multiple-Model Reinforcement Learning for Event-Triggered Human-Robot Co-Adaptation in Decoupled Task Spaces [3.349003999623489]
  This paper presents a shared-control rehabilitation policy for a custom 6-degree-of-freedom upper-limb robot.<n>The patient governs the primary reaching direction using binary commands, while the robot autonomously manages corrective motions.
  arXiv  Detail & Related papers  (2026-03-06T11:15:10Z)
• ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation [55.467742403416175]
  We introduce a physics-driven neural algorithm that translates large-scale motion capture to humanoid embodiments.<n>We learn a unified multimodal controller that supports both dense references and sparse task specifications.<n>Results show that ULTRA generalizes to autonomous, goal-conditioned whole-body loco-manipulation from egocentric perception.
  arXiv  Detail & Related papers  (2026-03-03T18:59:29Z)
• MolmoSpaces: A Large-Scale Open Ecosystem for Robot Navigation and Manipulation [56.30931340537373]
  MolmoSpaces is a fully open ecosystem to support benchmarking of robot policies.<n>MolmoSpaces consists of over 230k diverse indoor environments.<n>MolmoSpaces-Bench is a benchmark suite of 8 tasks in which robots interact with our diverse scenes and richly annotated objects.
  arXiv  Detail & Related papers  (2026-02-11T20:16:31Z)
• FAST: Efficient Action Tokenization for Vision-Language-Action Models [98.15494168962563]
  We propose a new compression-based tokenization scheme for robot actions, based on the discrete cosine transform.<n>Based on FAST, we release FAST+, a universal robot action tokenizer, trained on 1M real robot action trajectories.
  arXiv  Detail & Related papers  (2025-01-16T18:57:04Z)
• SABER: Data-Driven Motion Planner for Autonomously Navigating Heterogeneous Robots [112.2491765424719]
  We present an end-to-end online motion planning framework that uses a data-driven approach to navigate a heterogeneous robot team towards a global goal. We use model predictive control (SMPC) to calculate control inputs that satisfy robot dynamics, and consider uncertainty during obstacle avoidance with chance constraints. recurrent neural networks are used to provide a quick estimate of future state uncertainty considered in the SMPC finite-time horizon solution. A Deep Q-learning agent is employed to serve as a high-level path planner, providing the SMPC with target positions that move the robots towards a desired global goal.
  arXiv  Detail & Related papers  (2021-08-03T02:56:21Z)
• Relative Localization of Mobile Robots with Multiple Ultra-WideBand Ranging Measurements [15.209043435869189]
  We propose an approach to estimate the relative pose between a group of robots by equipping each robot with multiple UWB ranging nodes. To improve the localization accuracy, we propose to utilize the odometry constraints through a sliding window-based optimization.
  arXiv  Detail & Related papers  (2021-07-19T12:57:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.