OAA: Three Phases of Vocal Guidance in Human-Drone Teleoperation

• URL: http://arxiv.org/abs/2608.10651v1
• Date: Tue, 11 Aug 2026 08:35:08 GMT
• Title: OAA: Three Phases of Vocal Guidance in Human-Drone Teleoperation
• Abstract summary: We show that spontaneous vocal guidance consistently organizes into three kinematically and linguistically distinct phases: Orientation, Approach, and Adjustment.<n>We discuss implications for OAA-aware adaptive control in voice-guided teleoperation.
• Score: 1.3868431756593296
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Voice-guided teleoperation requires systems that adapt to the evolving dynamics of human guidance. Yet most voice-controlled robot systems treat spoken commands as a stationary stream, ignoring how the guide's communicative behavior changes as the task progresses. Using motion capture and speech data from two experimental configurations, humanhuman guidance (finger pointing, N =10 dyads) and humandrone teleoperation (gamepad control, N =29 dyads), we show that spontaneous vocal guidance consistently organizes into three kinematically and linguistically distinct phases: Orientation, Approach, and Adjustment. These phases are identified automatically via change point detection on 3D trajectory signals, and validated statistically (Kruskal-Wallis, p<.001). Three lexical families replicate across configurations: rotation vocabulary marks Orientation, translation vocabulary is scarce there, and attenuators accumulate toward Adjustment. Together with inter-utterance silence, these cues mark the Orientation boundary that speech rate alone leaves unmarked. The same three-phase structure emerges in both configurations despite radically different motor interfaces, suggesting it is an intrinsic property of human spatial guidance rather than an artifact of the experimental setup. We discuss implications for OAA-aware adaptive control in voice-guided teleoperation.

Related papers

• TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model [49.03909082056646]
  We study the problem of navigating cluttered indoor environments with a humanoid robot.<n>We introduce TANGO, the first whole-body vision-language navigation framework.<n>In simulation experiments, TANGO demonstrates state-of-the-art performance in vision-language navigation.
  arXiv  Detail & Related papers  (2026-09-08T17:59:55Z)
• Scalable Direction-Following TTS via Voice Impression-Guided Pseudo Triplet Construction [16.024524623990484]
  We study direction-following TTS, where a system generates a new utterance that reflects a given direction relative to a reference utterance.<n>Key challenge is the lack of training data capturing such relative modifications.<n>We propose a scalable pseudo-triplet construction pipeline that generates(reference utterance, direction text, modified utterance) triplets.
  arXiv  Detail & Related papers  (2026-09-02T14:02:15Z)
• Omni-Supervised Motion Editing: Balancing Change and Invariance through Positive-Negative Learning [55.410155357480654]
  We propose an Omni-Supervised Positive-Negative Learning framework, named OmniME.<n>Our method integrates three complementary components.<n>Experiments on the MotionFix and STANCE Adjustment datasets demonstrate that OmniME achieves state-of-the-art performance in editing alignment.
  arXiv  Detail & Related papers  (2026-05-29T08:09:51Z)
• SARAH: Spatially Aware Real-time Agentic Humans [58.32612596034656]
  We present the first real-time, fully causal method for spatially-aware conversational motion, deployable on a streaming VR headset.<n>Given a user's position and dyadic audio, our approach produces full-body motion that aligns gestures with speech while orienting the agent according to the user.<n>We validate our approach on a live VR system, bringing spatially-aware conversational agents to real-time deployment.
  arXiv  Detail & Related papers  (2026-02-20T18:59:35Z)
• 3DGesPolicy: Phoneme-Aware Holistic Co-Speech Gesture Generation Based on Action Control [3.606473077857744]
  3DGesPolicy is an action-based framework that reformulates holistic gesture generation as a continuous trajectory control problem.<n>By modeling frame-to-frame variations as unified holistic actions, our method effectively learns inter-frame holistic gesture motion patterns.<n>To further bridge the gap in expressive alignment, we propose a Gesture-Audio-Phoneme (GAP) fusion module.
  arXiv  Detail & Related papers  (2026-01-26T12:57:36Z)
• ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Body [25.087113475162592]
  ViBES (Voice in Behavioral Expression and Synchrony) is a conversational 3D agent that jointly plans language and movement.<n>ViBES goes beyond "speech-conditioned motion generation" toward agentic virtual bodies where language, prosody, and movement are jointly generated.
  arXiv  Detail & Related papers  (2025-12-16T09:41:21Z)
• Modeling Turn-Taking with Semantically Informed Gestures [56.31369237947851]
  We introduce DnD Gesture++, an extension of the multi-party DnD Gesture corpus enriched with 2,663 semantic gesture annotations.<n>We model turn-taking prediction through a Mixture-of-Experts framework integrating text, audio, and gestures.<n>Experiments show that incorporating semantically guided gestures yields consistent performance gains over baselines.
  arXiv  Detail & Related papers  (2025-10-22T08:17:54Z)
• doScenes: An Autonomous Driving Dataset with Natural Language Instruction for Human Interaction and Vision-Language Navigation [0.0]
  doScenes is a novel dataset designed to facilitate research on human-vehicle instruction interactions.<n>DoScenes bridges the gap between instruction and driving response, enabling context-aware and adaptive planning.
  arXiv  Detail & Related papers  (2024-12-08T11:16:47Z)
• Pragmatic Embodied Spoken Instruction Following in Human-Robot Collaboration with Theory of Mind [51.45478233267092]
  We present a cognitively inspired neurosymbolic model, Spoken Instruction Following through Theory of Mind (SIFToM)<n>SIFToM uses a Vision-Language Model with model-based mental inference to enable robots to pragmatically follow human instructions under diverse speech conditions.<n>Results show that SIFToM can significantly improve the performance of a lightweight base VLM (Gemini 2.5 Flash), outperforming state-of-the-art VLMs (Gemini 2.5 Pro) and approaching human-level accuracy on challenging spoken instruction following tasks.
  arXiv  Detail & Related papers  (2024-09-17T02:36:10Z)
• Evaluating Voice Command Pipelines for Drone Control: From STT and LLM to Direct Classification and Siamese Networks [0.0]
  The aim is to enhance human-machine interaction by enabling intuitive voice control of drone actions. The pipelines developed include: (1) a traditional Speech-to-Text (STT) followed by a Large Language Model (LLM) approach, (2) a direct voice-to-function mapping model, and (3) a Siamese neural network-based system.
  arXiv  Detail & Related papers  (2024-07-10T15:15:26Z)
• ConvoFusion: Multi-Modal Conversational Diffusion for Co-Speech Gesture Synthesis [50.69464138626748]
  We present ConvoFusion, a diffusion-based approach for multi-modal gesture synthesis. Our method proposes two guidance objectives that allow the users to modulate the impact of different conditioning modalities. Our method is versatile in that it can be trained either for generating monologue gestures or even the conversational gestures.
  arXiv  Detail & Related papers  (2024-03-26T17:59:52Z)
• Episodic Transformer for Vision-and-Language Navigation [142.6236659368177]
  This paper focuses on addressing two challenges: handling long sequence of subtasks, and understanding complex human instructions. We propose Episodic Transformer (E.T.), a multimodal transformer that encodes language inputs and the full episode history of visual observations and actions. Our approach sets a new state of the art on the challenging ALFRED benchmark, achieving 38.4% and 8.5% task success rates on seen and unseen test splits.
  arXiv  Detail & Related papers  (2021-05-13T17:51:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.