Read the Room: Adapting a Robot's Voice to Ambient and Social Contexts

• URL: http://arxiv.org/abs/2205.04952v3
• Date: Thu, 21 Sep 2023 14:55:04 GMT
• Title: Read the Room: Adapting a Robot's Voice to Ambient and Social Contexts
• Abstract summary: We describe a process and results toward selecting robot voice styles for perceived social appropriateness and ambiance awareness. Our results with N=120 participants provide evidence that the choice of voice style in different ambiances impacted a robot's perceived intelligence.
• Score: 1.0732907121422146
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: How should a robot speak in a formal, quiet and dark, or a bright, lively and noisy environment? By designing robots to speak in a more social and ambient-appropriate manner we can improve perceived awareness and intelligence for these agents. We describe a process and results toward selecting robot voice styles for perceived social appropriateness and ambiance awareness. Understanding how humans adapt their voices in different acoustic settings can be challenging due to difficulties in voice capture in the wild. Our approach includes 3 steps: (a) Collecting and validating voice data interactions in virtual Zoom ambiances, (b) Exploration and clustering human vocal utterances to identify primary voice styles, and (c) Testing robot voice styles in recreated ambiances using projections, lighting and sound. We focus on food service scenarios as a proof-of-concept setting. We provide results using the Pepper robot's voice with different styles, towards robots that speak in a contextually appropriate and adaptive manner. Our results with N=120 participants provide evidence that the choice of voice style in different ambiances impacted a robot's perceived intelligence in several factors including: social appropriateness, comfort, awareness, human-likeness and competency.

Related papers

• Do Robots Need Body Language? Comparing Communication Modalities for Legible Motion Intent in Human-Shared Spaces [22.039354879428497]
  High-DoF robots exhibit motion that people read as expressive, intentionally or not, making it important to understand how such cues are perceived.<n>We present an online video study evaluating how different signaling modalities, expressive motion, lights, text, and audio, shape people's ability to understand a quadruped robot's upcoming navigation actions.
  arXiv  Detail & Related papers  (2026-04-03T20:53:05Z)
• MOSS-VoiceGenerator: Create Realistic Voices with Natural Language Descriptions [60.69565588088127]
  We present MOSS-VoiceGenerator, an open-source instruction-driven voice generation model that creates new timbres directly from natural language prompts.<n>Motivated by the hypothesis that exposure to real-world acoustic variation produces more perceptually natural voices, we train on large-scale expressive speech data sourced from cinematic content.<n>Subjective preference studies demonstrate its superiority in overall performance, instruction-following, and naturalness compared to other voice design models.
  arXiv  Detail & Related papers  (2026-03-30T06:40:59Z)
• Taming Transformer for Emotion-Controllable Talking Face Generation [61.835295250047196]
  We propose a novel method to tackle the emotion-controllable talking face generation task discretely.<n>Specifically, we employ two pre-training strategies to disentangle audio into independent components and quantize videos into combinations of visual tokens.<n>We conduct experiments on the MEAD dataset that controls the emotion of videos conditioned on multiple emotional audios.
  arXiv  Detail & Related papers  (2025-08-20T02:16:52Z)
• OmniCharacter: Towards Immersive Role-Playing Agents with Seamless Speech-Language Personality Interaction [123.89581506075461]
  We propose OmniCharacter, a first seamless speech-language personality interaction model to achieve immersive RPAs with low latency.<n> Specifically, OmniCharacter enables agents to consistently exhibit role-specific personality traits and vocal traits throughout the interaction.<n>Our method yields better responses in terms of both content and style compared to existing RPAs and mainstream speech-language models, with a response latency as low as 289ms.
  arXiv  Detail & Related papers  (2025-05-26T17:55:06Z)
• ANAVI: Audio Noise Awareness using Visuals of Indoor environments for NAVIgation [26.460679530665487]
  We propose Audio Noise Awareness using Visuals of Indoors for NAVIgation for quieter robot path planning. We generate data on how loud an 'impulse' sounds at different listener locations in simulated homes, and train our Acoustic Noise Predictor (ANP) Unifying ANP with action acoustics, we demonstrate experiments with wheeled (Hello Robot Stretch) and legged (Unitree Go2) robots so that these robots adhere to the noise constraints of the environment.
  arXiv  Detail & Related papers  (2024-10-24T17:19:53Z)
• SIFToM: Robust Spoken Instruction Following through Theory of Mind [51.326266354164716]
  We present a cognitively inspired model, Speech Instruction Following through Theory of Mind (SIFToM), to enable robots to pragmatically follow human instructions under diverse speech conditions. Results show that the SIFToM model outperforms state-of-the-art speech and language models, approaching human-level accuracy on challenging speech instruction following tasks.
  arXiv  Detail & Related papers  (2024-09-17T02:36:10Z)
• No More Mumbles: Enhancing Robot Intelligibility through Speech Adaptation [7.675340768192281]
  We conduct a speech comprehension study involving 39 participants. Experiment's primary outcome shows that spaces with good acoustic quality positively correlate with intelligibility and user experience. We develop a convolutional neural network model to adapt the robot's speech parameters to different users and spaces.
  arXiv  Detail & Related papers  (2024-05-15T21:28:55Z)
• Developing Social Robots with Empathetic Non-Verbal Cues Using Large Language Models [2.5489046505746704]
  We design and label four types of empathetic non-verbal cues, abbreviated as SAFE: Speech, Action (gesture), Facial expression, and Emotion, in a social robot. Preliminary results show distinct patterns in the robot's responses, such as a preference for calm and positive social emotions like 'joy' and 'lively', and frequent nodding gestures. Our work lays the groundwork for future studies on human-robot interactions, emphasizing the essential role of both verbal and non-verbal cues in creating social and empathetic robots.
  arXiv  Detail & Related papers  (2023-08-31T08:20:04Z)
• See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation [49.925499720323806]
  We study how visual, auditory, and tactile perception can jointly help robots to solve complex manipulation tasks. We build a robot system that can see with a camera, hear with a contact microphone, and feel with a vision-based tactile sensor.
  arXiv  Detail & Related papers  (2022-12-07T18:55:53Z)
• Data-driven emotional body language generation for social robotics [58.88028813371423]
  In social robotics, endowing humanoid robots with the ability to generate bodily expressions of affect can improve human-robot interaction and collaboration. We implement a deep learning data-driven framework that learns from a few hand-designed robotic bodily expressions. The evaluation study found that the anthropomorphism and animacy of the generated expressions are not perceived differently from the hand-designed ones.
  arXiv  Detail & Related papers  (2022-05-02T09:21:39Z)
• Synthesis and Execution of Communicative Robotic Movements with Generative Adversarial Networks [59.098560311521034]
  We focus on how to transfer on two different robotic platforms the same kinematics modulation that humans adopt when manipulating delicate objects. We choose to modulate the velocity profile adopted by the robots' end-effector, inspired by what humans do when transporting objects with different characteristics. We exploit a novel Generative Adversarial Network architecture, trained with human kinematics examples, to generalize over them and generate new and meaningful velocity profiles.
  arXiv  Detail & Related papers  (2022-03-29T15:03:05Z)
• Robotic Speech Synthesis: Perspectives on Interactions, Scenarios, and Ethics [2.6959411243976175]
  We discuss the difficulties of synthesizing non-verbal and interaction-oriented speech signals, particularly backchannels. We present the findings of relevant literature and our prior work, trying to bring the attention of human-robot interaction researchers to design better conversational robots.
  arXiv  Detail & Related papers  (2022-03-17T20:24:17Z)
• Disambiguating Affective Stimulus Associations for Robot Perception and Dialogue [67.89143112645556]
  We provide a NICO robot with the ability to learn the associations between a perceived auditory stimulus and an emotional expression. NICO is able to do this for both individual subjects and specific stimuli, with the aid of an emotion-driven dialogue system. The robot is then able to use this information to determine a subject's enjoyment of perceived auditory stimuli in a real HRI scenario.
  arXiv  Detail & Related papers  (2021-03-05T20:55:48Z)
• Self-supervised reinforcement learning for speaker localisation with the iCub humanoid robot [58.2026611111328]
  Looking at a person's face is one of the mechanisms that humans rely on when it comes to filtering speech in noisy environments. Having a robot that can look toward a speaker could benefit ASR performance in challenging environments. We propose a self-supervised reinforcement learning-based framework inspired by the early development of humans.
  arXiv  Detail & Related papers  (2020-11-12T18:02:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.