Conveying Emotions to Robots through Touch and Sound

• URL: http://arxiv.org/abs/2412.03300v1
• Date: Wed, 04 Dec 2024 13:17:42 GMT
• Title: Conveying Emotions to Robots through Touch and Sound
• Authors: Qiaoqiao Ren, Remko Proesmans, Frederick Bossuyt, Jan Vanfleteren, Francis Wyffels, Tony Belpaeme,
• Abstract summary: There is a lack of understanding of how consistently emotions can be conveyed to robots through touch. This study explores the consistency of touch-based emotional expression toward a robot by integrating tactile and auditory sensory reading of affective haptic expressions.
• Score: 0.9307543852830681
• License:
• Abstract: Human emotions can be conveyed through nuanced touch gestures. However, there is a lack of understanding of how consistently emotions can be conveyed to robots through touch. This study explores the consistency of touch-based emotional expression toward a robot by integrating tactile and auditory sensory reading of affective haptic expressions. We developed a piezoresistive pressure sensor and used a microphone to mimic touch and sound channels, respectively. In a study with 28 participants, each conveyed 10 emotions to a robot using spontaneous touch gestures. Our findings reveal a statistically significant consistency in emotion expression among participants. However, some emotions obtained low intraclass correlation values. Additionally, certain emotions with similar levels of arousal or valence did not exhibit significant differences in the way they were conveyed. We subsequently constructed a multi-modal integrating touch and audio features to decode the 10 emotions. A support vector machine (SVM) model demonstrated the highest accuracy, achieving 40% for 10 classes, with "Attention" being the most accurately conveyed emotion at a balanced accuracy of 87.65%.

Related papers

• EmoSphere-TTS: Emotional Style and Intensity Modeling via Spherical Emotion Vector for Controllable Emotional Text-to-Speech [34.03787613163788]
  EmoSphere-TTS synthesizes expressive emotional speech by using a spherical emotion vector to control the emotional style and intensity of the synthetic speech. We propose a dual conditional adversarial network to improve the quality of generated speech by reflecting the multi-aspect characteristics.
  arXiv  Detail & Related papers  (2024-06-12T01:40:29Z)
• Self context-aware emotion perception on human-robot interaction [3.775456992482295]
  Humans consider that contextual information and different contexts can lead to completely different emotional expressions. We introduce self context-aware model (SCAM) that employs a two-dimensional emotion coordinate system for anchoring and re-labeling distinct emotions. This approach has yielded significant improvements across audio, video, and multimodal environments.
  arXiv  Detail & Related papers  (2024-01-18T10:58:27Z)
• emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation [42.29118614670941]
  We propose emotion2vec, a universal speech emotion representation model. emotion2vec is pre-trained on unlabeled emotion data through self-supervised online distillation. It outperforms state-of-the-art pre-trained universal models and emotion specialist models.
  arXiv  Detail & Related papers  (2023-12-23T07:46:55Z)
• WEARS: Wearable Emotion AI with Real-time Sensor data [0.8740570557632509]
  We propose a system to predict user emotion using smartwatch sensors. We design a framework to collect ground truth in real-time utilizing a mix of English and regional language-based videos. We also did an ablation study to understand the impact of features including Heart Rate, Accelerometer, and Gyroscope sensor data on mood.
  arXiv  Detail & Related papers  (2023-08-22T11:03:00Z)
• Speech Synthesis with Mixed Emotions [77.05097999561298]
  We propose a novel formulation that measures the relative difference between the speech samples of different emotions. We then incorporate our formulation into a sequence-to-sequence emotional text-to-speech framework. At run-time, we control the model to produce the desired emotion mixture by manually defining an emotion attribute vector.
  arXiv  Detail & Related papers  (2022-08-11T15:45:58Z)
• Emotion Intensity and its Control for Emotional Voice Conversion [77.05097999561298]
  Emotional voice conversion (EVC) seeks to convert the emotional state of an utterance while preserving the linguistic content and speaker identity. In this paper, we aim to explicitly characterize and control the intensity of emotion. We propose to disentangle the speaker style from linguistic content and encode the speaker style into a style embedding in a continuous space that forms the prototype of emotion embedding.
  arXiv  Detail & Related papers  (2022-01-10T02:11:25Z)
• A Circular-Structured Representation for Visual Emotion Distribution Learning [82.89776298753661]
  We propose a well-grounded circular-structured representation to utilize the prior knowledge for visual emotion distribution learning. To be specific, we first construct an Emotion Circle to unify any emotional state within it. On the proposed Emotion Circle, each emotion distribution is represented with an emotion vector, which is defined with three attributes.
  arXiv  Detail & Related papers  (2021-06-23T14:53:27Z)
• Enhancing Cognitive Models of Emotions with Representation Learning [58.2386408470585]
  We present a novel deep learning-based framework to generate embedding representations of fine-grained emotions. Our framework integrates a contextualized embedding encoder with a multi-head probing model. Our model is evaluated on the Empathetic Dialogue dataset and shows the state-of-the-art result for classifying 32 emotions.
  arXiv  Detail & Related papers  (2021-04-20T16:55:15Z)
• Emotion Recognition From Gait Analyses: Current Research and Future Directions [48.93172413752614]
  gait conveys information about the walker's emotion. The mapping between various emotions and gait patterns provides a new source for automated emotion recognition. gait is remotely observable, more difficult to imitate, and requires less cooperation from the subject.
  arXiv  Detail & Related papers  (2020-03-13T08:22:33Z)
• ProxEmo: Gait-based Emotion Learning and Multi-view Proxemic Fusion for Socially-Aware Robot Navigation [65.11858854040543]
  We present ProxEmo, a novel end-to-end emotion prediction algorithm for robot navigation among pedestrians. Our approach predicts the perceived emotions of a pedestrian from walking gaits, which is then used for emotion-guided navigation.
  arXiv  Detail & Related papers  (2020-03-02T17:47:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.