A Multimodal Neural Network for Recognizing Subjective Self-Disclosure Towards Social Robots

• URL: http://arxiv.org/abs/2508.10828v1
• Date: Thu, 14 Aug 2025 16:50:51 GMT
• Title: A Multimodal Neural Network for Recognizing Subjective Self-Disclosure Towards Social Robots
• Authors: Henry Powell, Guy Laban, Emily S. Cross,
• Abstract summary: We develop a custom multimodal attention network based on models from the emotion recognition literature.<n>We construct a new loss function, the scale preserving cross entropy loss, that improves upon both classification and regression versions of this problem.
• Score: 3.786116485837376
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Subjective self-disclosure is an important feature of human social interaction. While much has been done in the social and behavioural literature to characterise the features and consequences of subjective self-disclosure, little work has been done thus far to develop computational systems that are able to accurately model it. Even less work has been done that attempts to model specifically how human interactants self-disclose with robotic partners. It is becoming more pressing as we require social robots to work in conjunction with and establish relationships with humans in various social settings. In this paper, our aim is to develop a custom multimodal attention network based on models from the emotion recognition literature, training this model on a large self-collected self-disclosure video corpus, and constructing a new loss function, the scale preserving cross entropy loss, that improves upon both classification and regression versions of this problem. Our results show that the best performing model, trained with our novel loss function, achieves an F1 score of 0.83, an improvement of 0.48 from the best baseline model. This result makes significant headway in the aim of allowing social robots to pick up on an interaction partner's self-disclosures, an ability that will be essential in social robots with social cognition.

Related papers

• SIME: Enhancing Policy Self-Improvement with Modal-level Exploration [49.86173021151849]
  We identify the key to successful self-improvement: modal-level exploration and data selection.<n>By incorporating a modal-level exploration mechanism during policy execution, the robot can produce more diverse and multi-modal interactions.<n>We successfully demonstrate effective robot self-improvement on both simulation benchmarks and real-world experiments.
  arXiv  Detail & Related papers  (2025-05-02T17:13:03Z)
• The Human Robot Social Interaction (HSRI) Dataset: Benchmarking Foundational Models' Social Reasoning [49.32390524168273]
  Our work aims to advance the social reasoning of embodied artificial intelligence (AI) agents in real-world social interactions.<n>We introduce a large-scale real-world Human Robot Social Interaction (HSRI) dataset to benchmark the capabilities of language models (LMs) and foundational models (FMs)<n>Our dataset consists of 400 real-world human social robot interaction videos and over 10K annotations, detailing the robot's social errors, competencies, rationale, and corrective actions.
  arXiv  Detail & Related papers  (2025-04-07T06:27:02Z)
• Nadine: An LLM-driven Intelligent Social Robot with Affective Capabilities and Human-like Memory [3.3906920519220054]
  We describe our approach to developing an intelligent and robust social robotic system for the Nadine platform. We achieve this by integrating Large Language Models (LLMs) and skilfully leveraging the powerful reasoning and instruction-following capabilities of these types of models. This approach is novel compared to the current state-of-the-art LLM-based agents which do not implement human-like long-term memory or sophisticated emotional appraisal.
  arXiv  Detail & Related papers  (2024-05-30T15:55:41Z)
• Real-time Addressee Estimation: Deployment of a Deep-Learning Model on the iCub Robot [52.277579221741746]
  Addressee Estimation is a skill essential for social robots to interact smoothly with humans. Inspired by human perceptual skills, a deep-learning model for Addressee Estimation is designed, trained, and deployed on an iCub robot. The study presents the procedure of such implementation and the performance of the model deployed in real-time human-robot interaction.
  arXiv  Detail & Related papers  (2023-11-09T13:01:21Z)
• Data-driven emotional body language generation for social robotics [58.88028813371423]
  In social robotics, endowing humanoid robots with the ability to generate bodily expressions of affect can improve human-robot interaction and collaboration. We implement a deep learning data-driven framework that learns from a few hand-designed robotic bodily expressions. The evaluation study found that the anthropomorphism and animacy of the generated expressions are not perceived differently from the hand-designed ones.
  arXiv  Detail & Related papers  (2022-05-02T09:21:39Z)
• A General, Evolution-Inspired Reward Function for Social Robotics [0.0]
  We present the Social Reward Function as a mechanism to provide a real-time, dense reward function necessary for the deployment of reinforcement learning agents in social robotics. The Social Reward Function is designed to closely mimic those genetically endowed social perception capabilities of humans in an effort to provide a simple, stable and culture-agnostic reward function.
  arXiv  Detail & Related papers  (2022-02-01T18:05:31Z)
• Cognitive architecture aided by working-memory for self-supervised multi-modal humans recognition [54.749127627191655]
  The ability to recognize human partners is an important social skill to build personalized and long-term human-robot interactions. Deep learning networks have achieved state-of-the-art results and demonstrated to be suitable tools to address such a task. One solution is to make robots learn from their first-hand sensory data with self-supervision.
  arXiv  Detail & Related papers  (2021-03-16T13:50:24Z)
• A Graph Neural Network to Model User Comfort in Robot Navigation [1.6751551703600527]
  This paper leverages Graph Neural Networks to model robot disruption considering the movement of the humans and the robot. The model trained close-to-human performance in the dataset.
  arXiv  Detail & Related papers  (2021-02-17T16:44:52Z)
• Affect-Driven Modelling of Robot Personality for Collaborative Human-Robot Interactions [16.40684407420441]
  Collaborative interactions require social robots to adapt to the dynamics of human affective behaviour. We propose a novel framework for personality-driven behaviour generation in social robots.
  arXiv  Detail & Related papers  (2020-10-14T16:34:14Z)
• Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks [83.37025218216888]
  We propose a novel explainable AI (XAI) framework for achieving human-like communication in human-robot collaborations. The robot builds a hierarchical mind model of the human user and generates explanations of its own mind as a form of communications. Results show that the generated explanations of our approach significantly improves the collaboration performance and user perception of the robot.
  arXiv  Detail & Related papers  (2020-07-24T23:35:03Z)
• Hyperparameters optimization for Deep Learning based emotion prediction for Human Robot Interaction [0.2549905572365809]
  We have proposed an Inception module based Convolutional Neural Network Architecture. The model is implemented in a humanoid robot, NAO in real time and robustness of the model is evaluated.
  arXiv  Detail & Related papers  (2020-01-12T05:25:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.