Expanding the Role of Affective Phenomena in Multimodal Interaction Research

• URL: http://arxiv.org/abs/2305.10827v1
• Date: Thu, 18 May 2023 09:08:39 GMT
• Title: Expanding the Role of Affective Phenomena in Multimodal Interaction Research
• Authors: Leena Mathur and Maja J Matari\'c and Louis-Philippe Morency
• Abstract summary: We examined over 16,000 papers from selected conferences in multimodal interaction, affective computing, and natural language processing. We identify 910 affect-related papers and present our analysis of the role of affective phenomena in these papers. We find limited research on how affect and emotion predictions might be used by AI systems to enhance machine understanding of human social behaviors and cognitive states.
• Score: 57.069159905961214
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent decades, the field of affective computing has made substantial progress in advancing the ability of AI systems to recognize and express affective phenomena, such as affect and emotions, during human-human and human-machine interactions. This paper describes our examination of research at the intersection of multimodal interaction and affective computing, with the objective of observing trends and identifying understudied areas. We examined over 16,000 papers from selected conferences in multimodal interaction, affective computing, and natural language processing: ACM International Conference on Multimodal Interaction, AAAC International Conference on Affective Computing and Intelligent Interaction, Annual Meeting of the Association for Computational Linguistics, and Conference on Empirical Methods in Natural Language Processing. We identified 910 affect-related papers and present our analysis of the role of affective phenomena in these papers. We find that this body of research has primarily focused on enabling machines to recognize and express affect and emotion. However, we find limited research on how affect and emotion predictions might be used by AI systems to enhance machine understanding of human social behaviors and cognitive states. Based on our analysis, we discuss directions to expand the role of affective phenomena in multimodal interaction research.

Related papers

• Combining psychoanalysis and computer science: an empirical study of the relationship between emotions and the Lacanian discourses [0.0]
  This research explores the interdisciplinary interaction between psychoanalysis and computer science. In particular, this research aims to apply computer science methods to establish relationships between emotions and Lacanian discourses. Although the main contribution of this paper is inherently theoretical (psychoanalytic), it can also facilitate major practical applications in the realm of interactive digital systems.
  arXiv  Detail & Related papers  (2024-10-30T10:49:33Z)
• Multimodal Fusion with LLMs for Engagement Prediction in Natural Conversation [70.52558242336988]
  We focus on predicting engagement in dyadic interactions by scrutinizing verbal and non-verbal cues, aiming to detect signs of disinterest or confusion. In this work, we collect a dataset featuring 34 participants engaged in casual dyadic conversations, each providing self-reported engagement ratings at the end of each conversation. We introduce a novel fusion strategy using Large Language Models (LLMs) to integrate multiple behavior modalities into a multimodal transcript''
  arXiv  Detail & Related papers  (2024-09-13T18:28:12Z)
• Re-mine, Learn and Reason: Exploring the Cross-modal Semantic Correlations for Language-guided HOI detection [57.13665112065285]
  Human-Object Interaction (HOI) detection is a challenging computer vision task. We present a framework that enhances HOI detection by incorporating structured text knowledge.
  arXiv  Detail & Related papers  (2023-07-25T14:20:52Z)
• A Comprehensive Survey on Affective Computing; Challenges, Trends, Applications, and Future Directions [3.8370454072401685]
  affective computing aims to recognize human emotions, sentiments, and feelings. No research has ever been done to determine how machine learning (ML) and mixed reality (XR) interact together.
  arXiv  Detail & Related papers  (2023-05-08T10:42:46Z)
• Machine Psychology [54.287802134327485]
  We argue that a fruitful direction for research is engaging large language models in behavioral experiments inspired by psychology. We highlight theoretical perspectives, experimental paradigms, and computational analysis techniques that this approach brings to the table. It paves the way for a "machine psychology" for generative artificial intelligence (AI) that goes beyond performance benchmarks.
  arXiv  Detail & Related papers  (2023-03-24T13:24:41Z)
• Social Influence Dialogue Systems: A Scoping Survey of the Efforts Towards Influence Capabilities of Dialogue Systems [50.57882213439553]
  Social influence dialogue systems are capable of persuasion, negotiation, and therapy. There exists no formal definition or category for dialogue systems with these skills. This study serves as a comprehensive reference for social influence dialogue systems to inspire more dedicated research and discussion in this emerging area.
  arXiv  Detail & Related papers  (2022-10-11T17:57:23Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• Empathetic Conversational Systems: A Review of Current Advances, Gaps, and Opportunities [2.741266294612776]
  A growing number of studies have recognized the benefits of empathy and started to incorporate empathy in conversational systems. This paper examines this rapidly growing field using five review dimensions.
  arXiv  Detail & Related papers  (2022-05-09T05:19:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.