AudioInsight: Detecting Social Contexts Relevant to Social Anxiety from Speech

• URL: http://arxiv.org/abs/2407.14458v1
• Date: Fri, 19 Jul 2024 17:01:12 GMT
• Title: AudioInsight: Detecting Social Contexts Relevant to Social Anxiety from Speech
• Authors: Varun Reddy, Zhiyuan Wang, Emma Toner, Max Larrazabal, Mehdi Boukhechba, Bethany A. Teachman, Laura E. Barnes,
• Abstract summary: This study presents a novel approach that harnesses ambient audio segments to detect social threat contexts. We focus on two key dimensions: number of interaction partners (dyadic vs. group) and degree of evaluative threat. Under sample-wide 5-fold Cross Validation (CV), our model distinguished dyadic from group interactions with 90% accuracy and detected evaluative threat at 83%.
• Score: 10.648118103594586
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: During social interactions, understanding the intricacies of the context can be vital, particularly for socially anxious individuals. While previous research has found that the presence of a social interaction can be detected from ambient audio, the nuances within social contexts, which influence how anxiety provoking interactions are, remain largely unexplored. As an alternative to traditional, burdensome methods like self-report, this study presents a novel approach that harnesses ambient audio segments to detect social threat contexts. We focus on two key dimensions: number of interaction partners (dyadic vs. group) and degree of evaluative threat (explicitly evaluative vs. not explicitly evaluative). Building on data from a Zoom-based social interaction study (N=52 college students, of whom the majority N=45 are socially anxious), we employ deep learning methods to achieve strong detection performance. Under sample-wide 5-fold Cross Validation (CV), our model distinguished dyadic from group interactions with 90\% accuracy and detected evaluative threat at 83\%. Using a leave-one-group-out CV, accuracies were 82\% and 77\%, respectively. While our data are based on virtual interactions due to pandemic constraints, our method has the potential to extend to diverse real-world settings. This research underscores the potential of passive sensing and AI to differentiate intricate social contexts, and may ultimately advance the ability of context-aware digital interventions to offer personalized mental health support.

Related papers

• Multimodal Fusion with LLMs for Engagement Prediction in Natural Conversation [70.52558242336988]
  We focus on predicting engagement in dyadic interactions by scrutinizing verbal and non-verbal cues, aiming to detect signs of disinterest or confusion. In this work, we collect a dataset featuring 34 participants engaged in casual dyadic conversations, each providing self-reported engagement ratings at the end of each conversation. We introduce a novel fusion strategy using Large Language Models (LLMs) to integrate multiple behavior modalities into a multimodal transcript''
  arXiv  Detail & Related papers  (2024-09-13T18:28:12Z)
• Heterogeneous Subgraph Network with Prompt Learning for Interpretable Depression Detection on Social Media [5.570905441172371]
  Existing works about early depression detection on social media lacked interpretability. We develop a novel method that leverages a Heterogeneous Subgraph Network with Prompt Learning. Our proposed method significantly outperforms state-of-the-art methods for depression detection on social media.
  arXiv  Detail & Related papers  (2024-07-12T06:20:59Z)
• SocialBench: Sociality Evaluation of Role-Playing Conversational Agents [85.6641890712617]
  Large language models (LLMs) have advanced the development of various AI conversational agents. SocialBench is the first benchmark designed to evaluate the sociality of role-playing conversational agents at both individual and group levels. We find that agents excelling in individual level does not imply their proficiency in group level.
  arXiv  Detail & Related papers  (2024-03-20T15:38:36Z)
• Personalized State Anxiety Detection: An Empirical Study with Linguistic Biomarkers and A Machine Learning Pipeline [7.512067061195175]
  Individuals high in social anxiety symptoms often exhibit elevated state anxiety in social situations. Research has shown it is possible to detect state anxiety by leveraging digital biomarkers and machine learning techniques.
  arXiv  Detail & Related papers  (2023-04-19T19:06:42Z)
• Wearable Sensor-based Multimodal Physiological Responses of Socially Anxious Individuals across Social Contexts [7.85990334927929]
  We present results using passively collected data from a within-subject experiment that assessed physiological response across different social contexts. Our results suggest that social context is more reliably distinguishable than social phase, group size, or level of social threat, but that there is considerable variability in physiological response patterns even among these distinguishable contexts.
  arXiv  Detail & Related papers  (2023-04-03T18:34:54Z)
• Self-supervised Hypergraph Representation Learning for Sociological Analysis [52.514283292498405]
  We propose a fundamental methodology to support the further fusion of data mining techniques and sociological behavioral criteria. First, we propose an effective hypergraph awareness and a fast line graph construction framework. Second, we propose a novel hypergraph-based neural network to learn social influence flowing from users to users.
  arXiv  Detail & Related papers  (2022-12-22T01:20:29Z)
• Bodily Behaviors in Social Interaction: Novel Annotations and State-of-the-Art Evaluation [0.0]
  We present BBSI, the first set of annotations of complex Bodily Behaviors embedded in continuous Social Interactions. Based on previous work in psychology, we manually annotated 26 hours of spontaneous human behavior. We adapt the Pyramid Dilated Attention Network (PDAN), a state-of-the-art approach for human action detection.
  arXiv  Detail & Related papers  (2022-07-26T11:24:00Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception [50.551003004553806]
  We create a dataset of physically-grounded abstract social events, PHASE, that resemble a wide range of real-life social interactions. Phase is validated with human experiments demonstrating that humans perceive rich interactions in the social events. As a baseline model, we introduce a Bayesian inverse planning approach, SIMPLE, which outperforms state-of-the-art feed-forward neural networks.
  arXiv  Detail & Related papers  (2021-03-02T18:44:57Z)
• Can You be More Social? Injecting Politeness and Positivity into Task-Oriented Conversational Agents [60.27066549589362]
  Social language used by human agents is associated with greater users' responsiveness and task completion. The model uses a sequence-to-sequence deep learning architecture, extended with a social language understanding element. Evaluation in terms of content preservation and social language level using both human judgment and automatic linguistic measures shows that the model can generate responses that enable agents to address users' issues in a more socially appropriate way.
  arXiv  Detail & Related papers  (2020-12-29T08:22:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.