Social Processes: Self-Supervised Forecasting of Nonverbal Cues in Social Conversations

• URL: http://arxiv.org/abs/2107.13576v1
• Date: Wed, 28 Jul 2021 18:01:08 GMT
• Title: Social Processes: Self-Supervised Forecasting of Nonverbal Cues in Social Conversations
• Authors: Chirag Raman, Hayley Hung, Marco Loog
• Abstract summary: We take the first step in the direction of a bottom-up self-supervised approach in the domain of social human interactions. We formulate the task of Social Cue Forecasting to leverage the larger amount of unlabeled low-level behavior cues. We propose the Social Process (SP) models--socially aware sequence-to-sequence (Seq2Seq) models within the Neural Process (NP) family.
• Score: 22.302509912465077
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The default paradigm for the forecasting of human behavior in social conversations is characterized by top-down approaches. These involve identifying predictive relationships between low level nonverbal cues and future semantic events of interest (e.g. turn changes, group leaving). A common hurdle however, is the limited availability of labeled data for supervised learning. In this work, we take the first step in the direction of a bottom-up self-supervised approach in the domain. We formulate the task of Social Cue Forecasting to leverage the larger amount of unlabeled low-level behavior cues, and characterize the modeling challenges involved. To address these, we take a meta-learning approach and propose the Social Process (SP) models--socially aware sequence-to-sequence (Seq2Seq) models within the Neural Process (NP) family. SP models learn extractable representations of non-semantic future cues for each participant, while capturing global uncertainty by jointly reasoning about the future for all members of the group. Evaluation on synthesized and real-world behavior data shows that our SP models achieve higher log-likelihood than the NP baselines, and also highlights important considerations for applying such techniques within the domain of social human interactions.

Related papers

• Context-Aware Prediction of User Engagement on Online Social Platforms [15.847199578750924]
  We present data suggesting that context-aware modeling approaches may offer a holistic yet lightweight representation of user engagement on online social platforms. We analyze more than 100 million Snapchat sessions from almost 80.000 users. Features related to smartphone connectivity status, location, temporal context, and weather were found to capture non-redundant variance in user engagement.
  arXiv  Detail & Related papers  (2023-10-23T03:36:35Z)
• Training Socially Aligned Language Models on Simulated Social Interactions [99.39979111807388]
  Social alignment in AI systems aims to ensure that these models behave according to established societal values. Current language models (LMs) are trained to rigidly replicate their training corpus in isolation. This work presents a novel training paradigm that permits LMs to learn from simulated social interactions.
  arXiv  Detail & Related papers  (2023-05-26T14:17:36Z)
• Social-DualCVAE: Multimodal Trajectory Forecasting Based on Social Interactions Pattern Aware and Dual Conditional Variational Auto-Encoder [14.05141917351931]
  We present a conditional variational auto-encoder (Social-DualCVAE) for multi-modal trajectory forecasting. It is based on a generative model conditioned not only on the past trajectories but also the unsupervised classification of interaction patterns. The proposed model is evaluated on widely used trajectory benchmarks and outperforms the prior state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-02-08T16:04:47Z)
• Learning from Heterogeneous Data Based on Social Interactions over Graphs [58.34060409467834]
  This work proposes a decentralized architecture, where individual agents aim at solving a classification problem while observing streaming features of different dimensions. We show that the. strategy enables the agents to learn consistently under this highly-heterogeneous setting. We show that the. strategy enables the agents to learn consistently under this highly-heterogeneous setting.
  arXiv  Detail & Related papers  (2021-12-17T12:47:18Z)
• SocAoG: Incremental Graph Parsing for Social Relation Inference in Dialogues [112.94918467195637]
  Inferring social relations from dialogues is vital for building emotionally intelligent robots. We model the social network as an And-or Graph, named SocAoG, for the consistency of relations among a group. Empirical results on DialogRE and MovieGraph show that our model infers social relations more accurately than the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-02T08:07:42Z)
• PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception [50.551003004553806]
  We create a dataset of physically-grounded abstract social events, PHASE, that resemble a wide range of real-life social interactions. Phase is validated with human experiments demonstrating that humans perceive rich interactions in the social events. As a baseline model, we introduce a Bayesian inverse planning approach, SIMPLE, which outperforms state-of-the-art feed-forward neural networks.
  arXiv  Detail & Related papers  (2021-03-02T18:44:57Z)
• Human Trajectory Forecasting in Crowds: A Deep Learning Perspective [89.4600982169]
  We present an in-depth analysis of existing deep learning-based methods for modelling social interactions. We propose two knowledge-based data-driven methods to effectively capture these social interactions. We develop a large scale interaction-centric benchmark TrajNet++, a significant yet missing component in the field of human trajectory forecasting.
  arXiv  Detail & Related papers  (2020-07-07T17:19:56Z)
• Recursive Social Behavior Graph for Trajectory Prediction [49.005219590582676]
  We formulate social representations supervised by group-based annotations into a social behavior graph, called Recursive Social Behavior Graph. With the guidance of Recursive Social Behavior Graph, we surpass state-of-the-art method on ETH and UCY dataset for 11.1% in ADE and 10.8% in FDE.
  arXiv  Detail & Related papers  (2020-04-22T06:01:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.