JRDB-Social: A Multifaceted Robotic Dataset for Understanding of Context and Dynamics of Human Interactions Within Social Groups

• URL: http://arxiv.org/abs/2404.04458v1
• Date: Sat, 6 Apr 2024 00:33:39 GMT
• Title: JRDB-Social: A Multifaceted Robotic Dataset for Understanding of Context and Dynamics of Human Interactions Within Social Groups
• Authors: Simindokht Jahangard, Zhixi Cai, Shiki Wen, Hamid Rezatofighi,
• Abstract summary: JRDB-Social fills gaps in human understanding across diverse indoor and outdoor social contexts. This dataset aims to enhance our grasp of human social dynamics for robotic applications.
• Score: 8.415759777703125
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding human social behaviour is crucial in computer vision and robotics. Micro-level observations like individual actions fall short, necessitating a comprehensive approach that considers individual behaviour, intra-group dynamics, and social group levels for a thorough understanding. To address dataset limitations, this paper introduces JRDB-Social, an extension of JRDB. Designed to fill gaps in human understanding across diverse indoor and outdoor social contexts, JRDB-Social provides annotations at three levels: individual attributes, intra-group interactions, and social group context. This dataset aims to enhance our grasp of human social dynamics for robotic applications. Utilizing the recent cutting-edge multi-modal large language models, we evaluated our benchmark to explore their capacity to decipher social human behaviour.

Related papers

• The Human Robot Social Interaction (HSRI) Dataset: Benchmarking Foundational Models' Social Reasoning [49.32390524168273]
  Our work aims to advance the social reasoning of embodied artificial intelligence (AI) agents in real-world social interactions. We introduce a large-scale real-world Human Robot Social Interaction (HSRI) dataset to benchmark the capabilities of language models (LMs) and foundational models (FMs) Our dataset consists of 400 real-world human social robot interaction videos and over 10K annotations, detailing the robot's social errors, competencies, rationale, and corrective actions.
  arXiv  Detail & Related papers  (2025-04-07T06:27:02Z)
• SocialGen: Modeling Multi-Human Social Interaction with Language Models [30.068879125411726]
  We introduce SocialGen, the first unified motion-language model capable of modeling interaction behaviors among varying numbers of individuals. Unlike prior methods that are limited to two-person interactions, we propose a novel social motion representation that supports tokenizing the motions of an arbitrary number of individuals. This alignment enables the model to leverage rich, pretrained linguistic knowledge to better understand and reason about human social behaviors.
  arXiv  Detail & Related papers  (2025-03-28T22:57:25Z)
• Social Processes: Probabilistic Meta-learning for Adaptive Multiparty Interaction Forecasting [3.9134031118910264]
  We introduce Social Process (SP) models, which predict a distribution over future multimodal cues jointly for all group members. We also analyze the generalization capabilities of SP models in both their outputs and latent spaces through the use of realistic synthetic datasets.
  arXiv  Detail & Related papers  (2025-01-03T17:34:53Z)
• Characterizing User Archetypes and Discussions on Scored.co [0.6321194486116923]
  We present a framework for characterizing nodes and hyperedges in social hypernetworks. We focus on the understudied alt-right platform Scored.co. Our findings highlight the importance of higher-order interactions in understanding social dynamics.
  arXiv  Detail & Related papers  (2024-07-31T17:18:25Z)
• SocialBench: Sociality Evaluation of Role-Playing Conversational Agents [85.6641890712617]
  Large language models (LLMs) have advanced the development of various AI conversational agents. SocialBench is the first benchmark designed to evaluate the sociality of role-playing conversational agents at both individual and group levels. We find that agents excelling in individual level does not imply their proficiency in group level.
  arXiv  Detail & Related papers  (2024-03-20T15:38:36Z)
• Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future [59.78608958395464]
  We build a Social AI Data Infrastructure, which consists of a comprehensive social AI taxonomy and a data library of 480 NLP datasets. Our infrastructure allows us to analyze existing dataset efforts, and also evaluate language models' performance in different social intelligence aspects. We show there is a need for multifaceted datasets, increased diversity in language and culture, more long-tailed social situations, and more interactive data in future social intelligence data efforts.
  arXiv  Detail & Related papers  (2024-02-28T00:22:42Z)
• SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents [107.4138224020773]
  We present SOTOPIA, an open-ended environment to simulate complex social interactions between artificial agents and humans. In our environment, agents role-play and interact under a wide variety of scenarios; they coordinate, collaborate, exchange, and compete with each other to achieve complex social goals. We find that GPT-4 achieves a significantly lower goal completion rate than humans and struggles to exhibit social commonsense reasoning and strategic communication skills.
  arXiv  Detail & Related papers  (2023-10-18T02:27:01Z)
• Self-supervised Social Relation Representation for Human Group Detection [18.38523753680367]
  We propose a new two-stage multi-head framework for human group detection. In the first stage, we propose a human behavior simulator head to learn the social relation feature embedding. In the second stage, based on the social relation embedding, we develop a self-attention inspired network for human group detection.
  arXiv  Detail & Related papers  (2022-03-08T04:26:07Z)
• Social Processes: Self-Supervised Forecasting of Nonverbal Cues in Social Conversations [22.302509912465077]
  We take the first step in the direction of a bottom-up self-supervised approach in the domain of social human interactions. We formulate the task of Social Cue Forecasting to leverage the larger amount of unlabeled low-level behavior cues. We propose the Social Process (SP) models--socially aware sequence-to-sequence (Seq2Seq) models within the Neural Process (NP) family.
  arXiv  Detail & Related papers  (2021-07-28T18:01:08Z)
• JRDB-Act: A Large-scale Multi-modal Dataset for Spatio-temporal Action, Social Group and Activity Detection [54.696819174421584]
  We introduce JRDB-Act, a multi-modal dataset that reflects a real distribution of human daily life actions in a university campus environment. JRDB-Act has been densely annotated with atomic actions, comprises over 2.8M action labels. JRDB-Act comes with social group identification annotations conducive to the task of grouping individuals based on their interactions in the scene.
  arXiv  Detail & Related papers  (2021-06-16T14:43:46Z)
• PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception [50.551003004553806]
  We create a dataset of physically-grounded abstract social events, PHASE, that resemble a wide range of real-life social interactions. Phase is validated with human experiments demonstrating that humans perceive rich interactions in the social events. As a baseline model, we introduce a Bayesian inverse planning approach, SIMPLE, which outperforms state-of-the-art feed-forward neural networks.
  arXiv  Detail & Related papers  (2021-03-02T18:44:57Z)
• Human Trajectory Forecasting in Crowds: A Deep Learning Perspective [89.4600982169]
  We present an in-depth analysis of existing deep learning-based methods for modelling social interactions. We propose two knowledge-based data-driven methods to effectively capture these social interactions. We develop a large scale interaction-centric benchmark TrajNet++, a significant yet missing component in the field of human trajectory forecasting.
  arXiv  Detail & Related papers  (2020-07-07T17:19:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.