Human Behavior Atlas: Benchmarking Unified Psychological and Social Behavior Understanding

• URL: http://arxiv.org/abs/2510.04899v1
• Date: Mon, 06 Oct 2025 15:16:45 GMT
• Title: Human Behavior Atlas: Benchmarking Unified Psychological and Social Behavior Understanding
• Authors: Keane Ong, Wei Dai, Carol Li, Dewei Feng, Hengzhi Li, Jingyao Wu, Jiaee Cheong, Rui Mao, Gianmarco Mengaldo, Erik Cambria, Paul Pu Liang,
• Abstract summary: Human Behavior Atlas comprises over 100,000 samples spanning text, audio, and visual modalities.<n>Our unification efforts can reduce redundancy and cost, enable training to scale efficiently across tasks, and enhance generalization of behavioral features across domains.
• Score: 50.34821397821815
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Using intelligent systems to perceive psychological and social behaviors, that is, the underlying affective, cognitive, and pathological states that are manifested through observable behaviors and social interactions, remains a challenge due to their complex, multifaceted, and personalized nature. Existing work tackling these dimensions through specialized datasets and single-task systems often miss opportunities for scalability, cross-task transfer, and broader generalization. To address this gap, we curate Human Behavior Atlas, a unified benchmark of diverse behavioral tasks designed to support the development of unified models for understanding psychological and social behaviors. Human Behavior Atlas comprises over 100,000 samples spanning text, audio, and visual modalities, covering tasks on affective states, cognitive states, pathologies, and social processes. Our unification efforts can reduce redundancy and cost, enable training to scale efficiently across tasks, and enhance generalization of behavioral features across domains. On Human Behavior Atlas, we train three models: OmniSapiens-7B SFT, OmniSapiens-7B BAM, and OmniSapiens-7B RL. We show that training on Human Behavior Atlas enables models to consistently outperform existing multimodal LLMs across diverse behavioral tasks. Pretraining on Human Behavior Atlas also improves transfer to novel behavioral datasets; with the targeted use of behavioral descriptors yielding meaningful performance gains.

Related papers

• OmniSapiens: A Foundation Model for Social Behavior Processing via Heterogeneity-Aware Relative Policy Optimization [50.11607985532808]
  We introduce Heterogeneity-Aware Relative Policy Optimization (HARPO), an RL method that balances leaning across heterogeneous tasks and samples.<n>Using HARPO, we develop and release Omnisapiens-7B 2.0, a foundation model for social behavior processing.<n>Relative to existing behavioral foundation models, Omnisapiens-7B 2.0 achieves the strongest performance across behavioral tasks.
  arXiv  Detail & Related papers  (2026-02-11T08:35:59Z)
• HumanLLM: Towards Personalized Understanding and Simulation of Human Nature [72.55730315685837]
  HumanLLM is a foundation model designed for personalized understanding and simulation of individuals.<n>We first construct the Cognitive Genome, a large-scale corpus curated from real-world user data on platforms like Reddit, Twitter, Blogger, and Amazon.<n>We then formulate diverse learning tasks and perform supervised fine-tuning to empower the model to predict a wide range of individualized human behaviors, thoughts, and experiences.
  arXiv  Detail & Related papers  (2026-01-22T09:27:27Z)
• From Actions to Kinesics: Extracting Human Psychological States through Bodily Movements [1.2676356746752893]
  We present a kinesics recognition framework that infers the communicative functions of human activity from 3D skeleton joint data.<n>Our results on the Dyadic User EngagemenT dataset demonstrate that this method enables scalable, accurate, and human-centered modeling of behavior.
  arXiv  Detail & Related papers  (2025-10-06T14:31:53Z)
• Self-Supervised Learning-Based Multimodal Prediction on Prosocial Behavior Intentions [6.782784535456252]
  There are no large, labeled datasets available for prosocial behavior.<n>Small-scale datasets make it difficult to train deep-learning models effectively.<n>We propose a self-supervised learning approach that harnesses multi-modal data.
  arXiv  Detail & Related papers  (2025-07-11T00:49:46Z)
• Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learning [69.71072181304066]
  We introduce Perceptive Dexterous Control (PDC), a framework for vision-driven whole-body control with simulated humanoids.<n>PDC operates solely on egocentric vision for task specification, enabling object search, target placement, and skill selection through visual cues.<n>We show that training from scratch with reinforcement learning can produce emergent behaviors such as active search.
  arXiv  Detail & Related papers  (2025-05-18T07:33:31Z)
• Social Motion Prediction with Cognitive Hierarchies [19.71780279070757]
  We introduce a new benchmark, a novel formulation, and a cognition-inspired framework. We present Wusi, a 3D multi-person motion dataset under the context of team sports. We develop a cognitive hierarchy framework to predict strategic human social interactions.
  arXiv  Detail & Related papers  (2023-11-08T14:51:17Z)
• LISBET: a machine learning model for the automatic segmentation of social behavior motifs [0.0]
  We introduce LISBET (LISBET Is a Social BEhavior Transformer), a machine learning model for detecting and segmenting social interactions. Using self-supervised learning on body tracking data, our model eliminates the need for extensive human annotation. In vivo electrophysiology revealed distinct neural signatures in the Ventral Tegmental Area corresponding to motifs identified by our model.
  arXiv  Detail & Related papers  (2023-11-07T15:35:17Z)
• Learning to Influence Human Behavior with Offline Reinforcement Learning [70.7884839812069]
  We focus on influence in settings where there is a need to capture human suboptimality. Experiments online with humans is potentially unsafe, and creating a high-fidelity simulator of the environment is often impractical. We show that offline reinforcement learning can learn to effectively influence suboptimal humans by extending and combining elements of observed human-human behavior.
  arXiv  Detail & Related papers  (2023-03-03T23:41:55Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• The world seems different in a social context: a neural network analysis of human experimental data [57.729312306803955]
  We show that it is possible to replicate human behavioral data in both individual and social task settings by modifying the precision of prior and sensory signals. An analysis of the neural activation traces of the trained networks provides evidence that information is coded in fundamentally different ways in the network in the individual and in the social conditions.
  arXiv  Detail & Related papers  (2022-03-03T17:19:12Z)
• PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception [50.551003004553806]
  We create a dataset of physically-grounded abstract social events, PHASE, that resemble a wide range of real-life social interactions. Phase is validated with human experiments demonstrating that humans perceive rich interactions in the social events. As a baseline model, we introduce a Bayesian inverse planning approach, SIMPLE, which outperforms state-of-the-art feed-forward neural networks.
  arXiv  Detail & Related papers  (2021-03-02T18:44:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.