Modeling Human Perspectives with Socio-Demographic Representations

• URL: http://arxiv.org/abs/2604.18069v1
• Date: Mon, 20 Apr 2026 10:37:26 GMT
• Title: Modeling Human Perspectives with Socio-Demographic Representations
• Abstract summary: We propose Socio-Contrastive Learning, a method that jointly models annotator perspectives while learning socio-demographic representations.<n>Our method provides an effective approach for the fusion of socio-demographic features and textual representations to predict annotator perspectives.
• Score: 0.6138671548064355
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humans often hold different perspectives on the same issues. In many NLP tasks, annotation disagreement can reflect valid subjective perspectives. Modeling annotator perspectives and understanding their relationship with other human factors, such as socio-demographic attributes, have received increasing attention. Prior work typically focuses on single demographic factors or limited combinations. However, in real-world settings, annotator perspectives are shaped by complex social contexts, and finer-grained socio-demographic attributes can better explain human perspectives. In this work, we propose Socio-Contrastive Learning, a method that jointly models annotator perspectives while learning socio-demographic representations. Our method provides an effective approach for the fusion of socio-demographic features and textual representations to predict annotator perspectives, outperforming standard concatenation-based methods. The learned representations further enable analysis and visualization of how demographic factors relate to variation in annotator perspectives. Our code is available at GitHub: https://github.com/Leixin-Zhang/Socio_Contrastive_Learning

Related papers

• Human-Inspired Social Engagement Analysis via Interpretable Mutual Visual Attention [7.72838047070244]
  We propose an interpretable computational model of social engagement inspired by psychological theories of mutual visual attention.<n>Our framework explicitly models dyadic visual attention and aggregates these cues into interpretable measures of individual and group engagement.
  arXiv  Detail & Related papers  (2026-08-25T14:05:35Z)
• AlignFace: Human-Aligned Face Similarity Metric with Interpretable Concept Relations [13.228617286387887]
  We introduce the FACETS dataset and propose AlignFace, an interpretable, human-aligned, face similarity metric.<n>Experiments found AlignFace significantly improves alignment with human subpopulation perceptions compared to baseline metrics.
  arXiv  Detail & Related papers  (2026-08-14T09:36:28Z)
• Learning visual representations for compositional analysis of artworks and photographs [40.190936097282275]
  We compare two parallel paradigms for composition analysis: a human-inspired method grounded in perceptual grouping, and fine-tuned foundation models enabled by recent large-scale compositional datasets.<n>With frozen encoders, the human-inspired method achieves competitive performance while remaining interpretable.<n>When sufficient data enables fine-tuning, large self-supervised models outperform significantly, but at the cost of interpretability and cross-domain generalization.
  arXiv  Detail & Related papers  (2026-08-06T15:11:24Z)
• AI Alignment From Social Choice Perspectives [37.88586337953354]
  We show how social choice theory helps identify failure modes in the feedback aggregation layer.<n>We show how the social choice perspective helps identify a broader design space for handling disagreement in explicit and principled ways.
  arXiv  Detail & Related papers  (2026-06-19T15:47:01Z)
• Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions [33.76973308687867]
  We show that models do improve in sociodemographic prompting when trained.<n>This performance gain is largely due to models learning annotator-specific behaviour rather than sociodemographic patterns.<n>Across all tasks, our results suggest that models learn little meaningful connection between sociodemographics and annotation.
  arXiv  Detail & Related papers  (2025-02-28T09:53:42Z)
• Testing the Limits of Fine-Tuning for Improving Visual Cognition in Vision Language Models [51.58859621164201]
  We introduce visual stimuli and human judgments on visual cognition tasks to evaluate performance across cognitive domains.<n>We fine-tune models on ground truth data for intuitive physics and causal reasoning.<n>We find that task-specific fine-tuning does not contribute to robust human-like generalization to data with other visual characteristics.
  arXiv  Detail & Related papers  (2025-02-21T18:58:30Z)
• Perception of Visual Content: Differences Between Humans and Foundation Models [4.251488927334905]
  This study explores the similarity between human-generated and ML-generated annotations of images across diverse socio-economic contexts.<n>We aim to understand differences in perception and identify potential biases in content interpretation.
  arXiv  Detail & Related papers  (2024-11-28T07:37:04Z)
• When Does Perceptual Alignment Benefit Vision Representations? [76.32336818860965]
  We investigate how aligning vision model representations to human perceptual judgments impacts their usability. We find that aligning models to perceptual judgments yields representations that improve upon the original backbones across many downstream tasks. Our results suggest that injecting an inductive bias about human perceptual knowledge into vision models can contribute to better representations.
  arXiv  Detail & Related papers  (2024-10-14T17:59:58Z)
• How Do You Perceive My Face? Recognizing Facial Expressions in Multi-Modal Context by Modeling Mental Representations [5.895694050664867]
  We introduce a novel approach for facial expression classification that goes beyond simple classification tasks. Our model accurately classifies a perceived face and synthesizes the corresponding mental representation perceived by a human when observing a face in context. We evaluate synthesized expressions in a human study, showing that our model effectively produces approximations of human mental representations.
  arXiv  Detail & Related papers  (2024-09-04T09:32:40Z)
• Decoding Visual Sentiment of Political Imagery [0.0]
  This study introduces a novel approach to visual sentiment analysis by integrating attitudinal differences into visual sentiment classification. We trained a deep learning multi-task multi-class model to predict visual sentiment from different ideological viewpoints.
  arXiv  Detail & Related papers  (2024-08-07T21:44:56Z)
• From a Social Cognitive Perspective: Context-aware Visual Social Relationship Recognition [59.57095498284501]
  We propose a novel approach that recognizes textbfContextual textbfSocial textbfRelationships (textbfConSoR) from a social cognitive perspective. We construct social-aware descriptive language prompts with social relationships for each image. Impressively, ConSoR outperforms previous methods with a 12.2% gain on the People-in-Social-Context (PISC) dataset and a 9.8% increase on the People-in-Photo-Album (PIPA) benchmark.
  arXiv  Detail & Related papers  (2024-06-12T16:02:28Z)
• GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectives [18.574420136899978]
  We propose GRASP, a comprehensive disagreement analysis framework to measure group association in perspectives among different rater sub-groups. Our framework reveals specific rater groups that have significantly different perspectives than others on certain tasks, and helps identify demographic axes that are crucial to consider in specific task contexts.
  arXiv  Detail & Related papers  (2023-11-09T00:12:21Z)
• Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting [64.80538055623842]
  sociodemographic prompting is a technique that steers the output of prompt-based models towards answers that humans with specific sociodemographic profiles would give. We show that sociodemographic information affects model predictions and can be beneficial for improving zero-shot learning in subjective NLP tasks.
  arXiv  Detail & Related papers  (2023-09-13T15:42:06Z)
• Fairness meets Cross-Domain Learning: a new perspective on Models and Metrics [80.07271410743806]
  We study the relationship between cross-domain learning (CD) and model fairness. We introduce a benchmark on face and medical images spanning several demographic groups as well as classification and localization tasks. Our study covers 14 CD approaches alongside three state-of-the-art fairness algorithms and shows how the former can outperform the latter.
  arXiv  Detail & Related papers  (2023-03-25T09:34:05Z)
• Enhancing Social Relation Inference with Concise Interaction Graph and Discriminative Scene Representation [56.25878966006678]
  We propose an approach of textbfPRactical textbfInference in textbfSocial rtextbfElation (PRISE) It concisely learns interactive features of persons and discriminative features of holistic scenes. PRISE achieves 6.8$%$ improvement for domain classification in PIPA dataset.
  arXiv  Detail & Related papers  (2021-07-30T04:20:13Z)
• Quantifying Learnability and Describability of Visual Concepts Emerging in Representation Learning [91.58529629419135]
  We consider how to characterise visual groupings discovered automatically by deep neural networks. We introduce two concepts, visual learnability and describability, that can be used to quantify the interpretability of arbitrary image groupings.
  arXiv  Detail & Related papers  (2020-10-27T18:41:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.