Unifying Data Perspectivism and Personalization: An Application to Social Norms

• URL: http://arxiv.org/abs/2210.14531v3
• Date: Sun, 22 Oct 2023 20:16:42 GMT
• Title: Unifying Data Perspectivism and Personalization: An Application to Social Norms
• Authors: Joan Plepi, B\'ela Neuendorf, Lucie Flek, Charles Welch
• Abstract summary: We examine a corpus of social media posts about conflict from a set of 13k annotators and 210k judgements of social norms. We apply personalization methods to the modeling of annotators and compare their effectiveness for predicting the perception of social norms.
• Score: 10.480094567764606
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Instead of using a single ground truth for language processing tasks, several recent studies have examined how to represent and predict the labels of the set of annotators. However, often little or no information about annotators is known, or the set of annotators is small. In this work, we examine a corpus of social media posts about conflict from a set of 13k annotators and 210k judgements of social norms. We provide a novel experimental setup that applies personalization methods to the modeling of annotators and compare their effectiveness for predicting the perception of social norms. We further provide an analysis of performance across subsets of social situations that vary by the closeness of the relationship between parties in conflict, and assess where personalization helps the most.

Related papers

• Identifying Privacy Personas [27.301741710016223]
  Privacy personas capture the differences in user segments with respect to one's knowledge, behavioural patterns, level of self-efficacy, and perception of the importance of privacy protection. While various privacy personas have been derived in the literature, they group together people who differ from each other in terms of important attributes. We propose eight personas that we derive by combining qualitative and quantitative analysis of the responses to an interactive educational questionnaire.
  arXiv  Detail & Related papers  (2024-10-17T20:49:46Z)
• Towards "Differential AI Psychology" and in-context Value-driven Statement Alignment with Moral Foundations Theory [0.0]
  This work investigates the alignment between personalized language models and survey participants on a Moral Foundation questionnaire. We adapt text-to-text models to different political personas and survey the questionnaire repetitively to generate a synthetic population of persona and model combinations. Our findings indicate that adapted models struggle to represent the survey-leading assessment of political ideologies.
  arXiv  Detail & Related papers  (2024-08-21T08:20:41Z)
• Social Bias Probing: Fairness Benchmarking for Language Models [38.180696489079985]
  This paper proposes a novel framework for probing language models for social biases by assessing disparate treatment. We curate SoFa, a large-scale benchmark designed to address the limitations of existing fairness collections. We show that biases within language models are more nuanced than acknowledged, indicating a broader scope of encoded biases than previously recognized.
  arXiv  Detail & Related papers  (2023-11-15T16:35:59Z)
• Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems [103.416202777731]
  We study "persona biases", which we define to be the sensitivity of dialogue models' harmful behaviors contingent upon the personas they adopt. We categorize persona biases into biases in harmful expression and harmful agreement, and establish a comprehensive evaluation framework to measure persona biases in five aspects: Offensiveness, Toxic Continuation, Regard, Stereotype Agreement, and Toxic Agreement.
  arXiv  Detail & Related papers  (2023-10-08T21:03:18Z)
• Measuring the Effect of Influential Messages on Varying Personas [67.1149173905004]
  We present a new task, Response Forecasting on Personas for News Media, to estimate the response a persona might have upon seeing a news message. The proposed task not only introduces personalization in the modeling but also predicts the sentiment polarity and intensity of each response. This enables more accurate and comprehensive inference on the mental state of the persona.
  arXiv  Detail & Related papers  (2023-05-25T21:01:00Z)
• Understanding Interpersonal Conflict Types and their Impact on Perception Classification [7.907976678407914]
  We use a novel annotation scheme and release a new dataset of situations and conflict aspect annotations. We then build a classifier to predict whether someone will perceive the actions of one individual as right or wrong in a given situation. Our findings have important implications for understanding conflict and social norms.
  arXiv  Detail & Related papers  (2022-08-18T10:39:35Z)
• Visual Comparison of Language Model Adaptation [55.92129223662381]
  adapters are lightweight alternatives for model adaptation. In this paper, we discuss several design and alternatives for interactive, comparative visual explanation methods. We show that, for instance, an adapter trained on the language debiasing task according to context-0 embeddings introduces a new type of bias.
  arXiv  Detail & Related papers  (2022-08-17T09:25:28Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• Personalized multi-faceted trust modeling to determine trust links in social media and its potential for misinformation management [61.88858330222619]
  We present an approach for predicting trust links between peers in social media. We propose a data-driven multi-faceted trust modeling which incorporates many distinct features for a comprehensive analysis. Illustrated in a trust-aware item recommendation task, we evaluate the proposed framework in the context of a large Yelp dataset.
  arXiv  Detail & Related papers  (2021-11-11T19:40:51Z)
• On Predicting Personal Values of Social Media Users using Community-Specific Language Features and Personal Value Correlation [14.12186042953335]
  This work focuses on analyzing Singapore users' personal values and developing effective models to predict their personal values using their Facebook data. We incorporate the correlations among personal values into our proposed Stack Model consisting of a task-specific layer of base models and a cross-stitch layer model.
  arXiv  Detail & Related papers  (2020-07-16T04:36:13Z)
• Human Trajectory Forecasting in Crowds: A Deep Learning Perspective [89.4600982169]
  We present an in-depth analysis of existing deep learning-based methods for modelling social interactions. We propose two knowledge-based data-driven methods to effectively capture these social interactions. We develop a large scale interaction-centric benchmark TrajNet++, a significant yet missing component in the field of human trajectory forecasting.
  arXiv  Detail & Related papers  (2020-07-07T17:19:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.