Unfair Mistakes on Social Media: How Demographic Characteristics influence Authorship Attribution

• URL: http://arxiv.org/abs/2510.19708v1
• Date: Wed, 22 Oct 2025 15:57:54 GMT
• Title: Unfair Mistakes on Social Media: How Demographic Characteristics influence Authorship Attribution
• Abstract summary: Authorship attribution techniques are increasingly being used in online contexts such as sock puppet detection.<n> Bias in such techniques could lead to false accusations, account banning, and privacy violations disproportionately impacting users from certain demographics.<n>We systematically audit authorship attribution for bias with respect to gender, native language, and age.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Authorship attribution techniques are increasingly being used in online contexts such as sock puppet detection, malicious account linking, and cross-platform account linking. Yet, it is unknown whether these models perform equitably across different demographic groups. Bias in such techniques could lead to false accusations, account banning, and privacy violations disproportionately impacting users from certain demographics. In this paper, we systematically audit authorship attribution for bias with respect to gender, native language, and age. We evaluate fairness in 3 ways. First, we evaluate how the proportion of users with a certain demographic characteristic impacts the overall classifier performance. Second, we evaluate if a user's demographic characteristics influence the probability that their texts are misclassified. Our analysis indicates that authorship attribution does not demonstrate bias across demographic groups in the closed-world setting. Third, we evaluate the types of errors that occur when the true author is removed from the suspect set, thereby forcing the classifier to choose an incorrect author. Unlike the first two settings, this analysis demonstrates a tendency to attribute authorship to users who share the same demographic characteristic as the true author. Crucially, these errors do not only include texts that deviate from a user's usual style, but also those that are very close to the author's average. Our results highlight that though a model may appear fair in the closed-world setting for a performant classifier, this does not guarantee fairness when errors are inevitable.

Related papers

• Attribution Bias in Large Language Models [6.09327435498473]
  We introduce AttriBench, the first fame- and demographically-balanced quote attribution benchmark dataset.<n>We observe large and systematic disparities in attribution accuracy between race, gender, and intersectional groups.<n>We find that suppression is widespread and unevenly distributed across demographic groups, revealing systematic biases not captured by standard accuracy metrics.
  arXiv  Detail & Related papers  (2026-04-06T22:40:03Z)
• Addressing Personalized Bias for Unbiased Learning to Rank [56.663619153713434]
  Unbiased learning to rank (ULTR) aims to learn unbiased ranking models from biased user behavior logs.<n>We propose a novel user-aware inverse-propensity-score estimator for learning-to-rank objectives.
  arXiv  Detail & Related papers  (2025-08-28T14:01:31Z)
• Quantifying Misattribution Unfairness in Authorship Attribution [18.055643489884552]
  Authorship misattribution can have profound consequences in real life.<n>Standard evaluation measures for authorship attribution systems do not explicitly account for this notion of fairness.<n>We introduce a simple measure, Misattribution Unfairness Index (MAUIk), which is based on how often authors are ranked in the top k for texts they did not write.
  arXiv  Detail & Related papers  (2025-06-02T23:28:24Z)
• Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biases [77.3489598315447]
  We argue that identifying the boundary between fact and fair is essential for meaningful fairness evaluation.<n>We introduce Fact-or-Fair, a benchmark with (i) objective queries aligned with descriptive, fact-based judgments, and (ii) subjective queries aligned with normative, fairness-based judgments.
  arXiv  Detail & Related papers  (2025-02-09T10:54:11Z)
• Subjective Crowd Disagreements for Subjective Data: Uncovering Meaningful CrowdOpinion with Population-level Learning [8.530934084017966]
  We introduce emphCrowdOpinion, an unsupervised learning approach that uses language features and label distributions to pool similar items into larger samples of label distributions. We use five publicly available benchmark datasets (with varying levels of annotator disagreements) from social media. We also experiment in the wild using a dataset from Facebook, where annotations come from the platform itself by users reacting to posts.
  arXiv  Detail & Related papers  (2023-07-07T22:09:46Z)
• Gender Biases in Automatic Evaluation Metrics for Image Captioning [87.15170977240643]
  We conduct a systematic study of gender biases in model-based evaluation metrics for image captioning tasks. We demonstrate the negative consequences of using these biased metrics, including the inability to differentiate between biased and unbiased generations. We present a simple and effective way to mitigate the metric bias without hurting the correlations with human judgments.
  arXiv  Detail & Related papers  (2023-05-24T04:27:40Z)
• On Bias and Fairness in NLP: Investigating the Impact of Bias and Debiasing in Language Models on the Fairness of Toxicity Detection [7.297345802761503]
  representation bias, selection bias and overamplification bias are investigated. We show that overamplification bias is the most impactful type of bias on the fairness of the task of toxicity detection. We introduce a list of guidelines to ensure the fairness of the task of toxicity detection.
  arXiv  Detail & Related papers  (2023-05-22T08:44:00Z)
• Uncovering Bias in Face Generation Models [0.0]
  Recent advancements in GANs and diffusion models have enabled the creation of high-resolution, hyper-realistic images. These models may misrepresent certain social groups and present bias. This work is a novel analysis covering and embedding spaces for fine-grained understanding of bias over three approaches.
  arXiv  Detail & Related papers  (2023-02-22T18:57:35Z)
• Learning Fair Node Representations with Graph Counterfactual Fairness [56.32231787113689]
  We propose graph counterfactual fairness, which considers the biases led by the above facts. We generate counterfactuals corresponding to perturbations on each node's and their neighbors' sensitive attributes. Our framework outperforms the state-of-the-art baselines in graph counterfactual fairness.
  arXiv  Detail & Related papers  (2022-01-10T21:43:44Z)
• Balancing out Bias: Achieving Fairness Through Training Reweighting [58.201275105195485]
  Bias in natural language processing arises from models learning characteristics of the author such as gender and race. Existing methods for mitigating and measuring bias do not directly account for correlations between author demographics and linguistic variables. This paper introduces a very simple but highly effective method for countering bias using instance reweighting.
  arXiv  Detail & Related papers  (2021-09-16T23:40:28Z)
• The Authors Matter: Understanding and Mitigating Implicit Bias in Deep Text Classification [36.361778457307636]
  Deep text classification models can produce biased outcomes for texts written by authors of certain demographic groups. In this paper, we first demonstrate that implicit bias exists in different text classification tasks for different demographic groups. We then build a learning-based interpretation method to deepen our knowledge of implicit bias.
  arXiv  Detail & Related papers  (2021-05-06T16:17:38Z)
• Balancing Biases and Preserving Privacy on Balanced Faces in the Wild [50.915684171879036]
  There are demographic biases present in current facial recognition (FR) models. We introduce our Balanced Faces in the Wild dataset to measure these biases across different ethnic and gender subgroups. We find that relying on a single score threshold to differentiate between genuine and imposters sample pairs leads to suboptimal results. We propose a novel domain adaptation learning scheme that uses facial features extracted from state-of-the-art neural networks.
  arXiv  Detail & Related papers  (2021-03-16T15:05:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.