Quantifying Misattribution Unfairness in Authorship Attribution

• URL: http://arxiv.org/abs/2506.02321v1
• Date: Mon, 02 Jun 2025 23:28:24 GMT
• Title: Quantifying Misattribution Unfairness in Authorship Attribution
• Abstract summary: Authorship misattribution can have profound consequences in real life.<n>Standard evaluation measures for authorship attribution systems do not explicitly account for this notion of fairness.<n>We introduce a simple measure, Misattribution Unfairness Index (MAUIk), which is based on how often authors are ranked in the top k for texts they did not write.
• Score: 18.055643489884552
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Authorship misattribution can have profound consequences in real life. In forensic settings simply being considered as one of the potential authors of an evidential piece of text or communication can result in undesirable scrutiny. This raises a fairness question: Is every author in the candidate pool at equal risk of misattribution? Standard evaluation measures for authorship attribution systems do not explicitly account for this notion of fairness. We introduce a simple measure, Misattribution Unfairness Index (MAUIk), which is based on how often authors are ranked in the top k for texts they did not write. Using this measure we quantify the unfairness of five models on two different datasets. All models exhibit high levels of unfairness with increased risks for some authors. Furthermore, we find that this unfairness relates to how the models embed the authors as vectors in the latent search space. In particular, we observe that the risk of misattribution is higher for authors closer to the centroid (or center) of the embedded authors in the haystack. These results indicate the potential for harm and the need for communicating with and calibrating end users on misattribution risk when building and providing such models for downstream use.

Related papers

• Quantifying the Statistical Effect of Rubric Modifications on Human-Autorater Agreement [50.34437999083224]
  Autoraters are increasingly used for evaluation and automated moderation content.<n>There is limited statistical analysis of how modifications in a rubric presented to both humans and autoraters affect their score agreement.
  arXiv  Detail & Related papers  (2026-05-07T13:55:11Z)
• Attribution Bias in Large Language Models [6.09327435498473]
  We introduce AttriBench, the first fame- and demographically-balanced quote attribution benchmark dataset.<n>We observe large and systematic disparities in attribution accuracy between race, gender, and intersectional groups.<n>We find that suppression is widespread and unevenly distributed across demographic groups, revealing systematic biases not captured by standard accuracy metrics.
  arXiv  Detail & Related papers  (2026-04-06T22:40:03Z)
• Are Finer Citations Always Better? Rethinking Granularity for Attributed Generation [75.27453112832093]
  We show that enforcing fine-grained citations degrades attribution quality by 16-276% compared to the best-performing granularity.<n>Our analysis suggests that fine-grained (sentence-level) citations disrupt necessary semantic dependencies for attributing evidence to answer claims.<n>Strikingly, citation-optimal granularity leads to substantial gains in attribution quality while preserving or even improving answer correctness.
  arXiv  Detail & Related papers  (2026-04-01T22:16:23Z)
• Unfair Mistakes on Social Media: How Demographic Characteristics influence Authorship Attribution [0.0]
  Authorship attribution techniques are increasingly being used in online contexts such as sock puppet detection.<n> Bias in such techniques could lead to false accusations, account banning, and privacy violations disproportionately impacting users from certain demographics.<n>We systematically audit authorship attribution for bias with respect to gender, native language, and age.
  arXiv  Detail & Related papers  (2025-10-22T15:57:54Z)
• Authorship Without Writing: Large Language Models and the Senior Author Analogy [0.0]
  The use of large language models (LLMs) in bioethical, scientific, and medical writing remains controversial.<n>We argue that LLM use is analogous to a form of senior authorship.
  arXiv  Detail & Related papers  (2025-09-05T10:11:56Z)
• Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biases [77.3489598315447]
  We argue that identifying the boundary between fact and fair is essential for meaningful fairness evaluation.<n>We introduce Fact-or-Fair, a benchmark with (i) objective queries aligned with descriptive, fact-based judgments, and (ii) subjective queries aligned with normative, fairness-based judgments.
  arXiv  Detail & Related papers  (2025-02-09T10:54:11Z)
• Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges [16.35265384114857]
  The rapid advancements of Large Language Models (LLMs) have blurred the lines between human and machine authorship.<n>This literature review serves a roadmap for researchers and practitioners interested in understanding the state of the art in this rapidly evolving field.
  arXiv  Detail & Related papers  (2024-08-16T17:58:49Z)
• What Hides behind Unfairness? Exploring Dynamics Fairness in Reinforcement Learning [52.51430732904994]
  In reinforcement learning problems, agents must consider long-term fairness while maximizing returns. Recent works have proposed many different types of fairness notions, but how unfairness arises in RL problems remains unclear. We introduce a novel notion called dynamics fairness, which explicitly captures the inequality stemming from environmental dynamics.
  arXiv  Detail & Related papers  (2024-04-16T22:47:59Z)
• Foundation Models and Fair Use [96.04664748698103]
  In the U.S. and other countries, copyrighted content may be used to build foundation models without incurring liability due to the fair use doctrine. In this work, we survey the potential risks of developing and deploying foundation models based on copyrighted content. We discuss technical mitigations that can help foundation models stay in line with fair use.
  arXiv  Detail & Related papers  (2023-03-28T03:58:40Z)
• Can You Fool AI by Doing a 180? $\unicode{x2013}$ A Case Study on Authorship Analysis of Texts by Arata Osada [2.6954666679827137]
  This paper is an attempt at answering a twofold question covering the areas of ethics and authorship analysis. Firstly, we were interested in finding out whether it would be possible for an author identification system to correctly attribute works to authors if in the course of years they have undergone a major psychological transition. Secondly, and from the point of view of the evolution of an author's ethical values, we checked what it would mean if the authorship attribution system encounters difficulties in detecting single authorship.
  arXiv  Detail & Related papers  (2022-07-19T05:43:49Z)
• On the Usefulness of Embeddings, Clusters and Strings for Text Generator Evaluation [86.19634542434711]
  Mauve measures an information-theoretic divergence between two probability distributions over strings. We show that Mauve was right for the wrong reasons, and that its newly proposed divergence is not necessary for its high performance. We conclude that -- by encoding syntactic- and coherence-level features of text, while ignoring surface-level features -- such cluster-based substitutes to string distributions may simply be better for evaluating state-of-the-art language generators.
  arXiv  Detail & Related papers  (2022-05-31T17:58:49Z)
• Measuring Fairness of Text Classifiers via Prediction Sensitivity [63.56554964580627]
  ACCUMULATED PREDICTION SENSITIVITY measures fairness in machine learning models based on the model's prediction sensitivity to perturbations in input features. We show that the metric can be theoretically linked with a specific notion of group fairness (statistical parity) and individual fairness.
  arXiv  Detail & Related papers  (2022-03-16T15:00:33Z)
• Identifiability of Causal-based Fairness Notions: A State of the Art [4.157415305926584]
  Machine learning algorithms can produce biased outcome/prediction, typically, against minorities and under-represented sub-populations. This paper is a compilation of the major identifiability results which are of particular relevance for machine learning fairness.
  arXiv  Detail & Related papers  (2022-03-11T13:10:32Z)
• Learning Fair Node Representations with Graph Counterfactual Fairness [56.32231787113689]
  We propose graph counterfactual fairness, which considers the biases led by the above facts. We generate counterfactuals corresponding to perturbations on each node's and their neighbors' sensitive attributes. Our framework outperforms the state-of-the-art baselines in graph counterfactual fairness.
  arXiv  Detail & Related papers  (2022-01-10T21:43:44Z)
• FairCanary: Rapid Continuous Explainable Fairness [8.362098382773265]
  We present Quantile Demographic Drift (QDD), a novel model bias quantification metric. QDD is ideal for continuous monitoring scenarios, does not suffer from the statistical limitations of conventional threshold-based bias metrics. We incorporate QDD into a continuous model monitoring system, called FairCanary, that reuses existing explanations computed for each individual prediction.
  arXiv  Detail & Related papers  (2021-06-13T17:47:44Z)
• Fairness for Whom? Understanding the Reader's Perception of Fairness in Text Summarization [9.136419921943235]
  We study the interplay between the fairness notions and how readers perceive them in textual summaries. Standard ROUGE evaluation metrics are unable to quantify the perceived (un)fairness of the summaries.
  arXiv  Detail & Related papers  (2021-01-29T05:14:34Z)
• Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning [61.93730166203915]
  We argue that traditional notions of fairness are not sufficient when the model is vulnerable to adversarial attacks. We show that measuring robustness bias is a challenging task for DNNs and propose two methods to measure this form of bias.
  arXiv  Detail & Related papers  (2020-06-17T22:22:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.