Effects of algorithmic flagging on fairness: quasi-experimental evidence from Wikipedia

• URL: http://arxiv.org/abs/2006.03121v2
• Date: Tue, 6 Apr 2021 00:14:34 GMT
• Title: Effects of algorithmic flagging on fairness: quasi-experimental evidence from Wikipedia
• Authors: Nathan TeBlunthuis, Benjamin Mako Hill, Aaron Halfaker
• Abstract summary: We analyze moderator behavior in Wikipedia as mediated by RCFilters, a system which displays social signals and algorithmic flags. We show that algorithmically flagged edits are reverted more often, especially those by established editors with positive social signals. Our results suggest that algorithmic flagging systems can lead to increased fairness in some contexts but that the relationship is complex and contingent.
• Score: 9.885409727425433
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Online community moderators often rely on social signals such as whether or not a user has an account or a profile page as clues that users may cause problems. Reliance on these clues can lead to overprofiling bias when moderators focus on these signals but overlook the misbehavior of others. We propose that algorithmic flagging systems deployed to improve the efficiency of moderation work can also make moderation actions more fair to these users by reducing reliance on social signals and making norm violations by everyone else more visible. We analyze moderator behavior in Wikipedia as mediated by RCFilters, a system which displays social signals and algorithmic flags, and estimate the causal effect of being flagged on moderator actions. We show that algorithmically flagged edits are reverted more often, especially those by established editors with positive social signals, and that flagging decreases the likelihood that moderation actions will be undone. Our results suggest that algorithmic flagging systems can lead to increased fairness in some contexts but that the relationship is complex and contingent.

Related papers

• Measuring Strategization in Recommendation: Users Adapt Their Behavior to Shape Future Content [66.71102704873185]
  We test for user strategization by conducting a lab experiment and survey. We find strong evidence of strategization across outcome metrics, including participants' dwell time and use of "likes" Our findings suggest that platforms cannot ignore the effect of their algorithms on user behavior.
  arXiv  Detail & Related papers  (2024-05-09T07:36:08Z)
• Can Probabilistic Feedback Drive User Impacts in Online Platforms? [26.052963782865294]
  A common explanation for negative user impacts of content recommender systems is misalignment between the platform's objective and user welfare. In this work, we show that misalignment in the platform's objective is not the only potential cause of unintended impacts on users. The source of these user impacts is that different pieces of content may generate observable user reactions (feedback information) at different rates.
  arXiv  Detail & Related papers  (2024-01-10T18:12:31Z)
• User Strategization and Trustworthy Algorithms [81.82279667028423]
  We show that user strategization can actually help platforms in the short term. We then show that it corrupts platforms' data and ultimately hurts their ability to make counterfactual decisions.
  arXiv  Detail & Related papers  (2023-12-29T16:09:42Z)
• Algorithms, Incentives, and Democracy [0.0]
  We show how optimal classification by an algorithm designer can affect the distribution of behavior in a population. We then look at the effect of democratizing the rewards and punishments, or stakes, to the algorithmic classification to consider how a society can potentially stem (or facilitate!) predatory classification.
  arXiv  Detail & Related papers  (2023-07-05T14:22:01Z)
• Adversaries with Limited Information in the Friedkin--Johnsen Model [25.89905526128351]
  In recent years, online social networks have been the target of adversaries who seek to introduce discord into societies. We present approximation algorithms for detecting a small set of users who are highly influential for the disagreement and polarization in the network.
  arXiv  Detail & Related papers  (2023-06-17T10:32:02Z)
• Explainable Abuse Detection as Intent Classification and Slot Filling [66.80201541759409]
  We introduce the concept of policy-aware abuse detection, abandoning the unrealistic expectation that systems can reliably learn which phenomena constitute abuse from inspecting the data alone. We show how architectures for intent classification and slot filling can be used for abuse detection, while providing a rationale for model decisions.
  arXiv  Detail & Related papers  (2022-10-06T03:33:30Z)
• Mathematical Framework for Online Social Media Auditing [5.384630221560811]
  Social media platforms (SMPs) leverage algorithmic filtering (AF) as a means of selecting the content that constitutes a user's feed with the aim of maximizing their rewards. Selectively choosing the contents to be shown on the user's feed may yield a certain extent of influence, either minor or major, on the user's decision-making. We mathematically formalize this framework and utilize it to construct a data-driven statistical auditing procedure to regulate AF from deflecting users' beliefs over time, along with sample complexity guarantees.
  arXiv  Detail & Related papers  (2022-09-12T19:04:14Z)
• D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias [57.87117733071416]
  We propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases. A user can detect the presence of bias against a group by identifying unfair causal relationships in the causal network. For each interaction, say weakening/deleting a biased causal edge, the system uses a novel method to simulate a new (debiased) dataset.
  arXiv  Detail & Related papers  (2022-08-10T03:41:48Z)
• Information Consumption and Social Response in a Segregated Environment: the Case of Gab [74.5095691235917]
  This work provides a characterization of the interaction patterns within Gab around the COVID-19 topic. We find that there are no strong statistical differences in the social response to questionable and reliable content. Our results provide insights toward the understanding of coordinated inauthentic behavior and on the early-warning of information operation.
  arXiv  Detail & Related papers  (2020-06-03T11:34:25Z)
• DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correction [96.90215318875859]
  We show that bootstrapping-based Q-learning algorithms do not necessarily benefit from corrective feedback. We propose a new algorithm, DisCor, which computes an approximation to this optimal distribution and uses it to re-weight the transitions used for training.
  arXiv  Detail & Related papers  (2020-03-16T16:18:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.