Content Moderation and the Formation of Online Communities: A Theoretical Framework

• URL: http://arxiv.org/abs/2310.10573v1
• Date: Mon, 16 Oct 2023 16:49:44 GMT
• Title: Content Moderation and the Formation of Online Communities: A Theoretical Framework
• Authors: Cynthia Dwork, Chris Hays, Jon Kleinberg, Manish Raghavan
• Abstract summary: We study the impact of content moderation policies in online communities. We first characterize the effectiveness of a natural class of moderation policies for creating and sustaining stable communities.
• Score: 7.900694093691988
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the impact of content moderation policies in online communities. In our theoretical model, a platform chooses a content moderation policy and individuals choose whether or not to participate in the community according to the fraction of user content that aligns with their preferences. The effects of content moderation, at first blush, might seem obvious: it restricts speech on a platform. However, when user participation decisions are taken into account, its effects can be more subtle $\unicode{x2013}$ and counter-intuitive. For example, our model can straightforwardly demonstrate how moderation policies may increase participation and diversify content available on the platform. In our analysis, we explore a rich set of interconnected phenomena related to content moderation in online communities. We first characterize the effectiveness of a natural class of moderation policies for creating and sustaining stable communities. Building on this, we explore how resource-limited or ideological platforms might set policies, how communities are affected by differing levels of personalization, and competition between platforms. Our model provides a vocabulary and mathematically tractable framework for analyzing platform decisions about content moderation.

Related papers

• Characterizing the Fragmentation of the Social Media Ecosystem [39.58317527488534]
  We use a dataset of 126M URLs posted by nearly 6M users on nine social media platforms. We find a clear separation between mainstream and alt-tech platforms. These findings outline the main dimensions defining the fragmentation and polarization of the social media ecosystem.
  arXiv  Detail & Related papers  (2024-11-25T18:45:03Z)
• Let Community Rules Be Reflected in Online Content Moderation [2.4717834653693083]
  This study proposes a community rule-based content moderation framework. It integrates community rules into the moderation of user-generated content. In particular, incorporating community rules substantially enhances model performance in content moderation.
  arXiv  Detail & Related papers  (2024-08-21T23:38:02Z)
• User Welfare Optimization in Recommender Systems with Competing Content Creators [65.25721571688369]
  In this study, we perform system-side user welfare optimization under a competitive game setting among content creators. We propose an algorithmic solution for the platform, which dynamically computes a sequence of weights for each user based on their satisfaction of the recommended content. These weights are then utilized to design mechanisms that adjust the recommendation policy or the post-recommendation rewards, thereby influencing creators' content production strategies.
  arXiv  Detail & Related papers  (2024-04-28T21:09:52Z)
• Content Moderation on Social Media in the EU: Insights From the DSA Transparency Database [0.0]
  Digital Services Act (DSA) requires large social media platforms in the EU to provide clear and specific information whenever they restrict access to certain content. Statements of Reasons (SoRs) are collected in the DSA Transparency Database to ensure transparency and scrutiny of content moderation decisions. We empirically analyze 156 million SoRs within an observation period of two months to provide an early look at content moderation decisions of social media platforms in the EU.
  arXiv  Detail & Related papers  (2023-12-07T16:56:19Z)
• Analyzing Norm Violations in Live-Stream Chat [49.120561596550395]
  We study the first NLP study dedicated to detecting norm violations in conversations on live-streaming platforms. We define norm violation categories in live-stream chats and annotate 4,583 moderated comments from Twitch. Our results show that appropriate contextual information can boost moderation performance by 35%.
  arXiv  Detail & Related papers  (2023-05-18T05:58:27Z)
• A User-Driven Framework for Regulating and Auditing Social Media [94.70018274127231]
  We propose that algorithmic filtering should be regulated with respect to a flexible, user-driven baseline. We require that the feeds a platform filters contain "similar" informational content as their respective baseline feeds. We present an auditing procedure that checks whether a platform honors this requirement.
  arXiv  Detail & Related papers  (2023-04-20T17:53:34Z)
• Reliable Decision from Multiple Subtasks through Threshold Optimization: Content Moderation in the Wild [7.176020195419459]
  Social media platforms struggle to protect users from harmful content through content moderation. These platforms have recently leveraged machine learning models to cope with the vast amount of user-generated content daily. Third-party content moderation services provide prediction scores of multiple subtasks, such as predicting the existence of underage personnel, rude gestures, or weapons. We introduce a simple yet effective threshold optimization method that searches the optimal thresholds of the multiple subtasks to make a reliable moderation decision in a cost-effective way.
  arXiv  Detail & Related papers  (2022-08-16T03:51:43Z)
• SoK: Content Moderation in Social Media, from Guidelines to Enforcement, and Research to Practice [9.356143195807064]
  We study the 14 most popular social media content moderation guidelines and practices in the US. We identify the differences between the content moderation employed in mainstream social media platforms compared to fringe platforms. We highlight why platforms should shift from a one-size-fits-all model to a more inclusive model.
  arXiv  Detail & Related papers  (2022-06-29T18:48:04Z)
• This Must Be the Place: Predicting Engagement of Online Communities in a Large-scale Distributed Campaign [70.69387048368849]
  We study the behavior of communities with millions of active members. We develop a hybrid model, combining textual cues, community meta-data, and structural properties. We demonstrate the applicability of our model through Reddit's r/place a large-scale online experiment.
  arXiv  Detail & Related papers  (2022-01-14T08:23:16Z)
• News consumption and social media regulations policy [70.31753171707005]
  We analyze two social media that enforced opposite moderation methods, Twitter and Gab, to assess the interplay between news consumption and content regulation. Our results show that the presence of moderation pursued by Twitter produces a significant reduction of questionable content. The lack of clear regulation on Gab results in the tendency of the user to engage with both types of content, showing a slight preference for the questionable ones which may account for a dissing/endorsement behavior.
  arXiv  Detail & Related papers  (2021-06-07T19:26:32Z)
• Information Consumption and Social Response in a Segregated Environment: the Case of Gab [74.5095691235917]
  This work provides a characterization of the interaction patterns within Gab around the COVID-19 topic. We find that there are no strong statistical differences in the social response to questionable and reliable content. Our results provide insights toward the understanding of coordinated inauthentic behavior and on the early-warning of information operation.
  arXiv  Detail & Related papers  (2020-06-03T11:34:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.