Related papers: Content Moderation and the Formation of Online Communities: A Theoretical Framework

Content Moderation and the Formation of Online Communities: A Theoretical Framework

URL: http://arxiv.org/abs/2310.10573v1
Date: Mon, 16 Oct 2023 16:49:44 GMT
Title: Content Moderation and the Formation of Online Communities: A Theoretical Framework
Authors: Cynthia Dwork, Chris Hays, Jon Kleinberg, Manish Raghavan
Abstract summary: We study the impact of content moderation policies in online communities. We first characterize the effectiveness of a natural class of moderation policies for creating and sustaining stable communities.
Score: 7.900694093691988
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: We study the impact of content moderation policies in online communities. In our theoretical model, a platform chooses a content moderation policy and individuals choose whether or not to participate in the community according to the fraction of user content that aligns with their preferences. The effects of content moderation, at first blush, might seem obvious: it restricts speech on a platform. However, when user participation decisions are taken into account, its effects can be more subtle $\unicode{x2013}$ and counter-intuitive. For example, our model can straightforwardly demonstrate how moderation policies may increase participation and diversify content available on the platform. In our analysis, we explore a rich set of interconnected phenomena related to content moderation in online communities. We first characterize the effectiveness of a natural class of moderation policies for creating and sustaining stable communities. Building on this, we explore how resource-limited or ideological platforms might set policies, how communities are affected by differing levels of personalization, and competition between platforms. Our model provides a vocabulary and mathematically tractable framework for analyzing platform decisions about content moderation.

Related papers

Let Community Rules Be Reflected in Online Content Moderation [2.4717834653693083]
This study proposes a community rule-based content moderation framework. It integrates community rules into the moderation of user-generated content. In particular, incorporating community rules substantially enhances model performance in content moderation.
arXiv Detail & Related papers (2024-08-21T23:38:02Z)
Content Moderation Justice and Fairness on Social Media: Comparisons Across Different Contexts and Platforms [23.735552021636245]
We conduct an online experiment on 200 American social media users of Reddit and Twitter. We find that retributive moderation delivers higher justice and fairness for commercially moderated platforms in illegal violations. We discuss the opportunities for platform policymaking to improve moderation system design.
arXiv Detail & Related papers (2024-03-09T22:50:06Z)
Content Moderation on Social Media in the EU: Insights From the DSA Transparency Database [0.0]
Digital Services Act (DSA) requires large social media platforms in the EU to provide clear and specific information whenever they restrict access to certain content. Statements of Reasons (SoRs) are collected in the DSA Transparency Database to ensure transparency and scrutiny of content moderation decisions. We empirically analyze 156 million SoRs within an observation period of two months to provide an early look at content moderation decisions of social media platforms in the EU.
arXiv Detail & Related papers (2023-12-07T16:56:19Z)
Analyzing Norm Violations in Live-Stream Chat [49.120561596550395]
We study the first NLP study dedicated to detecting norm violations in conversations on live-streaming platforms. We define norm violation categories in live-stream chats and annotate 4,583 moderated comments from Twitch. Our results show that appropriate contextual information can boost moderation performance by 35%.
arXiv Detail & Related papers (2023-05-18T05:58:27Z)
A User-Driven Framework for Regulating and Auditing Social Media [94.70018274127231]
We propose that algorithmic filtering should be regulated with respect to a flexible, user-driven baseline. We require that the feeds a platform filters contain "similar" informational content as their respective baseline feeds. We present an auditing procedure that checks whether a platform honors this requirement.
arXiv Detail & Related papers (2023-04-20T17:53:34Z)
Reliable Decision from Multiple Subtasks through Threshold Optimization: Content Moderation in the Wild [7.176020195419459]
Social media platforms struggle to protect users from harmful content through content moderation. These platforms have recently leveraged machine learning models to cope with the vast amount of user-generated content daily. Third-party content moderation services provide prediction scores of multiple subtasks, such as predicting the existence of underage personnel, rude gestures, or weapons. We introduce a simple yet effective threshold optimization method that searches the optimal thresholds of the multiple subtasks to make a reliable moderation decision in a cost-effective way.
arXiv Detail & Related papers (2022-08-16T03:51:43Z)
SoK: Content Moderation in Social Media, from Guidelines to Enforcement, and Research to Practice [9.356143195807064]
We study the 14 most popular social media content moderation guidelines and practices in the US. We identify the differences between the content moderation employed in mainstream social media platforms compared to fringe platforms. We highlight why platforms should shift from a one-size-fits-all model to a more inclusive model.
arXiv Detail & Related papers (2022-06-29T18:48:04Z)
This Must Be the Place: Predicting Engagement of Online Communities in a Large-scale Distributed Campaign [70.69387048368849]
We study the behavior of communities with millions of active members. We develop a hybrid model, combining textual cues, community meta-data, and structural properties. We demonstrate the applicability of our model through Reddit's r/place a large-scale online experiment.
arXiv Detail & Related papers (2022-01-14T08:23:16Z)
News consumption and social media regulations policy [70.31753171707005]
We analyze two social media that enforced opposite moderation methods, Twitter and Gab, to assess the interplay between news consumption and content regulation. Our results show that the presence of moderation pursued by Twitter produces a significant reduction of questionable content. The lack of clear regulation on Gab results in the tendency of the user to engage with both types of content, showing a slight preference for the questionable ones which may account for a dissing/endorsement behavior.
arXiv Detail & Related papers (2021-06-07T19:26:32Z)
Detecting Harmful Content On Online Platforms: What Platforms Need Vs. Where Research Efforts Go [44.774035806004214]
harmful content on online platforms comes in many different forms including hate speech, offensive language, bullying and harassment, misinformation, spam, violence, graphic content, sexual abuse, self harm, and many other. Online platforms seek to moderate such content to limit societal harm, to comply with legislation, and to create a more inclusive environment for their users. There is currently a dichotomy between what types of harmful content online platforms seek to curb, and what research efforts there are to automatically detect such content.
arXiv Detail & Related papers (2021-02-27T08:01:10Z)
Information Consumption and Social Response in a Segregated Environment: the Case of Gab [74.5095691235917]
This work provides a characterization of the interaction patterns within Gab around the COVID-19 topic. We find that there are no strong statistical differences in the social response to questionable and reliable content. Our results provide insights toward the understanding of coordinated inauthentic behavior and on the early-warning of information operation.
arXiv Detail & Related papers (2020-06-03T11:34:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.