Protecting Young Users on Social Media: Evaluating the Effectiveness of Content Moderation and Legal Safeguards on Video Sharing Platforms

• URL: http://arxiv.org/abs/2505.11160v1
• Date: Fri, 16 May 2025 12:06:42 GMT
• Title: Protecting Young Users on Social Media: Evaluating the Effectiveness of Content Moderation and Legal Safeguards on Video Sharing Platforms
• Authors: Fatmaelzahraa Eltaher, Rahul Krishna Gajula, Luis Miralles-Pechuán, Patrick Crotty, Juan Martínez-Otero, Christina Thorpe, Susan McKeever,
• Abstract summary: We evaluated the effectiveness of video moderation for different age groups on TikTok, YouTube, and Instagram.<n>For passive scrolling, accounts assigned to the age 13 group encountered videos that were deemed harmful more frequently and quickly than those assigned to the age 18 group.<n>Exposure occurred without user-initiated searches, indicating weaknesses in the algorithmic filtering systems.
• Score: 0.8198234257428011
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video-sharing social media platforms, such as TikTok, YouTube, and Instagram, implement content moderation policies aimed at reducing exposure to harmful videos among minor users. As video has become the dominant and most immersive form of online content, understanding how effectively this medium is moderated for younger audiences is urgent. In this study, we evaluated the effectiveness of video moderation for different age groups on three of the main video-sharing platforms: TikTok, YouTube, and Instagram. We created experimental accounts for the children assigned ages 13 and 18. Using these accounts, we evaluated 3,000 videos served up by the social media platforms, in passive scrolling and search modes, recording the frequency and speed at which harmful videos were encountered. Each video was manually assessed for level and type of harm, using definitions from a unified framework of harmful content. The results show that for passive scrolling or search-based scrolling, accounts assigned to the age 13 group encountered videos that were deemed harmful, more frequently and quickly than those assigned to the age 18 group. On YouTube, 15\% of recommended videos to 13-year-old accounts during passive scrolling were assessed as harmful, compared to 8.17\% for 18-year-old accounts. On YouTube, videos labelled as harmful appeared within an average of 3:06 minutes of passive scrolling for the younger age group. Exposure occurred without user-initiated searches, indicating weaknesses in the algorithmic filtering systems. These findings point to significant gaps in current video moderation practices by social media platforms. Furthermore, the ease with which underage users can misrepresent their age demonstrates the urgent need for more robust verification methods.

Related papers

• Catching Dark Signals in Algorithms: Unveiling Audiovisual and Thematic Markers of Unsafe Content Recommended for Children and Teenagers [13.39320891153433]
  The prevalence of short form video platforms, combined with the ineffectiveness of age verification mechanisms, raises concerns about the potential harms facing children and teenagers in an algorithm-moderated online environment.<n>We conducted multimodal feature analysis and thematic topic modeling of 4,492 short videos recommended to children and teenagers on Instagram Reels, TikTok, and YouTube Shorts.<n>This feature-level and content-level analysis revealed that unsafe (i.e., problematic, mentally distressing) short videos possess darker visual features and contain explicitly harmful content and implicit harm from anxiety-inducing ordinary content.
  arXiv  Detail & Related papers  (2025-07-16T18:41:42Z)
• Harmful YouTube Video Detection: A Taxonomy of Online Harm and MLLMs as Alternative Annotators [0.0]
  This study advances measures and methods to detect harm in video content. We develop a comprehensive taxonomy for online harm on video platforms, categorizing it into six categories: Information, Hate and harassment, Addictive, Clickbait, Sexual, and Physical harms. We analyze 19,422 YouTube videos using 14 image frames, 1 thumbnail, and text metadata, comparing the accuracy of crowdworkers (Mturk) and GPT-4-Turbo.
  arXiv  Detail & Related papers  (2024-11-06T23:48:30Z)
• Beyond Raw Videos: Understanding Edited Videos with Large Multimodal Model [62.38322742493649]
  We build a video VQA benchmark covering editing categories, i.e., effect, funny, meme, and game. Most of the open-source video LMMs perform poorly on the benchmark, indicating a huge domain gap between edited short videos on social media and regular raw videos. To improve the generalization ability of LMMs, we collect a training set for the proposed benchmark based on both Panda-70M/WebVid raw videos and small-scale TikTok/CapCut edited videos.
  arXiv  Detail & Related papers  (2024-06-15T03:28:52Z)
• An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software [64.367830425115]
  Social media platforms are being increasingly misused to spread toxic content, including hate speech, malicious advertising, and pornography. Despite tremendous efforts in developing and deploying content moderation methods, malicious users can evade moderation by embedding texts into images. We propose a metamorphic testing framework for content moderation software.
  arXiv  Detail & Related papers  (2023-08-18T20:33:06Z)
• How to Train Your YouTube Recommender to Avoid Unwanted Videos [51.6864681332515]
  "Not interested" and "Don't recommend channel" buttons allow users to indicate disinterest when presented with unwanted recommendations. We simulated YouTube users with sock puppet agents. We found that the "Not interested" button worked best, significantly reducing such recommendations in all topics tested.
  arXiv  Detail & Related papers  (2023-07-27T00:21:29Z)
• Malicious or Benign? Towards Effective Content Moderation for Children's Videos [1.0323063834827415]
  This paper introduces our toolkit Malicious or Benign for promoting research on automated content moderation of children's videos. We present 1) a customizable annotation tool for videos, 2) a new dataset with difficult to detect test cases of malicious content, and 3) a benchmark suite of state-of-the-art video classification models.
  arXiv  Detail & Related papers  (2023-05-24T20:33:38Z)
• Analyzing Norm Violations in Live-Stream Chat [49.120561596550395]
  We study the first NLP study dedicated to detecting norm violations in conversations on live-streaming platforms. We define norm violation categories in live-stream chats and annotate 4,583 moderated comments from Twitch. Our results show that appropriate contextual information can boost moderation performance by 35%.
  arXiv  Detail & Related papers  (2023-05-18T05:58:27Z)
• YouTubers Not madeForKids: Detecting Channels Sharing Inappropriate Videos Targeting Children [3.936965297430477]
  We study YouTube channels found to post suitable or disturbing videos targeting kids in the past. We identify a clear discrepancy between what YouTube assumes and flags as inappropriate content and channel, vs. what is found to be disturbing content and still available on the platform.
  arXiv  Detail & Related papers  (2022-05-27T10:34:15Z)
• Attacking Video Recognition Models with Bullet-Screen Comments [79.53159486470858]
  We introduce a novel adversarial attack, which attacks video recognition models with bullet-screen comment (BSC) attacks. BSCs can be regarded as a kind of meaningful patch, adding it to a clean video will not affect people' s understanding of the video content, nor will arouse people' s suspicion.
  arXiv  Detail & Related papers  (2021-10-29T08:55:50Z)
• VPN: Video Provenance Network for Robust Content Attribution [72.12494245048504]
  We present VPN - a content attribution method for recovering provenance information from videos shared online. We learn a robust search embedding for matching such video, using full-length or truncated video queries. Once matched against a trusted database of video clips, associated information on the provenance of the clip is presented to the user.
  arXiv  Detail & Related papers  (2021-09-21T09:07:05Z)
• Characterizing Abhorrent, Misinformative, and Mistargeted Content on YouTube [1.9138099871648453]
  We study the degree of problematic content on YouTube and the role of the recommendation algorithm in the dissemination of such content. Our analysis reveals that young children are likely to encounter disturbing content when they randomly browse the platform. We find that Incel activity is increasing over time and that platforms may play an active role in steering users towards extreme content.
  arXiv  Detail & Related papers  (2021-05-20T15:10:48Z)
• Hate, Obscenity, and Insults: Measuring the Exposure of Children to Inappropriate Comments in YouTube [8.688428251722911]
  In this paper, we investigate the exposure of young users to inappropriate comments posted on YouTube videos targeting this demographic. We collected a large-scale dataset of approximately four million records and studied the presence of five age-inappropriate categories and the amount of exposure to each category. Using natural language processing and machine learning techniques, we constructed ensemble classifiers that achieved high accuracy in detecting inappropriate comments.
  arXiv  Detail & Related papers  (2021-03-03T20:15:22Z)
• Learning Person Re-identification Models from Videos with Weak Supervision [53.53606308822736]
  We introduce the problem of learning person re-identification models from videos with weak supervision. We propose a multiple instance attention learning framework for person re-identification using such video-level labels. The attention weights are obtained based on all person images instead of person tracklets in a video, making our learned model less affected by noisy annotations.
  arXiv  Detail & Related papers  (2020-07-21T07:23:32Z)
• Quantifying the Vulnerabilities of the Online Public Square to Adversarial Manipulation Tactics [43.98568073610101]
  We use a social media model to quantify the impacts of several adversarial manipulation tactics on the quality of content. We find that the presence of influential accounts, a hallmark of social media, exacerbates the vulnerabilities of online communities to manipulation. These insights suggest countermeasures that platforms could employ to increase the resilience of social media users to manipulation.
  arXiv  Detail & Related papers  (2019-07-13T21:12:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.