Auditing Algorithmic Personalization in TikTok Comment Sections

• URL: http://arxiv.org/abs/2603.25061v1
• Date: Thu, 26 Mar 2026 05:52:41 GMT
• Title: Auditing Algorithmic Personalization in TikTok Comment Sections
• Abstract summary: We trained sock-puppet accounts to exhibit left-leaning or right-leaning preferences and validated 17 of them by analyzing the videos recommended on their For You Pages.<n>We then scraped the comment sections shown to these trained partisan accounts, along with five cold-start accounts, across 65 politically neutral videos.<n>We find that while the composition of top comments remains largely consistent for all videos, ranking divergence between accounts from different political groups is significantly greater than that observed within the same group for some videos.
• Score: 4.784448295763737
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Personalization algorithms are ubiquitous in modern social computing systems, yet their effects on comment sections remain underexplored. In this work, we conducted an algorithmic auditing experiment to examine comment personalization on TikTok. We trained sock-puppet accounts to exhibit left-leaning or right-leaning preferences and successfully validated 17 of them by analyzing the videos recommended on their For You Pages. We then scraped the comment sections shown to these trained partisan accounts, along with five cold-start accounts, across 65 politically neutral videos related to the 2024 U.S. presidential election that contain abundant discussions from both left-leaning and right-leaning perspectives. We find that while the composition of top comments remains largely consistent for all videos, ranking divergence between accounts from different political groups is significantly greater than that observed within the same group for some videos. This effect is strongly correlated with video-level metrics such as comment volume, engagement inequality, and partisan skew in the comment sections. Furthermore, through an exploratory case study, we find preliminary evidence that personalization can result in comment exposure aligned with an account's political leaning. However, this pattern is not universal, suggesting that the extent of politically oriented comment personalization is context-dependent.

Related papers

• Auditing Differential Visibility of Political Content on TikTok [0.0]
  We find no evidence of moderate-to-large reach suppression on any topic.<n>Oppositional content earns more engagement per view rather than less reach.
  arXiv  Detail & Related papers  (2026-07-19T17:48:37Z)
• Using AI Agents to Automate Black-Box Audits of Personalization Algorithms at Scale [76.72846830885882]
  Personalization algorithms determine what content users encounter on online platforms.<n> auditing these systems is difficult because independent auditors have only black-box access to the algorithms.<n>We introduce a framework for black-box audits of personalization algorithms using generative AI agents as behavioral engines for synthetic accounts.
  arXiv  Detail & Related papers  (2026-06-29T18:25:09Z)
• How Large Language Models play humans in online conversations: a simulated study of the 2016 US politics on Reddit [0.0]
  Large Language Models (LLMs) have recently emerged as powerful tools for natural language generation.<n>We evaluate the performance of LLMs in replicating user-generated content within a real-world, divisive scenario: Reddit conversations during the 2016 US Presidential election.<n>We find GPT-4 is able to produce realistic comments, both in favor of or against the candidate supported by the community, yet tending to create consensus more easily than dissent.
  arXiv  Detail & Related papers  (2025-06-23T08:54:32Z)
• TikTok's recommendations skewed towards Republican content during the 2024 U.S. presidential race [1.340487372205839]
  TikTok is a major force among social media platforms with over a billion monthly active users worldwide and 170 million in the U.S.<n>Despite concerns, there is scant research investigating TikTok's recommendation algorithm for political biases.<n>We fill this gap by conducting 323 independent algorithmic audit experiments testing partisan content recommendations in the lead-up to the 2024 U.S. presidential elections.
  arXiv  Detail & Related papers  (2025-01-29T18:24:20Z)
• Filter bubbles and affective polarization in user-personalized large language model outputs [0.15540058359482856]
  Large language models (LLMs) have led to a push for increased personalization of model outputs to individual users. We explore how prompting a leading large language model, ChatGPT-3.5, with a user's political affiliation prior to asking factual questions leads to differing results.
  arXiv  Detail & Related papers  (2023-10-31T18:19:28Z)
• Engagement, User Satisfaction, and the Amplification of Divisive Content on Social Media [22.206581957044513]
  We find that Twitter's engagement-based ranking algorithm amplifies emotionally charged, out-group hostile content.<n>We explore the implications of an alternative approach that ranks content based on users' stated preferences.
  arXiv  Detail & Related papers  (2023-05-26T13:57:30Z)
• A User-Driven Framework for Regulating and Auditing Social Media [94.70018274127231]
  We propose that algorithmic filtering should be regulated with respect to a flexible, user-driven baseline. We require that the feeds a platform filters contain "similar" informational content as their respective baseline feeds. We present an auditing procedure that checks whether a platform honors this requirement.
  arXiv  Detail & Related papers  (2023-04-20T17:53:34Z)
• The Face of Populism: Examining Differences in Facial Emotional Expressions of Political Leaders Using Machine Learning [50.24983453990065]
  We use a deep-learning approach to process a sample of 220 YouTube videos of political leaders from 15 different countries.<n>We observe statistically significant differences in the average score of negative emotions between groups of leaders with varying degrees of populist rhetoric.
  arXiv  Detail & Related papers  (2023-04-19T18:32:49Z)
• Whose Opinions Do Language Models Reflect? [88.35520051971538]
  We investigate the opinions reflected by language models (LMs) by leveraging high-quality public opinion polls and their associated human responses. We find substantial misalignment between the views reflected by current LMs and those of US demographic groups. Our analysis confirms prior observations about the left-leaning tendencies of some human feedback-tuned LMs.
  arXiv  Detail & Related papers  (2023-03-30T17:17:08Z)
• Algorithmic Amplification of Politics on Twitter [17.631887805091733]
  We provide quantitative evidence from a massive-scale randomized experiment on the Twitter platform. We studied Tweets by elected legislators from major political parties in 7 countries. In 6 out of 7 countries studied, the mainstream political right enjoys higher algorithmic amplification than the mainstream political left.
  arXiv  Detail & Related papers  (2021-10-21T09:25:39Z)
• News consumption and social media regulations policy [70.31753171707005]
  We analyze two social media that enforced opposite moderation methods, Twitter and Gab, to assess the interplay between news consumption and content regulation. Our results show that the presence of moderation pursued by Twitter produces a significant reduction of questionable content. The lack of clear regulation on Gab results in the tendency of the user to engage with both types of content, showing a slight preference for the questionable ones which may account for a dissing/endorsement behavior.
  arXiv  Detail & Related papers  (2021-06-07T19:26:32Z)
• Right and left, partisanship predicts (asymmetric) vulnerability to misinformation [71.46564239895892]
  We analyze the relationship between partisanship, echo chambers, and vulnerability to online misinformation by studying news sharing behavior on Twitter. We find that vulnerability to misinformation is most strongly influenced by partisanship for both left- and right-leaning users.
  arXiv  Detail & Related papers  (2020-10-04T01:36:14Z)
• Echo Chambers on Social Media: A comparative analysis [64.2256216637683]
  We introduce an operational definition of echo chambers and perform a massive comparative analysis on 1B pieces of contents produced by 1M users on four social media platforms. We infer the leaning of users about controversial topics and reconstruct their interaction networks by analyzing different features. We find support for the hypothesis that platforms implementing news feed algorithms like Facebook may elicit the emergence of echo-chambers.
  arXiv  Detail & Related papers  (2020-04-20T20:00:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.