Who Decides What Is Harmful? Content Moderation Policy Through A Multi-Agent Personalised Inference Framework

• URL: http://arxiv.org/abs/2605.01416v1
• Date: Sat, 02 May 2026 12:28:09 GMT
• Title: Who Decides What Is Harmful? Content Moderation Policy Through A Multi-Agent Personalised Inference Framework
• Abstract summary: This paper proposes an LLM-based multi-agent personalised inference framework that filters content based on sensitivity unique profiles of individual users.<n>Our architecture combines domain-specific Expert Agents, a Manager Agent for orchestrating content analysis and agent selection, and a Ghost Profile Agent for simulating user perspectives.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The increasing scale and complexity of online platforms raises critical policy questions around harmful content, digital well-being, and user autonomy. Traditional content moderation systems rely on centralised, top-down rules, often failing to accommodate the subjective nature of harm perception. This paper proposes an LLM-based multi-agent personalised inference framework that filters content based on unique sensitivity profiles of individual users. Our architecture combines domain-specific Expert Agents, a Manager Agent for orchestrating content analysis and agent selection, and a Ghost Profile Agent for simulating user perspectives, to inform moderation decisions. Evaluated against a range of non-personalised baselines, the system demonstrates up to a 32% improvement in accuracy, showing increased alignment with individual user sensitivities. Beyond technical performance, our framework provides policy-relevant insights for platform governance, providing a scalable way to reconcile moderation policies with societal and individual digital rights

Related papers

• Agentic AI: User Empowerment or Enclosure? [3.9727752701702728]
  We argue that the answer to whether Agentic AI will empower users is an open question.<n>We conduct a comparative case analysis of four more mature domains where similar forms of agency arose.<n>We identify this as depoliticization, a concept from political theory, here at work in technological systems.
  arXiv  Detail & Related papers  (2026-08-06T18:49:13Z)
• Using AI Agents to Automate Black-Box Audits of Personalization Algorithms at Scale [76.72846830885882]
  Personalization algorithms determine what content users encounter on online platforms.<n> auditing these systems is difficult because independent auditors have only black-box access to the algorithms.<n>We introduce a framework for black-box audits of personalization algorithms using generative AI agents as behavioral engines for synthetic accounts.
  arXiv  Detail & Related papers  (2026-06-29T18:25:09Z)
• Agent-Based User-Adaptive Filtering for Categorized Harassing Communication [0.0]
  We propose an agent-based framework for personalized filtering of categorized harassing communication in online social networks.<n>Unlike global moderation systems that apply uniform filtering rules, our approach models user-specific tolerance levels and preferences through adaptive filtering agents.
  arXiv  Detail & Related papers  (2026-02-28T01:52:48Z)
• Decision Quality Evaluation Framework at Pinterest [0.36944296923226316]
  The framework is centered on a high-trust Golden Set (GDS) curated by subject matter experts (SMEs)<n>We introduce an automated intelligent sampling pipeline that uses propensity scores to efficiently expand dataset coverage.<n>The framework enables a shift from subjective assessments to a data-driven and quantitative practice for managing content safety systems.
  arXiv  Detail & Related papers  (2026-02-17T18:45:55Z)
• Real-Time Personalized Content Adaptation through Matrix Factorization and Context-Aware Federated Learning [0.0]
  This study presents a multifaceted approach to enhancing user interaction and content relevance in social media platforms.<n>We introduce personalized LLM Federated Learning and Context-based Social Media models.<n>Key modules focus on categorizing user-generated content, computing user persona scores, and identifying relevant posts from friends networks.
  arXiv  Detail & Related papers  (2025-11-23T15:18:11Z)
• Personalized Reasoning: Just-In-Time Personalization and Why LLMs Fail At It [81.50711040539566]
  Current large language model (LLM) development treats task-solving and preference alignment as separate challenges.<n>We introduce PREFDISCO, an evaluation methodology that transforms static benchmarks into interactive personalization tasks.<n>Our framework creates scenarios where identical questions require different reasoning chains depending on user context.
  arXiv  Detail & Related papers  (2025-09-30T18:55:28Z)
• Customize Multi-modal RAI Guardrails with Precedent-based predictions [55.63757336900865]
  A multi-modal guardrail must effectively filter image content based on user-defined policies.<n>Existing fine-tuning methods typically condition predictions on pre-defined policies.<n>We propose to condition model's judgment on "precedents", which are the reasoning processes of prior data points similar to the given input.
  arXiv  Detail & Related papers  (2025-07-28T03:45:34Z)
• Trust-Oriented Adaptive Guardrails for Large Language Models [9.719986610417441]
  Guardrails are designed to ensure that large language models (LLMs) align with human values by moderating harmful or toxic responses.<n>This paper addresses a critical issue: existing guardrails lack a well-founded methodology to accommodate the diverse needs of different user groups.<n>We introduce an adaptive guardrail mechanism, to dynamically moderate access to sensitive content based on user trust metrics.
  arXiv  Detail & Related papers  (2024-08-16T18:07:48Z)
• Explainable Abuse Detection as Intent Classification and Slot Filling [66.80201541759409]
  We introduce the concept of policy-aware abuse detection, abandoning the unrealistic expectation that systems can reliably learn which phenomena constitute abuse from inspecting the data alone. We show how architectures for intent classification and slot filling can be used for abuse detection, while providing a rationale for model decisions.
  arXiv  Detail & Related papers  (2022-10-06T03:33:30Z)
• Personalized multi-faceted trust modeling to determine trust links in social media and its potential for misinformation management [61.88858330222619]
  We present an approach for predicting trust links between peers in social media. We propose a data-driven multi-faceted trust modeling which incorporates many distinct features for a comprehensive analysis. Illustrated in a trust-aware item recommendation task, we evaluate the proposed framework in the context of a large Yelp dataset.
  arXiv  Detail & Related papers  (2021-11-11T19:40:51Z)
• Privacy-Constrained Policies via Mutual Information Regularized Policy Gradients [54.98496284653234]
  We consider the task of training a policy that maximizes reward while minimizing disclosure of certain sensitive state variables through the actions. We solve this problem by introducing a regularizer based on the mutual information between the sensitive state and the actions. We develop a model-based estimator for optimization of privacy-constrained policies.
  arXiv  Detail & Related papers  (2020-12-30T03:22:35Z)
• Style Normalization and Restitution for Generalizable Person Re-identification [89.482638433932]
  We design a generalizable person ReID framework which trains a model on source domains yet is able to generalize/perform well on target domains. We propose a simple yet effective Style Normalization and Restitution (SNR) module. Our models empowered by the SNR modules significantly outperform the state-of-the-art domain generalization approaches on multiple widely-used person ReID benchmarks.
  arXiv  Detail & Related papers  (2020-05-22T07:15:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.