Large Language Models in the Abuse Detection Pipeline

• URL: http://arxiv.org/abs/2604.00323v1
• Date: Tue, 31 Mar 2026 23:42:06 GMT
• Title: Large Language Models in the Abuse Detection Pipeline
• Abstract summary: Online abuse has grown increasingly complex, spanning toxic language, harassment, manipulation, and fraudulent behavior.<n>Traditional machine-learning approaches dependent on static classifiers and labor-intensive labeling struggle to keep pace with evolving threat patterns and nuanced policy requirements.<n>Large Language Models introduce new capabilities for contextual reasoning, policy interpretation, explanation generation, and cross-modal understanding.
• Score: 0.23301643766310373
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online abuse has grown increasingly complex, spanning toxic language, harassment, manipulation, and fraudulent behavior. Traditional machine-learning approaches dependent on static classifiers and labor-intensive labeling struggle to keep pace with evolving threat patterns and nuanced policy requirements. Large Language Models introduce new capabilities for contextual reasoning, policy interpretation, explanation generation, and cross-modal understanding, enabling them to support multiple stages of modern safety systems. This survey provides a lifecycle-oriented analysis of how LLMs are being integrated into the Abuse Detection Lifecycle (ADL), which we define across four stages: (I) Label \& Feature Generation, (II) Detection, (III) Review \& Appeals, and (IV) Auditing \& Governance. For each stage, we synthesize emerging research and industry practices, highlight architectural considerations for production deployment, and examine the strengths and limitations of LLM-driven approaches. We conclude by outlining key challenges including latency, cost-efficiency, determinism, adversarial robustness, and fairness and discuss future research directions needed to operationalize LLMs as reliable, accountable components of large-scale abuse-detection and governance systems.

Related papers

• Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability [65.56750741053266]
  Large language models (LLMs) have transformed misinformation from a content-centric problem into a broader ecosystem-level security challenge.<n>In this paper, we introduce a role-layer framework to unify these risks and defenses.<n>The role characterizes LLMs as attackers, defenders, and vulnerable components of verification systems, while the layer dimension covers content, social contexts, evidence environments, and verification.
  arXiv  Detail & Related papers  (2026-07-11T17:04:17Z)
• Grounded in Reality: Learning and Deploying Proactive LLM from Offline Logs [72.08224879435762]
  textttLearn-to-Ask is a simulator-free framework for learning and deploying proactive dialogue agents.<n>Our approach culminates in the successful deployment of LLMs into a live, large-scale online AI service.
  arXiv  Detail & Related papers  (2025-10-29T12:08:07Z)
• Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey [69.45421620616486]
  This work presents the first structured taxonomy and analysis of discrete tokenization methods designed for large language models (LLMs)<n>We categorize 8 representative VQ variants that span classical and modern paradigms and analyze their algorithmic principles, training dynamics, and integration challenges with LLM pipelines.<n>We identify key challenges including codebook collapse, unstable gradient estimation, and modality-specific encoding constraints.
  arXiv  Detail & Related papers  (2025-07-21T10:52:14Z)
• On the Surprising Efficacy of LLMs for Penetration-Testing [3.11537581064266]
  The paper thoroughly reviews the evolution of Large Language Models (LLMs) in penetration testing.<n>It showcases their application across various offensive security tasks and covering broader phases of the cyber kill chain.<n>The paper identifies and discusses significant obstacles impeding wider adoption and safe deployment.
  arXiv  Detail & Related papers  (2025-07-01T15:01:18Z)
• MLLM-CL: Continual Learning for Multimodal Large Language Models [39.19456474036905]
  We introduce MLLM-CL, a novel benchmark encompassing domain and ability continual learning.<n>We propose preventing catastrophic interference through parameter isolation and an MLLM-based routing mechanism.<n>Our approach can integrate domain-specific knowledge and functional abilities with minimal forgetting, significantly outperforming existing methods.
  arXiv  Detail & Related papers  (2025-06-05T17:58:13Z)
• Large Language Models and Their Applications in Roadway Safety and Mobility Enhancement: A Comprehensive Review [14.611584622270405]
  This paper reviews the application and customization of Large Language Models (LLMs) for enhancing roadway safety and mobility.<n>A key focus is how LLMs are adapted -- via architectural, training, prompting, and multimodal strategies -- to bridge the "modality gap" with transportation's unique-temporal and physical data.<n>Despite significant potential, challenges persist regarding inherent LLM limitations (hallucinations, reasoning deficits), data governance (privacy, bias complexity), complexities (sim-to-real, latency), and rigorous safety assurance.
  arXiv  Detail & Related papers  (2025-05-19T21:51:18Z)
• Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A Survey [54.90240495777929]
  Ambiguity remains a fundamental challenge in Natural Language Processing (NLP)<n>With the advent of Large Language Models (LLMs), addressing ambiguity has become even more critical due to their expanded capabilities and applications.<n>This paper explores the definition, forms, and implications of ambiguity for language driven systems.
  arXiv  Detail & Related papers  (2025-05-18T20:53:41Z)
• Large Language Models Meet Stance Detection: A Survey of Tasks, Methods, Applications, Challenges and Future Directions [0.37865171120254354]
  Stance detection is essential for understanding subjective content across various platforms such as social media, news articles, and online reviews.<n>Recent advances in Large Language Models (LLMs) have revolutionized stance detection by introducing novel capabilities.<n>We present a novel taxonomy for LLM-based stance detection approaches, structured along three key dimensions.<n>Key applications in stance detection, political analysis, public health monitoring, and social media moderation are discussed.
  arXiv  Detail & Related papers  (2025-05-13T11:47:49Z)
• A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems [93.8285345915925]
  Reasoning is a fundamental cognitive process that enables logical inference, problem-solving, and decision-making.<n>With the rapid advancement of large language models (LLMs), reasoning has emerged as a key capability that distinguishes advanced AI systems.<n>We categorize existing methods along two dimensions: (1) Regimes, which define the stage at which reasoning is achieved; and (2) Architectures, which determine the components involved in the reasoning process.
  arXiv  Detail & Related papers  (2025-04-12T01:27:49Z)
• LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights [12.424610893030353]
  Large Language Models (LLMs) are emerging as transformative tools for software vulnerability detection.<n>This paper provides a detailed survey of LLMs in vulnerability detection.<n>We address challenges such as cross-language vulnerability detection, multimodal data integration, and repository-level analysis.
  arXiv  Detail & Related papers  (2025-02-10T21:33:38Z)
• Unique Security and Privacy Threats of Large Language Models: A Comprehensive Survey [63.4581186135101]
  Large language models (LLMs) have made remarkable advancements in natural language processing.<n>Privacy and security issues have been revealed throughout their life cycle.<n>This survey outlines and analyzes potential countermeasures.
  arXiv  Detail & Related papers  (2024-06-12T07:55:32Z)
• STPA for Learning-Enabled Systems: A Survey and A New Practice [12.665507596261266]
  Systems Theoretic Process Analysis (STPA) is a systematic approach for hazard analysis that has been used across many industrial sectors including transportation, energy, and defense. The trend of using Machine Learning (ML) in safety-critical systems has led to the need of extendingSTPA to Learning-Enabled Systems (LESs) We present a systematic survey of 31 papers, summarising them from five perspectives (attributes of concern, objects under study, modifications, derivatives and processes being modelled) We introduce DeepSTPA, which enhances DeepSTPA from two aspects that are missing from the state-of-the-
  arXiv  Detail & Related papers  (2023-02-21T10:43:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.