Model-Agnostic Sentiment Distribution Stability Analysis for Robust LLM-Generated Texts Detection

• URL: http://arxiv.org/abs/2508.06913v1
• Date: Sat, 09 Aug 2025 09:55:47 GMT
• Title: Model-Agnostic Sentiment Distribution Stability Analysis for Robust LLM-Generated Texts Detection
• Abstract summary: We propose SentiDetect, a model-agnostic framework for detecting large language models (LLMs)-generated text.<n>Our method is motivated by the empirical observation that LLM outputs tend to exhibit emotionally consistent patterns, whereas human-written texts display greater emotional variability.<n>We evaluate SentiDetect on five diverse datasets and a range of advanced LLMs, including Gemini-1.5-Pro, Claude-3, GPT-4-0613, and LLaMa-3.3.
• Score: 23.153044933861988
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid advancement of large language models (LLMs) has resulted in increasingly sophisticated AI-generated content, posing significant challenges in distinguishing LLM-generated text from human-written language. Existing detection methods, primarily based on lexical heuristics or fine-tuned classifiers, often suffer from limited generalizability and are vulnerable to paraphrasing, adversarial perturbations, and cross-domain shifts. In this work, we propose SentiDetect, a model-agnostic framework for detecting LLM-generated text by analyzing the divergence in sentiment distribution stability. Our method is motivated by the empirical observation that LLM outputs tend to exhibit emotionally consistent patterns, whereas human-written texts display greater emotional variability. To capture this phenomenon, we define two complementary metrics: sentiment distribution consistency and sentiment distribution preservation, which quantify stability under sentiment-altering and semantic-preserving transformations. We evaluate SentiDetect on five diverse datasets and a range of advanced LLMs,including Gemini-1.5-Pro, Claude-3, GPT-4-0613, and LLaMa-3.3. Experimental results demonstrate its superiority over state-of-the-art baselines, with over 16% and 11% F1 score improvements on Gemini-1.5-Pro and GPT-4-0613, respectively. Moreover, SentiDetect also shows greater robustness to paraphrasing, adversarial attacks, and text length variations, outperforming existing detectors in challenging scenarios.

Related papers

• WaveDetect: Robust Framework for Machine-Generated Text Detection via Wavelet Transform [2.7227283756285288]
  Existing detectors often falter when facing three critical challenges: adversarial perturbations, cross-domain shifts, and the rapid temporal evolution of the foundation model.<n>We propose wavedetect, a novel framework that reformulates text detection as a signal processing task within the time-frequency domain.
  arXiv  Detail & Related papers  (2026-06-22T13:42:36Z)
• Hidden Human-Like Nature of Machine-Generated Texts: Theory and Detection Enhancement [108.30620357325559]
  Machine-generated texts (MGTs) produced by large language models (LLMs) are increasingly prevalent across various applications.<n>Existing paragraph-level detection methods treat MGTs as entirely machine-like, overlooking the hidden human-like nature of MGTs.<n>We propose a model-agnostic stacked enhancement framework that improves existing detectors by reducing the influence of hidden human-like spans.
  arXiv  Detail & Related papers  (2026-05-22T03:17:21Z)
• DSIPA: Detecting LLM-Generated Texts via Sentiment-Invariant Patterns Divergence Analysis [13.9689104714669]
  DSIPA is a training-free framework that detects large language models (LLMs) content by quantifying sentiment distributional stability under controlled stylistic variation.<n>It operates in a zero-shot, black-box manner, leveraging two unsupervised metrics, sentiment distribution consistency and sentiment distribution preservation.<n>Our framework exhibits superior generalizability across domains and strong resilience to adversarial conditions.
  arXiv  Detail & Related papers  (2026-04-29T06:22:08Z)
• Luminol-AIDetect: Fast Zero-shot Machine-Generated Text Detection based on Perplexity under Text Shuffling [9.241565393225953]
  We propose Luminol-AIDetect, a novel, zero-shot statistical approach for machine-generated text (MGT) detection.<n>By applying a simple randomized text-shuffling procedure, we demonstrate that the resulting shift in perplexity serves as a principled, model-agnostic discriminant.<n>We show that Luminol-AIDetect demonstrates state-of-the-art performance, with gains up to 17x lower FPR while being cheaper than prior methods.
  arXiv  Detail & Related papers  (2026-04-28T16:58:55Z)
• Same Meaning, Different Scores: Lexical and Syntactic Sensitivity in LLM Evaluation [40.210132040677]
  This paper examines how controlled, truth-conditionally equivalent lexical and syntactic perturbations affect the absolute performance and relative ranking of 23 contemporary Large Language Models (LLMs)<n>Results show that lexical perturbations consistently induce substantial, statistically significant performance degradation across nearly all models and tasks, while syntactic perturbations have more heterogeneous effects, occasionally improving results.
  arXiv  Detail & Related papers  (2026-02-19T12:24:42Z)
• Sensitivity of Small Language Models to Fine-tuning Data Contamination [0.0]
  Small Language Models (SLMs) are increasingly being deployed in resource-constrained environments.<n>We measure susceptibility to syntactic and semantic transformation types during instruction tuning.<n>Character reversal produces near-complete failure across all models regardless of size or family.<n>Semantic transformations demonstrate distinct threshold behaviors and greater resilience in core linguistic capabilities.
  arXiv  Detail & Related papers  (2025-11-10T06:44:29Z)
• DeLeaker: Dynamic Inference-Time Reweighting For Semantic Leakage Mitigation in Text-to-Image Models [55.30555646945055]
  Text-to-Image (T2I) models are vulnerable to semantic leakage.<n>We introduce DeLeaker, a lightweight approach that mitigates leakage by directly intervening on the model's attention maps.<n>SLIM is the first dataset dedicated to semantic leakage.
  arXiv  Detail & Related papers  (2025-10-16T17:39:21Z)
• StyleDecipher: Robust and Explainable Detection of LLM-Generated Texts with Stylistic Analysis [18.44456241158174]
  StyleDecipher is a robust and explainable detection framework.<n>It revisits text detection using combined feature extractors to quantify stylistic differences.<n>It consistently achieves state-of-the-art in-domain accuracy.
  arXiv  Detail & Related papers  (2025-10-14T15:07:27Z)
• Diversity Boosts AI-Generated Text Detection [51.56484100374058]
  DivEye is a novel framework that captures how unpredictability fluctuates across a text using surprisal-based features.<n>Our method outperforms existing zero-shot detectors by up to 33.2% and achieves competitive performance with fine-tuned baselines.
  arXiv  Detail & Related papers  (2025-09-23T10:21:22Z)
• RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patterns [50.401907401444404]
  Large language models (LLMs) are crucial for preventing misuse and building trustworthy AI systems.<n>We propose RepreGuard, an efficient statistics-based detection method.<n> Experimental results show that RepreGuard outperforms all baselines with average 94.92% AUROC on both in-distribution (ID) and OOD scenarios.
  arXiv  Detail & Related papers  (2025-08-18T17:59:15Z)
• When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs [55.20230501807337]
  We present the first systematic evaluation of 5 methods for improving prompt robustness within a unified experimental framework.<n>We benchmark these techniques on 8 models from Llama, Qwen and Gemma families across 52 tasks from Natural Instructions dataset.
  arXiv  Detail & Related papers  (2025-08-15T10:32:50Z)
• Quantifying Fairness in LLMs Beyond Tokens: A Semantic and Statistical Perspective [13.739343897204568]
  Large Language Models (LLMs) often generate responses with inherent biases, undermining their reliability in real-world applications.<n>Existing evaluation methods often overlook biases in long-form responses and the intrinsic variability of LLM outputs.<n>We propose FiSco, a novel statistical framework to evaluate group-level fairness in LLMs by detecting subtle semantic differences in long-form responses across demographic groups.
  arXiv  Detail & Related papers  (2025-06-23T18:31:22Z)
• Hallucination Detection in LLMs with Topological Divergence on Attention Graphs [64.74977204942199]
  Hallucination, i.e., generating factually incorrect content, remains a critical challenge for large language models.<n>We introduce TOHA, a TOpology-based HAllucination detector in the RAG setting.
  arXiv  Detail & Related papers  (2025-04-14T10:06:27Z)
• Estimating Commonsense Plausibility through Semantic Shifts [66.06254418551737]
  We propose ComPaSS, a novel discriminative framework that quantifies commonsense plausibility by measuring semantic shifts.<n> Evaluations on two types of fine-grained commonsense plausibility estimation tasks show that ComPaSS consistently outperforms baselines.
  arXiv  Detail & Related papers  (2025-02-19T06:31:06Z)
• Unveiling Large Language Models Generated Texts: A Multi-Level Fine-Grained Detection Framework [9.976099891796784]
  Large language models (LLMs) have transformed human writing by enhancing grammar correction, content expansion, and stylistic refinement. Existing detection methods, which mainly rely on single-feature analysis and binary classification, often fail to effectively identify LLM-generated text in academic contexts. We propose a novel Multi-level Fine-grained Detection framework that detects LLM-generated text by integrating low-level structural, high-level semantic, and deep-level linguistic features.
  arXiv  Detail & Related papers  (2024-10-18T07:25:00Z)
• Training-free LLM-generated Text Detection by Mining Token Probability Sequences [18.955509967889782]
  Large language models (LLMs) have demonstrated remarkable capabilities in generating high-quality texts across diverse domains. Training-free methods, which focus on inherent discrepancies through carefully designed statistical features, offer improved generalization and interpretability. We introduce a novel training-free detector, termed textbfLastde that synergizes local and global statistics for enhanced detection.
  arXiv  Detail & Related papers  (2024-10-08T14:23:45Z)
• Detecting Machine-Generated Long-Form Content with Latent-Space Variables [54.07946647012579]
  Existing zero-shot detectors primarily focus on token-level distributions, which are vulnerable to real-world domain shifts. We propose a more robust method that incorporates abstract elements, such as event transitions, as key deciding factors to detect machine versus human texts.
  arXiv  Detail & Related papers  (2024-10-04T18:42:09Z)
• Learning to Rewrite: Generalized LLM-Generated Text Detection [19.9477991969521]
  Large language models (LLMs) present significant risks when used to generate non-factual content and spread disinformation at scale.<n>We introduce Learning2Rewrite, a novel framework for detecting AI-generated text with exceptional generalization to unseen domains.
  arXiv  Detail & Related papers  (2024-08-08T05:53:39Z)
• MGTBench: Benchmarking Machine-Generated Text Detection [54.81446366272403]
  This paper proposes the first benchmark framework for MGT detection against powerful large language models (LLMs) We show that a larger number of words in general leads to better performance and most detection methods can achieve similar performance with much fewer training samples. Our findings indicate that the model-based detection methods still perform well in the text attribution task.
  arXiv  Detail & Related papers  (2023-03-26T21:12:36Z)
• In and Out-of-Domain Text Adversarial Robustness via Label Smoothing [64.66809713499576]
  We study the adversarial robustness provided by various label smoothing strategies in foundational models for diverse NLP tasks. Our experiments show that label smoothing significantly improves adversarial robustness in pre-trained models like BERT, against various popular attacks. We also analyze the relationship between prediction confidence and robustness, showing that label smoothing reduces over-confident errors on adversarial examples.
  arXiv  Detail & Related papers  (2022-12-20T14:06:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.