Evaluating Criterion-Conditioned Behaviour of Large Language Models in Content Moderation

• URL: http://arxiv.org/abs/2609.03814v1
• Date: Thu, 03 Sep 2026 13:17:38 GMT
• Title: Evaluating Criterion-Conditioned Behaviour of Large Language Models in Content Moderation
• Abstract summary: We study whether large language models (LLMs) exhibit criterion-conditioned behaviour.<n>We find that strong benchmark performance can hide substantial failures at the criterion level.<n>These findings call for the development of evaluation methods that explicitly measure criterion-conditioned behaviour.
• Score: 1.3629559589878444
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) demonstrate strong performance on standard content moderation benchmarks. However, these benchmarks often aggregate multiple moderation criteria into a single label, making it unclear whether models can disentangle them and reliably apply each criterion when making decisions. To study whether LLMs exhibit criterion-conditioned behaviour, we introduce Diagnostic Evaluation of COntent (DECO), a criterion-independent factorisation of content that enables controlled, criterion-level evaluation. We also introduce pairwise evaluation to compare model outputs across different criteria for the same input. Across four moderation datasets and four LLMs, we find that strong benchmark performance can hide substantial failures at the criterion level. Models struggle most when correct decisions depend not on overall harmfulness, but on the specific aspect of the content that the criterion requires them to assess. Our results highlight a key limitation of current content moderation benchmarks: strong performance on aggregated labels does not provide sufficient evidence that LLMs can reliably evaluate content with respect to individual moderation criteria. These findings call for the development of evaluation methods that explicitly measure criterion-conditioned behaviour.

Related papers

• Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents [3.6300043516541263]
  Poor benchmarks may contain inconsistent tasks, simplistic scenarios, or limited policy coverage.<n>We introduce a reference-free framework that uses LLM judges to assess benchmark consistency, complexity, and policy coverage.<n>Our framework offers a practical approach for evaluating synthetic and manually curated conversational-agent benchmarks.
  arXiv  Detail & Related papers  (2026-08-06T17:39:21Z)
• Benchmarks Are Not Monolithic: Sample-Level Auditing and Orchestration for LLM Evaluation [0.42970700836450487]
  We introduce a dataset-centric meta-evaluation framework that audits benchmark datasets at the sample level along five latent dimensions.<n>We show how these annotations enable criterion-driven orchestration of composite benchmark subsets across datasets.
  arXiv  Detail & Related papers  (2026-07-30T19:51:55Z)
• Criterion-Conditional In-Context Learning: Evaluating Criterion-Shift Adaptation in Vision-Language Models [2.2633326936241196]
  Vision-language models can perform new tasks without parameter updates through in-context learning (ICL)<n>In real-world applications, many tasks exhibit a stable high-level intent, while their decision criteria shift according to specific requirements.<n>We introduce a new setting, denoted as Criterion-Conditional In-Context Learning (CC-ICL), where models must infer the latent criterion from context.
  arXiv  Detail & Related papers  (2026-06-30T17:22:20Z)
• CriterAlign: Criterion-Centric Rationale Alignment for Code Preference Judging [95.02210956333374]
  We propose a criterion-centric framework that adapts rubric-based judging to pairwise preference evaluation.<n>On BigCodeReward, CriterAlign improves a Qwen2.5-VL-32B monolithic judge from 60.4% to 66.3% accuracy.
  arXiv  Detail & Related papers  (2026-05-19T10:59:19Z)
• Evaluating Ill-Defined Tasks in Large Language Models [4.241892359077409]
  We analyze why evaluation benchmarks and metrics fail to provide reliable or diagnostic signals of model capability for ill-defined tasks.<n>We show how multi-faceted evaluation criteria can yield actionable insights beyond aggregate scores.
  arXiv  Detail & Related papers  (2026-03-17T18:52:47Z)
• IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation [85.56193980646981]
  We propose IF-RewardBench, a comprehensive meta-evaluation benchmark for instruction-following.<n>For each instruction, we construct a preference graph containing all pairwise preferences among multiple responses.<n>Experiments on IF-RewardBench reveal significant deficiencies in current judge models.
  arXiv  Detail & Related papers  (2026-03-05T02:21:17Z)
• Benchmark^2: Systematic Evaluation of LLM Benchmarks [66.2731798872668]
  We propose Benchmark2, a comprehensive framework comprising three complementary metrics.<n>We conduct experiments across 15 benchmarks spanning mathematics, reasoning, and knowledge domains.<n>Our analysis reveals significant quality variations among existing benchmarks and demonstrates that selective benchmark construction can achieve comparable evaluation performance.
  arXiv  Detail & Related papers  (2026-01-07T14:59:03Z)
• KOFFVQA: An Objectively Evaluated Free-form VQA Benchmark for Large Vision-Language Models in the Korean Language [2.594684920405059]
  We present KOFFVQA, a general-purpose free-form visual question answering benchmark in the Korean language.<n>Our benchmark consists of 275 carefully crafted questions each paired with an image and grading criteria.<n>We experimentally verify that our method of using pre-existing grading criteria for evaluation is much more reliable than existing methods.
  arXiv  Detail & Related papers  (2025-03-31T05:04:25Z)
• Evaluating Generative Language Models in Information Extraction as Subjective Question Correction [49.729908337372436]
  We propose a new evaluation method, SQC-Score. Inspired by the principles in subjective question correction, we propose a new evaluation method, SQC-Score. Results on three information extraction tasks show that SQC-Score is more preferred by human annotators than the baseline metrics.
  arXiv  Detail & Related papers  (2024-04-04T15:36:53Z)
• HD-Eval: Aligning Large Language Model Evaluators Through Hierarchical Criteria Decomposition [92.17397504834825]
  HD-Eval is a framework that iteratively aligns large language models evaluators with human preference. HD-Eval inherits the essence from the evaluation mindset of human experts and enhances the alignment of LLM-based evaluators. Extensive experiments on three evaluation domains demonstrate the superiority of HD-Eval in further aligning state-of-the-art evaluators.
  arXiv  Detail & Related papers  (2024-02-24T08:01:32Z)
• Self-Evaluation Improves Selective Generation in Large Language Models [54.003992911447696]
  We reformulate open-ended generation tasks into token-level prediction tasks. We instruct an LLM to self-evaluate its answers. We benchmark a range of scoring methods based on self-evaluation.
  arXiv  Detail & Related papers  (2023-12-14T19:09:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.