Beyond the Flag: Clinical Framing Closes the Moderation Gap in Suicide Risk Measurement

• URL: http://arxiv.org/abs/2609.06263v1
• Date: Sat, 05 Sep 2026 21:16:16 GMT
• Title: Beyond the Flag: Clinical Framing Closes the Moderation Gap in Suicide Risk Measurement
• Abstract summary: We ask how well deployed safety signals recover clinically meaningful severity.<n>We release a benchmark of 516 r/Watch posts rated by a licensed psychiatrist on a four-level ordinal schema.<n>We argue graded severity, not a binary flag, is what a proportionate duty of care requires.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Moderation APIs are built to flag policy-violating content, not to measure graded clinical risk. But a platform's duty does not end at detection: the response owed to passive distress differs sharply from the response owed to active planning with means access, and emerging regulation (e.g., California Senate Bill 243) is turning that distinction into a compliance requirement. We therefore ask how well deployed safety signals recover clinically meaningful severity. We release a benchmark of 516 r/SuicideWatch posts rated by a licensed psychiatrist on a four-level ordinal schema (Indicator, Ideation, Behavior, Attempt) grounded in the Columbia Suicide Severity Rating Scale, and evaluate moderation APIs, prompted LLMs, and supervised baselines under seven ordinal-aware metrics. Three findings. Vendor moderation APIs separate low- from high-severity posts well (0.860 high-risk F1) but measure severity poorly (0.395 macro F1), systematically over-predicting the most severe category. Clinically grounded zero-shot prompting recovers much of that gap (0.562 macro F1), and expert-authored framing (not fine-tuning, added reasoning, or naive multi-agent aggregation) is the effective lever. The value of reasoning depends on register: it hurts on long, noisy Reddit posts and helps on short, clinician-authored statements. We argue graded severity, not a binary flag, is what a proportionate duty of care requires, and release our evaluation framework to support that measurement.

Related papers

• DementiaCare-Bench: A Modality-Validated Video Benchmark [83.92703885588868]
  Dementia affects an estimated 57 million people worldwide.<n>For most families the hardest part of care is not memory loss but the behavioral and psychological symptoms of dementia.<n>We present DementiaCare-Bench: 56 professionally produced caregiver training videos segmented into 94 clips across nine BPSD categories.
  arXiv  Detail & Related papers  (2026-09-11T14:52:49Z)
• MedSNIP: Building and Benchmarking Snippet-Level Granularity for Medical Fact Verification [50.13873960887466]
  We introduce MedSNIP-Bench, a benchmark for snippet-level medical fact verification.<n>We also introduce MedSNIP, an automatic snippet-generation pipeline.<n>Across MedSNIP-Bench, HealthFC, and MedHallu, snippet-level verification preserves or improves false-class F1.
  arXiv  Detail & Related papers  (2026-09-11T14:08:44Z)
• Auditable Emergency Triage for Maternal and Newborn Care in India [1.121169121023536]
  Nurses at Noora Health answer more than 50,000 medical queries per month on our WhatsApp-based service.<n>To support them, we built a system that uses a large language model (LLM) to classify whether a message is an emergency.<n>We show that the new system raised recall from 0.565 to 0.810 and F1 from 0.606 to 0.702.
  arXiv  Detail & Related papers  (2026-09-08T18:45:01Z)
• Framework for Grounding Healthcare LLMs in a Causal Knowledge Graph: A Cardiovascular Example Pilot [0.9685837672183747]
  Large language models (LLMs) are increasingly proposed for healthcare decision support.<n>LLMs still reward single-answer accuracy rather than reasoning about interventions, mechanisms, harms, evidence, and uncertainty.<n>We propose a graph-centered evaluation framework for intervention-oriented LLM behavior in healthcare and stress-test it in a cardiovascular pilot.
  arXiv  Detail & Related papers  (2026-08-15T19:21:51Z)
• Labels Are Not Endpoints: Treatment Leakage and Construct Validity in MCP Agent Security Evaluation [0.0]
  We audit a preserved campaign by tracing 10,200 execution rows to 180 model-bound requests, 45 semantic requests, and 15 observable stimuli.<n>We contribute a seven-link Integrity Chain and an executable, scope-bounded endpoint-integrity linter.
  arXiv  Detail & Related papers  (2026-08-13T06:44:40Z)
• Trivial Prompt Reframing Bypasses Safety Guardrails in Googleś MedGemma-4B [0.0]
  Open-weight medical language models are increasingly used as the base of patient-facing and clinician-support applications.<n>We quantify that gap for MedGemma-4B-it under attacks that require no technical sophistication.
  arXiv  Detail & Related papers  (2026-07-09T18:31:43Z)
• Before the Labels: How Dataset Construction Shapes Suicidality Detection in Clinical Text [0.5483653499671707]
  We show how governance constraints, ICD-based cohort selection, single-annotator labeling, and hospital-stay-level aggregation produce labels that reflect clinician-documented judgments.<n>A linguistic analysis demonstrates that identical labels subsume heterogeneous clinical framings differing in temporality, negation, and uncertainty.
  arXiv  Detail & Related papers  (2026-06-17T22:31:27Z)
• Beyond Scalar Scores: Exploring LLM-based Metrics for Clinical Significance Evaluation in Radiology Reports [49.5225801722164]
  Existing metrics obscure this requirement by reducing report quality to a medically ungrounded scalar.<n>We study this boundary using ReEvalMed benchmark as testbed and evaluate metric-level clinical significance.
  arXiv  Detail & Related papers  (2026-06-17T08:10:30Z)
• Trust but Verify: Mitigating Medical Hallucinations via Post-Hoc Adversarial Auditing and Multi-Agent Feedback Loops [0.07249400282852114]
  Large Language Models (LLMs) are increasingly deployed in healthcare settings, yet their tendency to hallucinate poses risks when clinical decisions are involved.<n>This study examine whether LLMs recommend recently banned or withdrawn pharmaceuticals when answering clinical questions.<n>We developed a five-agent "Trust but verify" system using a single LLM backbone.
  arXiv  Detail & Related papers  (2026-06-12T06:21:19Z)
• Health-ORSC-Bench: A Benchmark for Measuring Over-Refusal and Safety Completion in Health Context [82.32380418146656]
  Health-ORSC-Bench is the first large-scale benchmark designed to measure textbfOver-Refusal and textbfSafe Completion quality in healthcare.<n>Our framework uses an automated pipeline with human validation to test models at varying levels of intent ambiguity.<n>Health-ORSC-Bench provides a rigorous standard for calibrating the next generation of medical AI assistants.
  arXiv  Detail & Related papers  (2026-01-25T01:28:52Z)
• A Granular Study of Safety Pretraining under Model Abliteration [64.24346997570275]
  We study model abliteration, a lightweight projection technique designed to remove refusal-sensitive directions.<n>We issue 100 prompts with balanced harmful and harmless cases, classify responses as **Refusal** or **Non-Refusal** using multiple judges, and validate judge fidelity.<n>Our study produces a checkpoint-level characterization of which data-centric safety components remain robust under abliteration.
  arXiv  Detail & Related papers  (2025-10-03T07:01:45Z)
• MedOmni-45°: A Safety-Performance Benchmark for Reasoning-Oriented LLMs in Medicine [69.08855631283829]
  We introduce Med Omni-45 Degrees, a benchmark designed to quantify safety-performance trade-offs under manipulative hint conditions.<n>It contains 1,804 reasoning-focused medical questions across six specialties and three task types, including 500 from MedMCQA.<n>Results show a consistent safety-performance trade-off, with no model surpassing the diagonal.
  arXiv  Detail & Related papers  (2025-08-22T08:38:16Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• Navigating the OverKill in Large Language Models [84.62340510027042]
  We investigate the factors for overkill by exploring how models handle and determine the safety of queries. Our findings reveal the presence of shortcuts within models, leading to an over-attention of harmful words like 'kill' and prompts emphasizing safety will exacerbate overkill. We introduce Self-Contrastive Decoding (Self-CD), a training-free and model-agnostic strategy, to alleviate this phenomenon.
  arXiv  Detail & Related papers  (2024-01-31T07:26:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.