CitePrism: Human-in-the-Loop AI for Citation Auditing and Editorial Integrity

• URL: http://arxiv.org/abs/2605.16000v1
• Date: Fri, 15 May 2026 14:31:45 GMT
• Title: CitePrism: Human-in-the-Loop AI for Citation Auditing and Editorial Integrity
• Abstract summary: CitePrism is a transparent hybrid decision-support framework for editorial citation auditing.<n>It extracts citation neighborhoods, enriches reference metadata, computes fused relevance scores, surfaces metadata and self-citation review prompts.<n>CitePrism is intended as pilot-stage decision support, not as an autonomous misconduct detector or automated editorial decision system.
• Score: 0.2625927677887832
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Editors and reviewers are expected to ensure that manuscripts cite relevant, accurate, current, and ethically appropriate literature, yet manuscript-level citation auditing remains largely manual, fragmented, and difficult to scale. Citation context, metadata quality, self-citation patterns, and bibliographic integrity all affect whether a reference appropriately supports a local claim. We present CitePrism, a transparent hybrid decision-support framework for editorial citation auditing that combines LLM-assisted contextual reasoning, embedding-based semantic similarity, metadata verification, integrity-oriented flags, and human-in-the-loop analyst review. CitePrism extracts citation neighborhoods, enriches reference metadata, computes fused relevance scores, surfaces metadata and self-citation review prompts, and supports configurable threshold-based triage. In a preliminary validation on a single case-study manuscript with 104 references from pavement engineering, agreement with human binary relevance labels reached Cohen's kappa = 0.429. At operating threshold tau = 17, CitePrism flagged all human-labeled irrelevant citations, while also producing false positives requiring analyst review. These results suggest that CitePrism may support conservative editorial screening and citation-quality triage, but they do not establish general editorial performance. CitePrism is intended as pilot-stage decision support, not as an autonomous misconduct detector or automated editorial decision system. Broader validation across manuscripts, domains, annotators, baselines, and deployment settings is required before operational use.

Related papers

• ATIBA: Grounded Integrity and Quality Checking for Research Papers [1.3241176321860364]
  We present ATIBA, a tool that runs five grounded integrity and quality checks on a manuscript.<n>It is designed around the same principle: an LLM is only trusted to judge, never to invent the evidence it judges against.
  arXiv  Detail & Related papers  (2026-09-03T17:24:15Z)
• FactReview: Evidence-Grounded Reviews with Literature Positioning and Execution-Based Claim Verification [57.196748998757954]
  We present FactReview, an evidence-grounded reviewing system that combines claim extraction, literature positioning, and execution-based claim verification.<n>Given a submission, FactReview identifies major claims and reported results, retrieves nearby work to clarify the paper's technical position, and, when code is available, executes the released repository.<n>It then produces a concise review and an evidence report that assigns each major claim one of five labels: Supported, Supported by the paper, Partially supported, In conflict, or Inconclusive.
  arXiv  Detail & Related papers  (2026-04-05T11:45:22Z)
• CRISP: Characterizing Relative Impact of Scholarly Publications [70.25915589244961]
  We propose CRISP, which jointly ranks all cited papers within a citing paper using large language models (LLMs)<n>To mitigate LLMs' positional bias, we rank each list three times in a randomized order and aggregate the impact labels through majority voting.<n>CRISP outperforms a prior state-of-the-art impact classifier by +9.5% accuracy and +8.3% F1 on a dataset of human-annotated citations.
  arXiv  Detail & Related papers  (2026-03-25T16:42:30Z)
• CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era [51.63024682584688]
  Large language models (LLMs) introduce a new risk: fabricated references that appear plausible but correspond to no real publications.<n>We present the first comprehensive benchmark and detection framework for hallucinated citations in scientific writing.<n>Our framework significantly outperforms prior methods in both accuracy and interpretability.
  arXiv  Detail & Related papers  (2026-02-26T19:17:39Z)
• SemanticCite: Citation Verification with AI-Powered Full-Text Analysis and Evidence-Based Reasoning [0.0]
  We introduce SemanticCite, an AI-powered system that verifies citation accuracy through full-text source analysis.<n>Our approach combines multiple retrieval methods with a four-class classification system that captures nuanced claim-source relationships.<n>We contribute a comprehensive dataset of over 1,000 citations with detailed alignments, functional classifications, semantic annotations, and bibliometric metadata.
  arXiv  Detail & Related papers  (2025-11-20T10:05:21Z)
• A Comparative Analysis of Faithfulness Metrics and Humans in Citation Evaluation [22.041561519672456]
  Large language models (LLMs) often generate content with unsupported or unverifiable content, known as "hallucinations" We propose a comparative evaluation framework that assesses the metric effectiveness in distinguishing citations between three-category support levels. Our results indicate no single metric consistently excels across all evaluations, highlighting the complexity of accurately evaluating fine-grained support levels.
  arXiv  Detail & Related papers  (2024-08-22T13:44:31Z)
• Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics [22.041561519672456]
  Large language models (LLMs) often produce unsupported or unverifiable content, known as "hallucinations" We propose a comparative evaluation framework that assesses the metric effectiveness in distinguishing citations between three-category support levels. Our results show no single metric consistently excels across all evaluations, revealing the complexity of assessing fine-grained support.
  arXiv  Detail & Related papers  (2024-06-21T15:57:24Z)
• ALiiCE: Evaluating Positional Fine-grained Citation Generation [54.19617927314975]
  We propose ALiiCE, the first automatic evaluation framework for fine-grained citation generation. Our framework first parses the sentence claim into atomic claims via dependency analysis and then calculates citation quality at the atomic claim level. We evaluate the positional fine-grained citation generation performance of several Large Language Models on two long-form QA datasets.
  arXiv  Detail & Related papers  (2024-06-19T09:16:14Z)
• Interpretable Automatic Fine-grained Inconsistency Detection in Text Summarization [56.94741578760294]
  We propose the task of fine-grained inconsistency detection, the goal of which is to predict the fine-grained types of factual errors in a summary. Motivated by how humans inspect factual inconsistency in summaries, we propose an interpretable fine-grained inconsistency detection model, FineGrainFact.
  arXiv  Detail & Related papers  (2023-05-23T22:11:47Z)
• End-to-End Page-Level Assessment of Handwritten Text Recognition [69.55992406968495]
  HTR systems increasingly face the end-to-end page-level transcription of a document. Standard metrics do not take into account the inconsistencies that might appear. We propose a two-fold evaluation, where the transcription accuracy and the RO goodness are considered separately.
  arXiv  Detail & Related papers  (2023-01-14T15:43:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.