ATIBA: Grounded Integrity and Quality Checking for Research Papers

• URL: http://arxiv.org/abs/2609.04123v1
• Date: Thu, 03 Sep 2026 17:24:15 GMT
• Title: ATIBA: Grounded Integrity and Quality Checking for Research Papers
• Abstract summary: We present ATIBA, a tool that runs five grounded integrity and quality checks on a manuscript.<n>It is designed around the same principle: an LLM is only trusted to judge, never to invent the evidence it judges against.
• Score: 1.3241176321860364
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Checking a manuscript's reference integrity, its compliance with a target venue's specific submission rules, and its adherence to community reporting standards is manual, repetitive, and different for every venue so in practice it is done inconsistently or skipped. We present ATIBA, a tool that runs five grounded integrity and quality checks on a manuscript: a reference-integrity check that verifies each citation against bibliographic sources and flags retracted or unfindable references; a venue/track compliance check that derives submission criteria directly from a venue's own call-for-papers page and evaluates the manuscript against them, each verdict anchored to a verbatim quote from that page; an empirical-standards compliance check against the ACM SIGSOFT Empirical Standards, with a hallucination defence that discards any evidence quote it cannot locate verbatim in the manuscript; a multi-mode AI review (venue-specific, formal, and page-anchored annotation) powered by GPT-5.4 through Azure OpenAI; and a citation-suggestion feature that proposes candidate references for a manuscript and verifies each against bibliographic sources before it is shown to the user. All five checks are designed around the same principle: an LLM is only trusted to judge, never to invent the evidence it judges against. We evaluated ATIBA through a moderated user study with 13 non-author participants. Agreement across the six survey items ranged from 69% to 92%, with a mean of 85%, providing initial evidence of positive perceived usefulness across the evaluated workflows. These findings establish perceived usefulness; objective accuracy remains to be measured.

Related papers

• RefVerifier: Semi-Automated Reference Claim Verification for Scientific Manuscripts [1.6950215926321557]
  RefVerifier is a citation-bounded reference verification prototype designed to support academic peer review.<n>It extracts citation-bearing sentences from manuscripts, checks metadata against scholarly databases, resolves references to PDFs, localizes relevant evidence passages, and generates verdicts with natural language explanations.<n>In an end-to-end test with eight manuscripts, RefVerifier achieves a verdict accuracy of 71%.
  arXiv  Detail & Related papers  (2026-09-07T15:39:59Z)
• AtomCite: Verification and Correction of Supplied Page-Level Citations in Multi-Page Documents [21.842813905857863]
  AtomCite parses an answer into claims, checks each claim against the image of the cited page, and applies a deterministic repair policy.<n>It reaches around 93% binary verification accuracy on the injected benchmark, significantly outperforming every OCR-only condition.<n>Its repair policy lifts citation precision on the injected mix from a constructed 34% to 87-90% while retaining over 90% of correct claims.
  arXiv  Detail & Related papers  (2026-09-05T01:31:50Z)
• Benchmarking and Enhancing LLMs for Rule-Intensive Review of National Standard Documents [7.397429318828859]
  Large language models (LLMs) increasingly support complex professional tasks, yet their capabilities in rule-intensive document review remain insufficiently evaluated.<n>Existing benchmarks focus on domain knowledge and question answering, largely overlooking quality review for professional documents.<n>We introduce GB/T-Bench, the first benchmark for the structured review of national standard documents.
  arXiv  Detail & Related papers  (2026-08-06T17:27:23Z)
• ResearchQA: Benchmarking Citation-Grounded Question-Answering on Scientific Papers [0.0]
  ResearchQA is a benchmark of 6,211 single-paper question-answer pairs from 494 open-access papers.<n>ResearchQA is designed for citation-grounded evaluation.<n>We evaluate eight leading closed- and open-weight models in a citation-grounded chat-with-paper setting.
  arXiv  Detail & Related papers  (2026-07-13T04:26:51Z)
• ProvenAI: Provenance-Native Traces of Evidence in Generated Answers [0.0]
  This paper introduces ProvenAI, a framework that decomposes transparency in multi-hop question answering into three independently measurable layers.<n>The system achieves 53.53% answer accuracy alongside a mean citation-fidelity score of 71.55%.<n>We formalise the relationship between the implemented surface proxy and a token-level KL-divergence target through a stated faithfulness condition.
  arXiv  Detail & Related papers  (2026-06-24T23:22:38Z)
• Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting [44.70980205167796]
  AI evaluation results are produced at scale but reported inconsistently across leaderboards, model cards, benchmark papers, and company blogs.<n>Recent efforts address isolated components but leave three gaps: they cover only narrow slices of the evaluation lifecycle and do not compose into a single interpretable record.<n>We present EvalCards, an operational reporting layer that composes benchmark metadata, evaluation run data, and model metadata into a unified record.
  arXiv  Detail & Related papers  (2026-06-08T17:55:02Z)
• ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidence [57.37494162084001]
  Chain-of-Evidence (CoE) is a verifiability framework requiring every claim to be traceable to its evidence source.<n>CoE Audit is a post-hoc audit whose four integrity checks -- score verification, specification violation, reference verification, and method-code alignment -- apply uniformly to all systems.
  arXiv  Detail & Related papers  (2026-05-25T21:30:27Z)
• CitePrism: Human-in-the-Loop AI for Citation Auditing and Editorial Integrity [0.2625927677887832]
  CitePrism is a transparent hybrid decision-support framework for editorial citation auditing.<n>It extracts citation neighborhoods, enriches reference metadata, computes fused relevance scores, surfaces metadata and self-citation review prompts.<n>CitePrism is intended as pilot-stage decision support, not as an autonomous misconduct detector or automated editorial decision system.
  arXiv  Detail & Related papers  (2026-05-15T14:31:45Z)
• ReviewGrounder: Improving Review Substantiveness with Rubric-Guided, Tool-Integrated Agents [50.27474750319121]
  Review text according to paper-specific rubrics derived from official guidelines, the paper's content, and human-written reviews.<n>We introduce REVIEWBENCH, a benchmark evaluating review text according to paper-specific rubrics derived from official guidelines, the paper's content, and human-written reviews.<n>We propose REVIEWGROUNDER, a rubric-guided, tool-integrated multi-agent framework that decomposes reviewing into drafting and grounding stages.
  arXiv  Detail & Related papers  (2026-04-15T16:33:04Z)
• FactReview: Evidence-Grounded Reviews with Literature Positioning and Execution-Based Claim Verification [57.196748998757954]
  We present FactReview, an evidence-grounded reviewing system that combines claim extraction, literature positioning, and execution-based claim verification.<n>Given a submission, FactReview identifies major claims and reported results, retrieves nearby work to clarify the paper's technical position, and, when code is available, executes the released repository.<n>It then produces a concise review and an evidence report that assigns each major claim one of five labels: Supported, Supported by the paper, Partially supported, In conflict, or Inconclusive.
  arXiv  Detail & Related papers  (2026-04-05T11:45:22Z)
• CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era [51.63024682584688]
  Large language models (LLMs) introduce a new risk: fabricated references that appear plausible but correspond to no real publications.<n>We present the first comprehensive benchmark and detection framework for hallucinated citations in scientific writing.<n>Our framework significantly outperforms prior methods in both accuracy and interpretability.
  arXiv  Detail & Related papers  (2026-02-26T19:17:39Z)
• OpenNovelty: An LLM-powered Agentic System for Verifiable Scholarly Novelty Assessment [63.662126457336534]
  OpenNovelty is an agentic system for transparent, evidence-based novelty analysis.<n>It grounds all assessments in retrieved real papers, ensuring verifiable judgments.<n>OpenNovelty aims to empower the research community with a scalable tool that promotes fair, consistent, and evidence-backed peer review.
  arXiv  Detail & Related papers  (2026-01-04T15:48:51Z)
• End-to-End Page-Level Assessment of Handwritten Text Recognition [69.55992406968495]
  HTR systems increasingly face the end-to-end page-level transcription of a document. Standard metrics do not take into account the inconsistencies that might appear. We propose a two-fold evaluation, where the transcription accuracy and the RO goodness are considered separately.
  arXiv  Detail & Related papers  (2023-01-14T15:43:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.