FACTWASH: Catching AI Rewrites That Wash Hearsay into Fact

• URL: http://arxiv.org/abs/2608.03372v1
• Date: Tue, 04 Aug 2026 09:20:54 GMT
• Title: FACTWASH: Catching AI Rewrites That Wash Hearsay into Fact
• Abstract summary: AI systems rewrite information constantly: conversations become stored memories, documents become answers.<n>We call that fact-washing, an open-source write-time gate that catches it deterministically, with named flags and evidence rather than an judge.<n>Building it answers a practical question: when does a cheap check suffice, and when do you need a model?
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI systems rewrite information constantly: conversations become stored memories, documents become answers. The rewrite can keep a claim while washing away what made it checkable, who said it, how sure they were, when it held. We call that failure factwashing, and release factwash, an open-source write-time gate that catches it deterministically, with named flags and evidence rather than an LLM judge. Building it answers a practical question: when does a cheap check suffice, and when do you need a model? What decides is whether the property has a bounded surface-cue inventory. Explicit negation cues are close to enumerable, so a word list finishes and transfers, reaching 0.91 F1 on untuned text. Hedging and attribution have open-ended realizations, so vocabulary plateaus near half recall, and a one-question LLM witness recovers +17 and +15 points of cue-detection recall at equal precision. Deployed, that witness may only lower a verdict, so it buys precision rather than coverage. We measure cue detection on 105,596 independently annotated sentences. A blind-labelled corpus of memory writes then locates the failure: 55% of bad writes in conversational hearsay, 7% in business email (p < 0.001), so the first deployment question is not which detector to use but whether the failure occurs at all. On unmodified mem0 2.0.7, the gate flags 5 of 8 hedged-hearsay writes.

Related papers

• From Scores to Evidence: Auditable Decisions Can Improve Speech Deepfake Detection [12.159476982130286]
  Speech deepfakes can mimic a speaker's voice convincingly enough to deceive listeners and automated systems.<n>Most detectors end with one score per utterance, but it says little about why a borderline item should be trusted, deferred, or reviewed.<n>We answer this question with an auditable decision record that carries four aligned cues into a late calibration step.
  arXiv  Detail & Related papers  (2026-09-08T15:38:57Z)
• What the Window Does Not Contain: Auditing Provenance in a Document-Grounded Instability Benchmark [0.0]
  We built a benchmark of 60 tasks and 470 items from real venture-financing filings to measure how often this happens.<n>We audited our own corpus and found a defect any excerpt-built benchmark can carry: items whose evidence is missing from the window of text the model is shown.<n>We release the corpus, all 112,800 raw responses, and the audit as a runnable check for any document benchmark.
  arXiv  Detail & Related papers  (2026-09-05T15:43:05Z)
• VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement [57.86962208273888]
  We present an auditable physical-verification system in which a text-only planner compiles the prompt into typed physical obligations.<n>Each action returns a provenance-carrying evidence record whose payload, when usable, is either a typed measurement or an explicitly tagged learned state.<n>We anchor evaluation in a 1,500-clip corpus of human-annotated flaw records that localize real generation failures in prompt reference, space, and time.
  arXiv  Detail & Related papers  (2026-09-02T20:36:45Z)
• LLM Judges Verify Presence, Not Absence: Omission Blindness in AI Clinical Notes and What Recovers It [0.0]
  Ambient AI scribes draft clinical notes, and published audits find their dominant error is omission.<n>Our benchmark has 500 single rubric-error note pairs from audited fact sheets.<n>On single notes, no design flags omissions reliably more often than perfect notes.
  arXiv  Detail & Related papers  (2026-08-31T15:59:54Z)
• Feedback That Backfires: Why Small Language Model Agents Repeat the Call They Just Watched Fail [0.0]
  Defining the corrective gain of a failure record as the change in log-probability of re-emitting the action that just failed, we find the gain is negative for every instruction-tuned model.<n>The problem is in the harness, not the model's grasp of error messages, and that predicts which remedies work.
  arXiv  Detail & Related papers  (2026-08-24T10:19:17Z)
• Quipu: A Governed Bitemporal Knowledge Graph Store [0.0]
  Quipu is an embeddable knowledge store that inverts four default defaults.<n>No fact enters except through a gate whose predicates evaluate the pending post-state.<n>A trace from a governed writer surfaces a live enforcement gap the audit names with its remediation.
  arXiv  Detail & Related papers  (2026-08-17T17:04:29Z)
• Wrong but Useful: Trajectory Value Beyond Answer Correctness in Multi-Agent Messages [44.340269814027174]
  Multi-agent reasoning systems often use agreement, confidence, or automated scores to decide which messages should shape a final answer.<n>Yet a wrong answer can contain a useful decomposition, constraint, or scientific principle.<n>We test this distinction with Diverse Hypothesis Deliberation (DHD), a controlled measurement protocol that caches five independently generated messages.
  arXiv  Detail & Related papers  (2026-08-14T15:16:57Z)
• Judging Is Not Enumerating: Silent Omissions in LLM-Authored Acceptable Sets [13.85834524293015]
  We measure the capability that role assumes and find it lacking under the protocol the role is usually deployed with.<n>Models judge whether a candidate belongs far better than they author the set itself.
  arXiv  Detail & Related papers  (2026-08-02T05:00:44Z)
• Voice Memory for Agentic Speech Recognition [66.69623133635868]
  We present Voice Memory, a inference-only scheme for agentic speech recognition.<n>Asynchronously, a score-gated example revises that file through bounded edits.<n>Across ten HyPoradise domains with an open corrector, Voice Memory, weighted word error rate from 8.36% to 7.52%.
  arXiv  Detail & Related papers  (2026-07-29T02:42:56Z)
• Manufactured Confidence: How Memory Consolidation Turns Hearsay into Confident Facts [0.0]
  LLM agents rewrite conversation into stored "facts" that later steps trust.<n>A role that was true once and never corrected is stored as a flat fact and acted on like a deliberate injection.<n>A single load-bearing memory is the hazard, and one redundant source restores correct decisions.
  arXiv  Detail & Related papers  (2026-06-28T08:56:48Z)
• Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One [0.0]
  A language model's memory can be worse than no memory at all.<n>A memory that keeps a wrong conclusion but drops the work behind it makes the model emit the stale value as a confident answer.
  arXiv  Detail & Related papers  (2026-06-24T06:24:53Z)
• Catching One in Five: LLM-as-Judge Blind Spots in Production Multi-Turn Transaction Agents [45.148328075418156]
  We study a deployed multi-turn food-and-beverage ordering agent and measure how many genuine quality problems it catches.<n>Our blind-spot taxonomy shows the failure is structured, not random.<n>For production multi-turn agents, automated judging is a regression floor, not a substitute for human review.
  arXiv  Detail & Related papers  (2026-06-09T02:11:01Z)
• Grounded Continuation: A Linear-Time Runtime Verifier for LLM Conversations [15.537674351419234]
  Context-manipulation attacks against deployed agents now actively exploit this gap.<n>We close it with a runtime verifier that maintains an explicit dependency graph.<n>Checking whether a continuation is supported reduces to a graph walk; retraction propagates through the same graph to flag exactly the conclusions that lose support.
  arXiv  Detail & Related papers  (2026-05-13T22:54:16Z)
• Probing for Knowledge Attribution in Large Language Models [45.47366023067617]
  Large language models (LLMs) often generate fluent but unfounded claims, or hallucinations.<n> Proper mitigation depends on knowing whether a model's answer is based on the prompt or its internal weights.<n>We show that a probe, a simple linear classifier trained on model hidden representations, can reliably predict contributive attribution.
  arXiv  Detail & Related papers  (2026-02-26T09:21:12Z)
• Is the Top Still Spinning? Evaluating Subjectivity in Narrative Understanding [40.78154629252038]
  Forcing binary labels upon ambiguous claims lowers the reliability of evaluation.<n>We introduce LLM-generated edits of summaries as a method of providing a nuanced evaluation of claims.<n>We show that ARM produces a absolute 21% improvement in annotator agreement on claim faithfulness.
  arXiv  Detail & Related papers  (2025-04-01T19:08:24Z)
• Improving Bot Response Contradiction Detection via Utterance Rewriting [45.55560596440624]
  This work aims to improve the contradiction detection via rewriting all bot utterances to restore antecedents and ellipsis. We empirically demonstrate that this model can produce satisfactory rewrites to make bot utterances more complete. Using rewritten utterances improves contradiction detection performance significantly, e.g., the AUPR and joint accuracy scores (detecting contradiction along with evidence) increase by 6.5% and 4.5%, respectively.
  arXiv  Detail & Related papers  (2022-07-25T00:54:30Z)
• Claim Check-Worthiness Detection as Positive Unlabelled Learning [53.24606510691877]
  Claim check-worthiness detection is a critical component of fact checking systems. We illuminate a central challenge in claim check-worthiness detection underlying all of these tasks. Our best performing method is a unified approach which automatically corrects for this using a variant of positive unlabelled learning.
  arXiv  Detail & Related papers  (2020-03-05T16:06:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.