CausalForge: A Formally Grounded, Self-Improving Agentic Framework for Automated Research in Causal Inference

• URL: http://arxiv.org/abs/2607.22511v1
• Date: Fri, 24 Jul 2026 17:32:35 GMT
• Title: CausalForge: A Formally Grounded, Self-Improving Agentic Framework for Automated Research in Causal Inference
• Abstract summary: CausalForge is a framework for automated theoretical research in causal inference.<n>It combines Causalean, a Lean library for causal inference, with CausalSmith, a self-improving agentic pipeline.
• Score: 17.447741518678374
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automating theoretical research is constrained not only by the generation of candidate results, but also by their reliable evaluation. A common approach is to close the research loop with a large language model (LLM) reviewer. However, such reviewers remain empirically unreliable: they may accept fabricated papers and detect them at rates close to chance (Bad Scientist, 2025). We present CausalForge, a framework for automated theoretical research in causal inference grounded in the Lean proof assistant. CausalForge combines Causalean, a foundational Lean library for causal inference containing 7,035 machine-checked declarations developed with language-model assistance under human design and review, with CausalSmith, a self-improving agentic pipeline that selects research topics, proposes results, formalizes statements, constructs proofs, and presents the resulting artifacts for human inspection. Because a machine-checked proof establishes only that a formal statement follows from its assumptions, not that the statement faithfully captures the intended scientific claim, the pipeline augments kernel verification with a statement audit that compares each formal theorem against the informal claim it is intended to express. We evaluate the system using artifacts produced by completed autonomous research runs. The source code, formal library, and run records are available at https://github.com/Jiyuan-Tan/CausalForge.

Related papers

• DERELAB: Probing Defeasible Reasoning and Confirmation Bias in LLMs with a Generative Benchmark [6.835919520017402]
  Defeasible reasoning is a type of reasoning where inferences are drawn from plausible current evidence, but can be retracted upon the introduction of newer evidence.<n>We introduce DeReLab, a generative framework that produces multi-turn belief-updating conversations from parameterized graph structures.<n>We evaluate nine open and proprietary large language models and find that nearly all exhibit a systematic tendency to accept congruent evidence while resisting incongruent updates.
  arXiv  Detail & Related papers  (2026-08-31T08:10:29Z)
• FormalTCS: Benchmarking End-to-End Frontier Formal Theoretical Computer Science Research of Large Language Models [53.68149869349268]
  Large language models (LLMs) have shown growing potential for automated theoretical computer science (TCS) research.<n>We introduce ourbenchmark, an expert-validated benchmark for evaluating LLMs on frontier, end-to-end TCS research.
  arXiv  Detail & Related papers  (2026-08-20T15:13:41Z)
• TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs [0.0]
  Large language models are being proposed as agents in scientific, in domains where no downstream verifier exists.<n>We introduce a probe corpus of 42 retracted, fraudulent, and pseudoscientific papers, paired with a methodology for eliciting and scoring single-shot model engagement with each paper's framing.<n>The probes span five claim types: fabricated observation, pseudophysical mechanism, magical premise, legitimization bridge, and cargo-cult experiment.
  arXiv  Detail & Related papers  (2026-08-11T20:30:32Z)
• TCS-BENCH: Benchmarking State-of-the-Art Generative AI Theoretical Computer Science Research Ability [59.079078796751496]
  We introduce TCS-Bench, a benchmark for evaluating Large Language Models (LLMs) on research-level Theoretical Computer Science (TCS) proof generation.<n> TCS-Bench consists of theorem-proving tasks from papers published at top theoretical computer science venues.
  arXiv  Detail & Related papers  (2026-08-10T12:36:16Z)
• The Calibration Turn in AI-Assisted Research: A Conceptual and Methodological Framework for Evidence-Licensed Claims [0.9805949492148788]
  This paper develops a conceptual framework for evidence-licensed claims in AI-assisted research.<n>It represents AI-assisted research as five operators: hypothesis generation, model-mediated consequence derivation, external validation, belief update, and claim calibration.
  arXiv  Detail & Related papers  (2026-06-30T07:46:54Z)
• Causal Discovery in the Era of Agents [16.999301469208877]
  We argue for a different role for agents in causal discovery.<n>We propose the principle that agents assist the workflow, while causal claims remain grounded in data, explicit assumptions, formal algorithms, diagnostics and user or domain-expert decisions.<n>We instantiate this principle in causal-learn+, an online platform that coordinates data analysis, preprocessing, method recommendation, expert-knowledge incorporation, formal discovery and interpretation around the algorithmic ecosystem of causal-learn.
  arXiv  Detail & Related papers  (2026-06-22T17:09:22Z)
• ResearchLoop: An Evidence-Gated Control Plane for AI-Assisted Research [4.9833735627186435]
  ResearchLoop is an evidence-gated control plane for AI-assisted computational research.<n>ResearchLoop treats research questions, task contracts, evidence objects, claim ledgers, closeouts, and paper bindings as durable project state.<n>This technical report provides the complete protocol specification, state model, transition rules, claim-admission algorithm, and insight-compounding mechanism.
  arXiv  Detail & Related papers  (2026-05-27T10:29:00Z)
• FactReview: Evidence-Grounded Reviews with Literature Positioning and Execution-Based Claim Verification [57.196748998757954]
  We present FactReview, an evidence-grounded reviewing system that combines claim extraction, literature positioning, and execution-based claim verification.<n>Given a submission, FactReview identifies major claims and reported results, retrieves nearby work to clarify the paper's technical position, and, when code is available, executes the released repository.<n>It then produces a concise review and an evidence report that assigns each major claim one of five labels: Supported, Supported by the paper, Partially supported, In conflict, or Inconclusive.
  arXiv  Detail & Related papers  (2026-04-05T11:45:22Z)
• CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era [51.63024682584688]
  Large language models (LLMs) introduce a new risk: fabricated references that appear plausible but correspond to no real publications.<n>We present the first comprehensive benchmark and detection framework for hallucinated citations in scientific writing.<n>Our framework significantly outperforms prior methods in both accuracy and interpretability.
  arXiv  Detail & Related papers  (2026-02-26T19:17:39Z)
• From Fluent to Verifiable: Claim-Level Auditability for Deep Research Agents [8.49451413641847]
  We argue that as research generation becomes cheap, auditability becomes the bottleneck.<n>This perspective proposes claim-level auditability as a first-class design and evaluation target for deep research agents.
  arXiv  Detail & Related papers  (2026-02-14T19:39:15Z)
• Project Ariadne: A Structural Causal Framework for Auditing Faithfulness in LLM Agents [0.0]
  We introduce textbfProject Ariadne, a novel XAI framework to audit the causal integrity of agentic reasoning.<n>Unlike existing interpretability methods that rely on surface-level textual similarity, Project Ariadne performs textbfhard interventions ($do$-calculus) on intermediate reasoning nodes.<n>Our empirical evaluation of state-of-the-art models reveals a persistent textitFaithfulness Gap.
  arXiv  Detail & Related papers  (2026-01-05T18:05:29Z)
• FIRE: Fact-checking with Iterative Retrieval and Verification [63.67320352038525]
  FIRE is a novel framework that integrates evidence retrieval and claim verification in an iterative manner.<n>It achieves slightly better performance while reducing large language model (LLM) costs by an average of 7.6 times and search costs by 16.5 times.<n>These results indicate that FIRE holds promise for application in large-scale fact-checking operations.
  arXiv  Detail & Related papers  (2024-10-17T06:44:18Z)
• Lean-STaR: Learning to Interleave Thinking and Proving [53.923617816215774]
  We present Lean-STaR, a framework for training language models to produce informal thoughts prior to each step of a proof.<n>Lean-STaR achieves state-of-the-art results on the miniF2F-test benchmark within the Lean theorem proving environment.
  arXiv  Detail & Related papers  (2024-07-14T01:43:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.