ECLAIR: A Causally-Grounded AI Framework for Scientific Discovery in Empirical Software Engineering

• URL: http://arxiv.org/abs/2608.02323v2
• Date: Tue, 04 Aug 2026 20:57:05 GMT
• Title: ECLAIR: A Causally-Grounded AI Framework for Scientific Discovery in Empirical Software Engineering
• Abstract summary: This paper introduces ECLAIR, a causally grounded AI framework that integrates Large Language Models into every stage of the scientific process.<n>ECLAIR treats LLMs as active scientific agents operating under the principles of causal inference.
• Score: 51.19819833971252
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The scientific method has long guided empirical research in Software Engineering (SE), but the complexity of modern software systems often hinders its systematic application. This paper introduces ECLAIR, a causally grounded AI framework that integrates Large Language Models (LLMs) into every stage of the scientific process, from hypothesis generation to analysis and interpretation. ECLAIR treats LLMs as active scientific agents operating under the principles of causal inference, within a human-in-the-loop design that safeguards against the risks of unsound automated reasoning. We demonstrate the framework through a case study examining how prompt design influences code generation accuracy in two LLMs. Results show that, for both models, instruction-style, longer few-shot, and signature-augmented prompts yield small negative causal effects on accuracy, illustrating how causal reasoning provides a principled foundation for explaining why software phenomena occur. This study presents the first causally grounded structured methodology for embedding LLMs within the scientific method in SE, designed around the epistemological demands of empirical SE research, establishing a basis for rigorous AI-assisted research.

Related papers

• CDFM: Towards a General-Purpose Causal Discovery Foundation Model [62.32138962508216]
  Causal discovery is the process of recovering underlying causal structures from observational data.<n>We formulate the Causal Discovery Foundation Model (CDFM) as a unified, general-purpose framework for zero-shot structural inference.<n>CDFM consistently outperforms traditional algorithms, driving a paradigm shift toward a general-purpose causal discovery foundation model.
  arXiv  Detail & Related papers  (2026-07-13T12:58:28Z)
• Rethinking Software Empirical Studies with Structural Causal Models [12.18241004961061]
  Causal Inference offers a fundamental approach for advancing empirical software engineering (ESE) beyond traditional statistical association.<n>This paper introduces CausalSE, a framework that operationalizes Judea Pearl's causal inference paradigm in ESE context.
  arXiv  Detail & Related papers  (2026-05-27T13:41:05Z)
• Scientific Logicality Enriched Methodology for LLM Reasoning: A Practice in Physics [32.822972062156005]
  We make the first systematic investigation into the internal logicality underlying Large Language Models (LLMs) scientific reasoning.<n>For data construction, we extract scientific problems from academic literature and sample a high-quality dataset exhibiting strong logicality.<n>Experiments reveal that: 1) the training data we constructed can effectively improve the scientific logicality in LLM reasoning; and 2) the enriched scientific logicality plays a critical role in solving scientific problems.
  arXiv  Detail & Related papers  (2026-05-16T18:15:10Z)
• BABE: Biology Arena BEnchmark [51.53220868983288]
  BABE is a benchmark designed to evaluate the experimental reasoning capabilities of biological AI systems.<n>Our benchmark provides a robust framework for assessing how well AI systems can reason like practicing scientists.
  arXiv  Detail & Related papers  (2026-02-05T16:39:20Z)
• Beyond the Black Box: Theory and Mechanism of Large Language Models [39.10631426330405]
  The rapid emergence of Large Language Models (LLMs) has precipitated a profound paradigm shift in Artificial Intelligence.<n>This survey proposes a unified lifecycle-based taxonomy that organizes the research landscape into six distinct stages: Data Preparation, Model Preparation, Training, Alignment, Inference, and Evaluation.
  arXiv  Detail & Related papers  (2026-01-06T10:45:53Z)
• Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows [203.3527268311731]
  We present an operational SGI definition grounded in the Practical Inquiry Model (PIM)<n>We operationalize it via four scientist-aligned tasks: deep research, idea generation, dry/wet experiments, and experimental reasoning.<n>Our PIM-grounded definition, workflow-centric benchmark, and empirical insights establish a foundation for AI systems that genuinely participate in scientific discovery.
  arXiv  Detail & Related papers  (2025-12-18T12:44:36Z)
• Evaluating Large Language Models in Scientific Discovery [91.732562776782]
  Large language models (LLMs) are increasingly applied to scientific research, yet prevailing science benchmarks probe decontextualized knowledge.<n>We introduce a scenario-grounded benchmark that evaluates LLMs across biology, chemistry, materials, and physics.<n>The framework assesses models at two levels: (i) question-level accuracy on scenario-tied items and (ii) project-level performance.
  arXiv  Detail & Related papers  (2025-12-17T16:20:03Z)
• Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning [53.82037883518254]
  We introduce SciReas, a diverse suite of existing benchmarks for scientific reasoning tasks.<n>We then propose KRUX, a probing framework for studying the distinct roles of reasoning and knowledge in scientific tasks.
  arXiv  Detail & Related papers  (2025-08-26T17:04:23Z)
• Causal-Copilot: An Autonomous Causal Analysis Agent [25.5944359329613]
  Causal-Copilot is an autonomous agent that operationalizes expert-level causal analysis.<n>It supports interactive refinement through natural language, lowering the barrier for non-specialists.<n>Our system fosters a virtuous cycle -- expanding access to advanced causal methods for domain experts.
  arXiv  Detail & Related papers  (2025-04-17T18:05:39Z)
• Causality can systematically address the monsters under the bench(marks) [64.36592889550431]
  Benchmarks are plagued by various biases, artifacts, or leakage.<n>Models may behave unreliably due to poorly explored failure modes.<n> causality offers an ideal framework to systematically address these challenges.
  arXiv  Detail & Related papers  (2025-02-07T17:01:37Z)
• Failure Modes of LLMs for Causal Reasoning on Narratives [51.19592551510628]
  We investigate the interaction between world knowledge and logical reasoning.<n>We find that state-of-the-art large language models (LLMs) often rely on superficial generalizations.<n>We show that simple reformulations of the task can elicit more robust reasoning behavior.
  arXiv  Detail & Related papers  (2024-10-31T12:48:58Z)
• Targeted Reduction of Causal Models [55.11778726095353]
  Causal Representation Learning offers a promising avenue to uncover interpretable causal patterns in simulations. We introduce Targeted Causal Reduction (TCR), a method for condensing complex intervenable models into a concise set of causal factors. Its ability to generate interpretable high-level explanations from complex models is demonstrated on toy and mechanical systems.
  arXiv  Detail & Related papers  (2023-11-30T15:46:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.