When Does Critique Improve AI-Assisted Theoretical Physics? SCALAR: Structured Critic--Actor Loop for Agentic Reasoning

• URL: http://arxiv.org/abs/2605.06772v1
• Date: Thu, 07 May 2026 18:00:01 GMT
• Title: When Does Critique Improve AI-Assisted Theoretical Physics? SCALAR: Structured Critic--Actor Loop for Agentic Reasoning
• Abstract summary: We study how the interaction between researchers and agents affect the results using SCALAR (Structured Critic--Actor Loop for AI Reasoning)<n>The Actor proposes solutions, the Critic provides iterative feedback, and an independent Judge evaluates the transcript against reference solutions.<n>We vary the Actor persona, the Critic feedback strategy, and the Actor model family and scale.
• Score: 39.871701555686634
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As large language models (LLMs) show increasing promise on research-level physics reasoning tasks and agentic AI becomes more common, a practical question emerges: How does the interaction between researchers and agents affect the results? We study this using SCALAR (Structured Critic--Actor Loop for AI Reasoning), an Actor--Critic--Judge pipeline applied to quantum field theory and string theory problems. The Actor proposes solutions, the Critic provides iterative feedback, and an independent Judge evaluates the transcript against reference solutions. We vary the Actor persona, the Critic feedback strategy, and the Actor model family and scale. Multi-turn dialogue improves over single-shot attempts throughout, but both the mechanism of improvement and the value of different prompting choices depend strongly on the Actor--Critic pairing. Increasing the scale within one model family (e.g. from the 8B-parameter DeepSeek-R1 variant to DeepSeek-R1 70B) improves some easier-problem behavior, but does not remove the hardest bottleneck we observe. Critic feedback strategy matters most clearly in the asymmetric Actor--Critic setting (e.g., a lightweight Haiku Actor guided by a stronger Sonnet Critic), where constructive feedback improves mean-score outcomes. In same-family Actor--Critic settings, strategy effects are weaker: lenient feedback is sometimes favored, while strict and adversarial feedback are not beneficial. Taken together, SCALAR provides a controlled testbed for evaluating which interaction structures help or hinder AI-driven scientific discovery.

Related papers

• Critic-R: Improving Agentic Search using Instruction-tuned Retrievers with Natural Language Introspective Feedback [33.73030625999133]
  Critic-R is a framework that closes the feedback loop between the reasoning agent and the retrieval model during both inference and training.<n> Critic-R has two complementary mechanisms: Critic-R-Zero, an inference-time query refinement loop, and Critic-Embed, an optimization approach for retrieval models.<n>Results show that Critic-R significantly improves both retrieval quality and downstream answer accuracy.
  arXiv  Detail & Related papers  (2026-05-30T07:42:22Z)
• ICRL: Learning to Internalize Self-Critique with Reinforcement Learning [29.197505133648047]
  Large language model-based agents make mistakes, yet critique can often guide the same model toward correct behavior.<n>A frozen critic cannot improve its feedback quality over time, limiting the potential for iterative self-improvement.<n>We propose learning to self-critique with reinforcement learning, a novel framework that jointly trains a solver and a critic from a shared backbone.
  arXiv  Detail & Related papers  (2026-05-13T08:50:05Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• Critique-RL: Training Language Models for Critiquing through Two-Stage Reinforcement Learning [89.60378227969643]
  We propose Critique-RL, an online RL approach for developing critiquing language models without stronger supervision.<n>Our approach operates on a two-player paradigm: the actor generates a response, the critic provides feedback, and the actor refines the response accordingly.<n>Experiments across various tasks and models show that Critique-RL delivers substantial performance improvements.
  arXiv  Detail & Related papers  (2025-10-28T11:37:01Z)
• Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization [58.390885294401066]
  Retrieval-augmented generation (RAG) has become a widely adopted paradigm for enabling knowledge-grounded large language models (LLMs)<n>RAG pipelines often fail to ensure that model reasoning remains consistent with the evidence retrieved, leading to factual inconsistencies or unsupported conclusions.<n>We propose AlignRAG, a novel iterative framework grounded in Critique-Driven Alignment (CDA)<n>We introduce AlignRAG-auto, an autonomous variant that dynamically terminates refinement, removing the need to pre-specify the number of critique iterations.
  arXiv  Detail & Related papers  (2025-04-21T04:56:47Z)
• On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows [71.92083784393418]
  Agentic AI (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low.<n>Inference-time alignment relies on three components: sampling, evaluation, and feedback.<n>We introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques.
  arXiv  Detail & Related papers  (2025-04-02T17:40:47Z)
• Intent-Aware Self-Correction for Mitigating Social Biases in Large Language Models [38.1620443730172]
  Self-Correction based on feedback improves the output quality of Large Language Models (LLMs)<n>In this study, we demonstrate that clarifying intentions is essential for effectively reducing biases in LLMs through Self-Correction.
  arXiv  Detail & Related papers  (2025-03-08T02:20:43Z)
• Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision [120.40788744292739]
  We propose a two-player paradigm that separates the roles of reasoning and critique models. We first propose AutoMathCritique, an automated and scalable framework for collecting critique data. We demonstrate that the critique models consistently improve the actor's performance on difficult queries at test-time.
  arXiv  Detail & Related papers  (2024-11-25T17:11:54Z)
• Recursive Chain-of-Feedback Prevents Performance Degradation from Redundant Prompting [0.4662017507844857]
  This paper studies such repetitive behavior and its effect by defining a novel setting, Chain-of-Feedback (CoF) To alleviate these troubles, we propose a novel method, Recursive Chain-of-Feedback (R-CoF)
  arXiv  Detail & Related papers  (2024-02-05T00:44:28Z)
• Quantifying the Effect of Feedback Frequency in Interactive Reinforcement Learning for Robotic Tasks [0.0]
  Reinforcement learning has become widely adopted in robot control. Interactive feedback has been shown to speed up RL considerably. There is no consensus about which feedback frequency is optimal.
  arXiv  Detail & Related papers  (2022-07-20T12:17:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.