Learning the ARTS of Search for Automated Discovery

• URL: http://arxiv.org/abs/2606.21891v1
• Date: Sat, 20 Jun 2026 05:47:37 GMT
• Title: Learning the ARTS of Search for Automated Discovery
• Abstract summary: We propose Agentic Reasoning for Tree Search (ARTS) to navigate scientific discovery.<n>We deploy a reasoning language model to navigate this space.<n>We show that ARTS outperforms leading algorithms.
• Score: 62.89970162621959
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scientific discovery can be formulated as an iterative search process over the space of hypotheses and experiments. Contemporary methods navigate this space using heuristics such as MCTS. These algorithms conflate the merit of a hypothesis with the quality of its experimental execution. A promising hypothesis with preliminary execution is therefore ranked below a modest hypothesis whose execution is refined. Moreover, prior methods prune the search logs as the search progresses because the accumulated history outgrows the context window. We propose Agentic Reasoning for Tree Search (ARTS), where we deploy a reasoning language model to navigate this space. The model inspects prior execution logs, diagnoses whether earlier failures arose from faulty implementations or bad hypotheses, and selects the hypothesis to build on next. To mitigate challenges with context length, ARTS uses test-time training to instill the knowledge of search tree in the model weights. Across 22 tasks from MLGym and MLEBench, we show that ARTS outperforms leading algorithms, with over 15.3% relative improvement in the normalized score. With test-time training we show that a Qwen3-4B agent can match performance with closed-source frontier models like Gemini-3 Pro and GPT o3-reasoning with upto 5x lower inference cost. We further observe that on partially observable RL tasks, the test-time trained Qwen3-4B scientist surpasses ARTS with the o3 scientist by rediscovering the human-best recurrent-memory solution that heuristic methods prune away.

Related papers

• When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Models [92.71325249013535]
  Test-Time Scaling (TTS) has emerged as an important direction for improving reasoning by allocating inference-time compute to a fixed model prior.<n>This survey systematizes recent progress in tree-search-based reasoning, viewing inference as instance-specific optimization rather than decoding.
  arXiv  Detail & Related papers  (2026-08-31T07:49:56Z)
• AREX: Towards a Recursively Self-Improving Agent for Deep Research [75.96468303743958]
  We introduce AREX, a family of Recursively Self-Improving deep research agents.<n>AREX alternates between an inner research loop that gathers evidence and constructs a provisional answer.<n>We train AREX on verified synthetic tasks and high-quality trajectories through agentic mid-training and long-horizon reinforcement learning.
  arXiv  Detail & Related papers  (2026-07-23T16:05:46Z)
• Temporal Backtracking Search for Test-time Generative Video Reasoning [51.4821857044108]
  generative video reasoning remains bottlenecked by a single-shot paradigm.<n>We introduce Temporal Backtracking Search (TBS), which shifts the search space to the temporal axis.<n>In a strict out-of-distribution setting, TBS achieves 22.7%, with every solved episode stemming from a restarted branch.
  arXiv  Detail & Related papers  (2026-06-11T19:41:18Z)
• Retrieval, Reward, and Training Protocols: What Matters in Training Search Agents? [15.605151408619895]
  We present a controlled empirical study that isolates three under-explored dimensions of search agent training.<n>First, we identify a critical data-coverage issue in the widely used Wikipedia 2018 corpus and show that correcting it alone yields larger gains than the differences between training algorithms.<n>Second, we systematically compare outcome-based and process-based reward methods across three base models, finding that the simplest outcome-based approach achieves competitive or superior performance in most settings.<n>Third, we analyze training data diversity, off-policy data utilization, and search budget scaling, distilling practical guidelines for training effective search agents.
  arXiv  Detail & Related papers  (2026-05-27T03:04:36Z)
• AutoResearch-RL: Perpetual Self-Evaluating Reinforcement Learning Agents for Autonomous Neural Architecture Discovery [5.110708177092157]
  We present AutoResearch-RL, a framework in which a reinforcement learning agent conducts open-ended neural architecture research without human supervision.<n>We formalise this as a Markov Decision Process, derive convergence guarantees under mild assumptions, and demonstrate empirically on a single GPU nanochat pretraining benchmark.
  arXiv  Detail & Related papers  (2026-03-07T17:49:44Z)
• Beneficial Reasoning Behaviors in Agentic Search and Effective Post-training to Obtain Them [23.986035712600657]
  We propose a reasoning-driven pipeline to study effective reasoning behavior patterns in agentic search.<n>We identify four beneficial reasoning behaviors: Information Verification, Authority Evaluation, Adaptive Search, and Error Recovery.<n>We show that behavior priming yields over 35% gains in Llama3.2-3B and Qwen3-1.7B compared to directly training agentic search models with RL.
  arXiv  Detail & Related papers  (2025-10-08T00:20:35Z)
• Tree Search for Language Model Agents [73.97960454223164]
  We propose an inference-time search algorithm for LM agents to perform exploration and multi-step planning in interactive web environments.<n>Our approach is a form of best-first tree search that operates within the actual environment space.<n>It is the first tree search algorithm for LM agents that shows effectiveness on realistic web tasks.
  arXiv  Detail & Related papers  (2024-07-01T17:07:55Z)
• Autonomous Tree-search Ability of Large Language Models [58.68735916408101]
  Large Language Models have excelled in remarkable reasoning capabilities with advanced prompting techniques. Recent works propose to utilize external programs to define search logic, such that LLMs can perform passive tree search to solve more challenging reasoning tasks. We propose a new concept called autonomous tree-search ability of LLM, which can automatically generate a response containing search trajectories for the correct answer.
  arXiv  Detail & Related papers  (2023-10-14T14:14:38Z)
• Reasoning-Based Software Testing [9.341830361844337]
  Reasoning-Based Software Testing (RBST) is a new way of thinking at the testing problem as a causal reasoning task. We claim that causal reasoning more naturally emulates the process that a human would do to ''smartly" search the space. Preliminary results reported in this paper are promising.
  arXiv  Detail & Related papers  (2023-03-02T14:27:21Z)
• Learning Optimal Tree Models Under Beam Search [27.92120639502327]
  Existing tree models suffer from the training-testing discrepancy. We develop the concept of Bayes optimality under beam search and calibration under beam search. We propose a novel algorithm for learning optimal tree models under beam search.
  arXiv  Detail & Related papers  (2020-06-27T17:20:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.