ReCoQA: A Benchmark for Tool-Augmented and Multi-Step Reasoning in Real Estate Question and Answering

• URL: http://arxiv.org/abs/2604.17944v1
• Date: Mon, 20 Apr 2026 08:22:18 GMT
• Title: ReCoQA: A Benchmark for Tool-Augmented and Multi-Step Reasoning in Real Estate Question and Answering
• Abstract summary: ReCoQA is a benchmark of 29,270 real-estate instances featuring machine-verifiable supervision for intermediate steps.<n>HIRE-Agent is a hierarchical framework instantiating an understand-plan-execute architecture as a strong baseline.
• Score: 9.45185831385459
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Developing agents capable of navigating fragmented, multi-source information remains challenging, primarily due to the scarcity of benchmarks reflecting hybrid workflows combining database querying with external APIs. To bridge this gap, we introduce ReCoQA, a large-scale benchmark of 29,270 real-estate instances featuring machine-verifiable supervision for intermediate steps, including structured intent labels, SQL queries, and API calls. Complementarily, we propose HIRE-Agent, a hierarchical framework instantiating an understand-plan-execute architecture as a strong baseline. By orchestrating a Front-end parser, a planning Supervisor, and execution Specialists, HIRE-Agent effectively integrates heterogeneous evidence. Extensive experiments demonstrate that HIRE-Agent constitutes a strong baseline and substantiates the necessity of hierarchical collaboration for complex, real-world reasoning tasks.

Related papers

• AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities [55.23913617185508]
  Agent is an open-source, lightweight, and infrastructure for evaluating Large Language Models (LLMs)-based agents.<n>It organizes the evaluation process around three independent components, namely Benchmark, Harness and Environment.<n>It features a fault-tolerant runtime and comprehensive trajectory analysis tools to transparently diagnose failure modes like reward-hacking.
  arXiv  Detail & Related papers  (2026-07-15T11:11:17Z)
• The Illusion of Agentic Complexity in README.md Generation: Evaluating Single-Agent vs. Multi-Agent RAG Systems [3.7149543017244238]
  This study empirically evaluates Retrieval-Augmented Generation (RAG) dependent architectures for the generation of files for GitHub repositories.<n>Results indicate a critical architectural trade-off: the Single-Agent pipeline achieves lexical quality comparable to MAS while reducing token consumption by 86% and operating at twice the speed.
  arXiv  Detail & Related papers  (2026-06-29T16:29:31Z)
• Securing the Agent: Vendor-Neutral, Multitenant Enterprise Retrieval and Tool Use [0.0]
  We introduce a layered isolation architecture combining policy-aware ingestion, retrieval-time gating, and shared inference.<n>This approach centralizes security-critical operations--tool execution authorization, state isolation, and policy enforcement--on the server.<n>We evaluate it empirically and show that ABAC gating eliminates cross-tenant leakage while introducing negligible overhead.
  arXiv  Detail & Related papers  (2026-05-06T17:59:21Z)
• An Empirical Study of Multi-Agent Collaboration for Automated Research [41.906658558789545]
  We present a study investigating the comparative efficacy of distinct multi-agent structures for automated machine learning optimization.<n>We benchmark a single-agent baseline against two multi-agent paradigms: a subagent architecture and an agent team architecture.<n>Our findings reveal a fundamental trade-off between operational stability and theoretical deliberation.
  arXiv  Detail & Related papers  (2026-03-31T11:57:00Z)
• SOPRAG: Multi-view Graph Experts Retrieval for Industrial Standard Operating Procedures [15.42553917257021]
  SOPRAG is a novel framework specifically designed to address the pain points in SOP retrieval.<n> SOPRAG replaces flat chunking with specialized Entity, Causal, and Flow graph experts.<n> SOPRAG significantly outperforms strong lexical, dense, and graph-based RAG baselines in both retrieval accuracy and response utility.
  arXiv  Detail & Related papers  (2026-02-02T09:30:43Z)
• Beyond Monolithic Architectures: A Multi-Agent Search and Knowledge Optimization Framework for Agentic Search [56.78490647843876]
  Agentic search has emerged as a promising paradigm for complex information seeking by enabling Large Language Models (LLMs) to interleave reasoning with tool use.<n>We propose bfM-ASK, a framework that explicitly decouples agentic search into two complementary roles: Search Behavior Agents, which plan and execute search actions, and Knowledge Management Agents, which aggregate, filter, and maintain a compact internal context.
  arXiv  Detail & Related papers  (2026-01-08T08:13:27Z)
• Designing Domain-Specific Agents via Hierarchical Task Abstraction Mechanism [61.01709143437043]
  We introduce a novel agent design framework centered on a Hierarchical Task Abstraction Mechanism (HTAM)<n>Specifically, HTAM moves beyond emulating social roles, instead structuring multi-agent systems into a logical hierarchy that mirrors the intrinsic task-dependency graph of a given domain.<n>We instantiate this framework as EarthAgent, a multi-agent system tailored for complex geospatial analysis.
  arXiv  Detail & Related papers  (2025-11-21T12:25:47Z)
• RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems [31.4909149697414]
  Retrieval-Augmented Generation (RAG) mitigates key limitations of Large Language Models (LLMs)<n>Recent work extends this paradigm through agentic RAG systems, where LLMs act as agents to iteratively plan, retrieve, and reason over complex queries.<n>We propose RAGCap-Bench, a capability-oriented benchmark for fine-grained evaluation of intermediate tasks in agentic RAG.
  arXiv  Detail & Related papers  (2025-10-15T04:13:00Z)
• AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answering [51.07491603393163]
  tAgent is a framework that formulates multi-agent QA as a knowledge-graph-guided routing problem supervised by empirical performance signals.<n>By leveraging soft supervision and weighted aggregation of agent outputs, Agent learns principled collaboration schemes that capture the complementary strengths of diverse agents.
  arXiv  Detail & Related papers  (2025-10-06T23:20:49Z)
• HiRA: A Hierarchical Reasoning Framework for Decoupled Planning and Execution in Deep Search [85.12447821237045]
  HiRA is a hierarchical framework that separates strategic planning from specialized execution.<n>Our approach decomposes complex search tasks into focused subtasks, assigns each subtask to domain-specific agents equipped with external tools and reasoning capabilities.<n> Experiments on four complex, cross-modal deep search benchmarks demonstrate that HiRA significantly outperforms state-of-the-art RAG and agent-based systems.
  arXiv  Detail & Related papers  (2025-07-03T14:18:08Z)
• Deep Research Agents: A Systematic Examination And Roadmap [109.53237992384872]
  Deep Research (DR) agents are designed to tackle complex, multi-turn informational research tasks.<n>In this paper, we conduct a detailed analysis of the foundational technologies and architectural components that constitute DR agents.
  arXiv  Detail & Related papers  (2025-06-22T16:52:48Z)
• HALO: Hierarchical Autonomous Logic-Oriented Orchestration for Multi-Agent LLM Systems [1.1930434318557155]
  We introduce HALO, a multi-agent collaboration framework based on a hierarchical reasoning architecture.<n>Specifically, we incorporate a high-level planning agent for task decomposition, mid-level role-design agents for subtask-specific agent instantiation, and low-level inference agents for subtask execution.<n>As the majority of users lack expertise in prompt engineering, we leverage an Adaptive Prompt Refinement module to transform raw queries into task-specific prompts.
  arXiv  Detail & Related papers  (2025-05-17T04:14:03Z)
• HM-RAG: Hierarchical Multi-Agent Multimodal Retrieval Augmented Generation [11.53083922927901]
  HM-RAG is a novel Hierarchical Multi-agent Multimodal RAG framework.<n>It pioneers collaborative intelligence for dynamic knowledge synthesis across structured, unstructured, and graph-based data.
  arXiv  Detail & Related papers  (2025-04-13T06:55:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.