AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning

• URL: http://arxiv.org/abs/2606.24526v1
• Date: Tue, 23 Jun 2026 12:57:18 GMT
• Title: AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning
• Abstract summary: Large language models are increasingly deployed as agents that reason over documents rather than answer from parametric knowledge.<n>We study archive-grounded reasoning: locating sparse evidence across a large, messy collection of workplace files, reconciling inconsistent terminology, units, and time conventions, and computing an answer.<n>We introduce Agora, a benchmark pairing 362 questions with eight domain collections of 9,664 authentic documents and 372M tokens, far exceeding any model's context window.
• Score: 52.23991730630292
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models are increasingly deployed as agents that reason over documents rather than answer from parametric knowledge. We study archive-grounded reasoning: locating sparse evidence across a large, messy collection of workplace files, reconciling inconsistent terminology, units, and time conventions, and computing an answer. Existing benchmarks address only parts of this setting and none jointly stresses archive-groundedness, agentic exploration, and cross-domain coverage. We introduce Agora, a benchmark pairing 362 questions with eight domain collections of 9,664 authentic documents and 372M tokens, far exceeding any model's context window, so agents must explore deliberately rather than scan exhaustively. Agora is built by an agentic pipeline combining cross-document task synthesis, leakage-preventing obfuscation, and difficulty filtering. Evaluating eight models, we find the task far from solved: even the strongest reaches only 59.4% accuracy, with notable variation across domains.

Related papers

• You Know What I Mean: A Benchmark for Agentic Conversational Reference Grounding [25.12009255757602]
  Collaborative conversations often contain references whose targets are indirect rather than named.<n>We formalize this problem as Conversational Reference Grounding (CoRG)<n>CoRG combines lexical, semantic, and temporal cues distributed across the conversation and the external workspace.<n>We study CoRG through RepoRef, a benchmark of 400 developer-chat segments grounded in GitHub issues, pull requests, and commits.
  arXiv  Detail & Related papers  (2026-08-30T15:06:51Z)
• Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction [57.138342889101345]
  Tencent WorkBuddy Bench is a multi-domain evaluation suite for coding agents.<n>This report documents its construction methodology, scoring protocol, and a cross-model leaderboard.
  arXiv  Detail & Related papers  (2026-07-23T04:34:06Z)
• Towards Retrieving Interaction Spaces for Agentic Search [62.666902610940525]
  Recent direct corpus (DCI) work shows that agents can interact with the raw corpus through shell tools such as 5.4 interaction and file reads.<n>We argue that the role of retrieval for agentic search is not just to select documents that fit in the LLM context window, but to construct an interaction space.
  arXiv  Detail & Related papers  (2026-06-05T03:47:40Z)
• MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA [23.25080415779712]
  We propose a Memory-Aware Refinement Agent framework that decouples long-document QA into three specialized agents.<n>Across iterations, the agents rely on a dynamically updated structured memory rather than a full accumulated interaction history.<n>Experiments on MMLongBench-Doc and DocBench show that MARDoc achieves strong results, outperforming same-backbone baselines.
  arXiv  Detail & Related papers  (2026-06-04T06:23:01Z)
• Self-Conditioned Positional HNSW for Overlap-Aware Retrieval in Chunked-Document RAG Systems: Method and Industrial Evidence-Quality Audit [0.0]
  Chunked-document retrieval is a common component of retrieval-augmented generation (RAG) systems.<n>We propose Self-Conditioned Positional HNSW, a lightweight modification that appends a low-dimensional positional code to chunk embeddings.
  arXiv  Detail & Related papers  (2026-06-01T01:42:55Z)
• REVEAL: Reference-Grounded Reasoning for Multimodal Manipulation Detection [33.33464433821003]
  Multimodal manipulation detection aims to simultaneously identify forged image--text pairs and localize tampered regions.<n>Inspired by human comparative reasoning, we reformulate this task as a reference-grounded verification problem.<n>We propose REVEAL, a framework explicitly designed for this comparative paradigm.
  arXiv  Detail & Related papers  (2026-05-27T13:24:41Z)
• OBLIQ-Bench: Exposing Overlooked Bottlenecks in Modern Retrievers with Latent and Implicit Queries [20.960193343595492]
  OBLIQ-Bench is a suite of five oblique search problems over real long-tail corpora.<n>OBLIQ-Bench exposes an overlooked asymmetry between retrieval and verification.<n>We hope that OBLIQ-Bench will drive research into retrieval architectures that efficiently capture latent patterns and implicit signals in large corpora.
  arXiv  Detail & Related papers  (2026-05-07T13:22:49Z)
• SPD-RAG: Sub-Agent Per Document Retrieval-Augmented Generation [0.0]
  We introduce SPD-RAG, a hierarchical multi-agent framework for exhaustive cross-document question answering.<n>Each document is processed by a dedicated document-level agent operating only on its own content, enabling focused retrieval.<n>SPD-RAG achieves an Avg Score of 58.1 (GPT-5 evaluation), outperforming Normal RAG (33.0) and Agentic RAG (32.8) while using only 38% of the API cost of a full-context baseline (68.0)
  arXiv  Detail & Related papers  (2026-03-09T12:46:32Z)
• DAPR: A Benchmark on Document-Aware Passage Retrieval [57.45793782107218]
  We propose and name this task emphDocument-Aware Passage Retrieval (DAPR) While analyzing the errors of the State-of-The-Art (SoTA) passage retrievers, we find the major errors (53.5%) are due to missing document context. Our created benchmark enables future research on developing and comparing retrieval systems for the new task.
  arXiv  Detail & Related papers  (2023-05-23T10:39:57Z)
• Generate rather than Retrieve: Large Language Models are Strong Context Generators [74.87021992611672]
  We present a novel perspective for solving knowledge-intensive tasks by replacing document retrievers with large language model generators. We call our method generate-then-read (GenRead), which first prompts a large language model to generate contextutal documents based on a given question, and then reads the generated documents to produce the final answer.
  arXiv  Detail & Related papers  (2022-09-21T01:30:59Z)
• Fast(er) Reconstruction of Shredded Text Documents via Self-Supervised Deep Asymmetric Metric Learning [62.34197797857823]
  A central problem in automatic reconstruction of shredded documents is the pairwise compatibility evaluation of the shreds. This work proposes a scalable deep learning approach for measuring pairwise compatibility in which the number of inferences scales linearly. Our method has accuracy comparable to the state-of-the-art with a speed-up of about 22 times for a test instance with 505 shreds.
  arXiv  Detail & Related papers  (2020-03-23T03:22:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.