Human-on-the-Loop Orchestration for AI-Assisted Legal Discovery

• URL: http://arxiv.org/abs/2606.19812v1
• Date: Thu, 18 Jun 2026 05:37:52 GMT
• Title: Human-on-the-Loop Orchestration for AI-Assisted Legal Discovery
• Abstract summary: We propose a structured taxonomy of agentic failures in legal information retrieval.<n>We introduce a four-layer verification architecture designed to intercept these failures before they compound.<n>Our results suggest that calibrated uncertainty thresholds can reduce privilegewaiver risk by up to 61% versus fully autonomous deployment.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous Large Language Model (LLM) agents are increasingly deployed in electronic discovery (e-discovery), where compounding errors across multi-step reasoning chains can constitute legal malpractice. Unlike single-turn retrieval, agentic workflows operating over privileged document corpora exhibit a class of failure we term "trajectory collapse": an early misclassification silently propagates, rendering an entire privilege review invalid. This paper makes three contributions. First, we propose a structured taxonomy of agentic failures in legal information retrieval, organized by functional stage. Second, we introduce a four-layer verification architecture -- spanning planning, reasoning, execution, and uncertainty quantification -- designed to intercept these failures before they compound. Third, we present a preliminary simulation study on a synthetic e-discovery corpus that demonstrates how mandatory Human-on-the-Loop (HOTL) escalation thresholds reduce privilege-waiver risk relative to fully autonomous baselines. Our results suggest that calibrated uncertainty thresholds can reduce privilege-waiver risk by up to 61% versus fully autonomous deployment, while routing fewer than one quarter of documents to attorney review.

Related papers

• HANSARD: A Reference Architecture for Forensic Readiness, Runtime Witnessing, and Graded Attribution in Autonomous Multi-Agent AI Systems [3.7015952547586832]
  An autonomous multi-agent system nowadays act in finance, software supply chains, and security operations.<n>No method can robustly establish what happened, what caused it, or who is accountable.<n>This is because provenance forensics works at the wrong abstraction, formal causality assumes the causal model, and agent auditing trusts self-recording.<n>The target failure mode is, thus, attribution laundering, spreading an act across redundant agents until none is a but-for cause.<n>This paper proposes HANSARD, a reference architecture treating accountability as a life-cycle property.
  arXiv  Detail & Related papers  (2026-08-23T17:18:22Z)
• Epistemic Norms for AI Safety and Alignment Research [0.989901717499058]
  We argue that the two domains differ along two analytically independent axes.<n>sc ECAISA does not certify that any AI system is safe.
  arXiv  Detail & Related papers  (2026-07-27T10:22:46Z)
• From Regression to Prior-Aware Inference: Solving the ILWE Family in Randomness Leakage Attacks against ML-DSA [11.81046613710983]
  A randomness leakage attack can be formulated as a two-stage key-recovery procedure.<n>This work presents a unified framework to evaluate different recovery solvers on leakage-derived ILWE-family instances.<n>Our experiments show that the solver has a significant impact on the secret-key recovery efficiency.
  arXiv  Detail & Related papers  (2026-07-07T07:20:32Z)
• Provably Secure Agent Guardrail [89.79561918065122]
  Existing defense architectures rely on empirical semantic guardrails and probabilistic large model adjudicators.<n>This paper proposes a new security paradigm for agents based on the fundamental limitations of logical reasoning.
  arXiv  Detail & Related papers  (2026-05-28T02:12:41Z)
• PAVE: A Cognitive Architecture for Legitimate Violation in Generative Agent Societies [3.8300637171362752]
  Generative agents can reproduce believable human behavior in cooperative settings, but how they should reason in situations where rule-breaking may be required remains poorly characterized.<n>We propose PAVE, a novel four- module cognitive architecture that addresses this gap end to end.<n>We instantiate PAVE in Voville, a tile-based traffic environment forked from Smallville, and evaluate across three scenarios.
  arXiv  Detail & Related papers  (2026-05-19T04:40:09Z)
• A2RBench: An Automatic Paradigm for Formally Verifiable Abstract Reasoning Benchmark Generation [59.98516959731531]
  Abstract reasoning ability reflects intelligence and capacity of LLMs to extract and apply abstract rules.<n>Existing benchmarks either rely on expensive manual annotation, limiting their scale, or risk measuring memorization rather than genuine reasoning.<n>We introduce an automated pipeline named A2RBench, encompassing generation, expansion, evaluation, and analysis.
  arXiv  Detail & Related papers  (2026-05-17T06:14:20Z)
• PRISM: : Planning and Reasoning with Intent in Simulated Embodied Environments [59.07829883257003]
  Built on five multi-room apartments, PRISM structures 300 human-verified tasks into three capability tiers.<n>PRISM exposes an agent-agnostic executable action API that allows arbitrary agents to be evaluated end-to-end.
  arXiv  Detail & Related papers  (2026-05-12T04:59:47Z)
• TwinGate: Stateful Defense against Decompositional Jailbreaks in Untraceable Traffic via Asymmetric Contrastive Learning [60.68349524623048]
  Decompositional jailbreaks pose a critical threat to large language models.<n>We present TwinGate, a stateful dual-encoder defense framework.<n>We construct a comprehensive dataset of over 3.62 million instructions spanning 8,600 distinct malicious intents.
  arXiv  Detail & Related papers  (2026-04-30T13:44:01Z)
• Agentic Adversarial Rewriting Exposes Architectural Vulnerabilities in Black-Box NLP Pipelines [34.37967130277969]
  We propose a two-agent evasion framework operating in a semantic perturbation space.<n>Evasion rates are 19.95 to 40.34% on modern large language model (LLM) based systems.<n>A pattern-informed defense reduces the evasion rate by up to 65.18%.
  arXiv  Detail & Related papers  (2026-04-26T00:58:36Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• AEGIS: From Clues to Verdicts -- Graph-Guided Deep Vulnerability Reasoning via Dialectics and Meta-Auditing [9.271196825503417]
  Large Language Models (LLMs) are increasingly adopted for vulnerability detection, yet their reasoning remains fundamentally unsound.<n>We present AEGIS, a novel multi-agent framework that shifts detection from ungrounded speculation to forensic verification over a closed factual substrate.<n>It reduces the false positive rate by up to 54.40% compared to leading baselines, at an average cost of $0.09 per sample without any task-specific training.
  arXiv  Detail & Related papers  (2026-03-21T04:12:04Z)
• To Throw a Stone with Six Birds: On Agents and Agenthood [0.0]
  Six Birds Theory (SBT) treats macroscopic objects as induced closures rather than primitives.<n>We give a type-correct account of agency within SBT.<n>We operationalize this contract in finite controlled systems using four checkable components.
  arXiv  Detail & Related papers  (2026-02-03T10:46:23Z)
• Making LLMs Reliable When It Matters Most: A Five-Layer Architecture for High-Stakes Decisions [51.56484100374058]
  Current large language models (LLMs) excel in verifiable domains where outputs can be checked before action but prove less reliable for high-stakes strategic decisions with uncertain outcomes.<n>This gap, driven by mutually cognitive biases in both humans and artificial intelligence (AI) systems, threatens the defensibility of valuations and sustainability of investments in the sector.<n>This report describes a framework emerging from systematic qualitative assessment across 7 frontier-grade LLMs and 3 market-facing venture vignettes under time pressure.
  arXiv  Detail & Related papers  (2025-11-10T22:24:21Z)
• VeriLLM: A Lightweight Framework for Publicly Verifiable Decentralized Inference [3.8760740008451156]
  We introduce VeriLLM, a publicly verifiable protocol for decentralized language models (LLMs) inference.<n>VeriLLM combines lightweight empirical rerunning with cryptographic commitments, allowing verifiers to validate results at approximately 1% of the underlying inference cost.<n>We show that VeriLLM achieves reliable public verifiability with minimal overhead.
  arXiv  Detail & Related papers  (2025-09-29T04:07:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.