CAR-bench: Evaluating the Consistency and Limit-Awareness of LLM Agents under Real-World Uncertainty

• URL: http://arxiv.org/abs/2601.22027v1
• Date: Thu, 29 Jan 2026 17:33:42 GMT
• Title: CAR-bench: Evaluating the Consistency and Limit-Awareness of LLM Agents under Real-World Uncertainty
• Abstract summary: Existing benchmarks for Large Language Model (LLM) agents focus on task completion under idealistic settings.<n>We introduce CAR-bench, a benchmark for evaluating consistency, uncertainty handling, and capability awareness in multi-turn, tool-using LLM agents in an in-car assistant domain.
• Score: 8.976559755301658
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Existing benchmarks for Large Language Model (LLM) agents focus on task completion under idealistic settings but overlook reliability in real-world, user-facing applications. In domains, such as in-car voice assistants, users often issue incomplete or ambiguous requests, creating intrinsic uncertainty that agents must manage through dialogue, tool use, and policy adherence. We introduce CAR-bench, a benchmark for evaluating consistency, uncertainty handling, and capability awareness in multi-turn, tool-using LLM agents in an in-car assistant domain. The environment features an LLM-simulated user, domain policies, and 58 interconnected tools spanning navigation, productivity, charging, and vehicle control. Beyond standard task completion, CAR-bench introduces Hallucination tasks that test agents' limit-awareness under missing tools or information, and Disambiguation tasks that require resolving uncertainty through clarification or internal information gathering. Baseline results reveal large gaps between occasional and consistent success on all task types. Even frontier reasoning LLMs achieve less than 50% consistent pass rate on Disambiguation tasks due to premature actions, and frequently violate policies or fabricate information to satisfy user requests in Hallucination tasks, underscoring the need for more reliable and self-aware LLM agents in real-world settings.

Related papers

• AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments [84.8170304874972]
  We present AgentGym2, a new evaluation framework with task instances grounded in real-world working demands.<n>It measures agents' ability to execute end-to-end procedures, discover tools via exploration, compose tools for unseen tasks, and remain robust to noisy and underspecified information.
  arXiv  Detail & Related papers  (2026-07-06T14:56:51Z)
• Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agents [4.301199871195023]
  Large Language Model (LLM) agents are increasingly deployed in open-ended domains like software engineering.<n>We evaluate the clarification-seeking abilities of LLM agents on an underspecified variant of SWE-bench Verified.<n>We propose an uncertainty-aware multi-agent scaffold that explicitly decouples underspecification detection from code execution.
  arXiv  Detail & Related papers  (2026-03-27T09:56:26Z)
• Benchmarking LLM Tool-Use in the Wild [10.664145474355445]
  Real user interactions are inherently wild, being intricate, messy, and flexible.<n>We identify three key challenges from user behaviour: compositional tasks that demand efficient orchestration of tool-call topologies, implicit intent spread across dialogue turns, and instruction transition.<n>We introduce WildToolBench, an LLM tool-use benchmark grounded in real-world user behaviour patterns.
  arXiv  Detail & Related papers  (2026-02-13T08:55:46Z)
• From Task Solving to Robust Real-World Adaptation in LLM Agents [17.122224644097304]
  Large language models are increasingly deployed as specialized agents that plan, call tools, and take actions over extended horizons.<n>We benchmark agentic LLMs in a grid-based game with a simple goal but long-horizon execution.<n>We find large gaps between nominal task-solving and deployment-like robustness.
  arXiv  Detail & Related papers  (2026-02-02T20:10:40Z)
• AgentIF-OneDay: A Task-level Instruction-Following Benchmark for General AI Agents in Daily Scenarios [49.90735676070039]
  The capacity of AI agents to effectively handle tasks of increasing duration and complexity continues to grow.<n>We argue that current evaluations prioritize increasing task difficulty without sufficiently addressing the diversity of agentic tasks.<n>We propose AgentIF-OneDay, aimed at determining whether general users can utilize natural language instructions and AI agents to complete a diverse array of daily tasks.
  arXiv  Detail & Related papers  (2026-01-28T13:49:18Z)
• LLM Agents Beyond Utility: An Open-Ended Perspective [50.809163251551894]
  We augment a pretrained LLM agent with the ability to generate its own tasks, accumulate knowledge, and interact extensively with its environment.<n>It can reliably follow complex multi-step instructions, store and reuse information across runs, and propose and solve its own tasks.<n>It remains sensitive to prompt design, prone to repetitive task generation, and unable to form self-representations.
  arXiv  Detail & Related papers  (2025-10-16T10:46:54Z)
• Learning-To-Measure: In-context Active Feature Acquisition [10.604433053831405]
  We formalize the meta-AFA problem, where the goal is to learn acquisition policies across various tasks.<n>We introduce Learning-to-Measure (L2M), which consists of i) reliable uncertainty quantification over unseen tasks, and ii) an uncertainty-guided greedy feature acquisition agent.<n>L2M operates directly on datasets with retrospective missingness and performs the meta-AFA task in-context, eliminating per-task retraining.
  arXiv  Detail & Related papers  (2025-10-14T15:23:32Z)
• InfoMosaic-Bench: Evaluating Multi-Source Information Seeking in Tool-Augmented Agents [60.89180545430896]
  InfoMosaic-Bench is the first benchmark dedicated to multi-source information seeking in tool-augmented agents.<n>It requires agents to combine general-purpose search with domain-specific tools.<n>This design guarantees both reliability and non-triviality.
  arXiv  Detail & Related papers  (2025-10-02T17:48:03Z)
• Teaching Language Models To Gather Information Proactively [53.85419549904644]
  Large language models (LLMs) are increasingly expected to function as collaborative partners.<n>In this work, we introduce a new task paradigm: proactive information gathering.<n>We design a scalable framework that generates partially specified, real-world tasks, masking key information.<n>Within this setup, our core innovation is a reinforcement finetuning strategy that rewards questions that elicit genuinely new, implicit user information.
  arXiv  Detail & Related papers  (2025-07-28T23:50:09Z)
• Interactive Agents to Overcome Ambiguity in Software Engineering [61.40183840499932]
  AI agents are increasingly being deployed to automate tasks, often based on ambiguous and underspecified user instructions.<n>Making unwarranted assumptions and failing to ask clarifying questions can lead to suboptimal outcomes.<n>We study the ability of LLM agents to handle ambiguous instructions in interactive code generation settings by evaluating proprietary and open-weight models on their performance.
  arXiv  Detail & Related papers  (2025-02-18T17:12:26Z)
• DetoxBench: Benchmarking Large Language Models for Multitask Fraud & Abuse Detection [15.933013428603152]
  Large language models (LLMs) have demonstrated remarkable capabilities in natural language processing tasks. We present a benchmark suite designed to assess the performance of LLMs in identifying and mitigating fraudulent and abusive language.
  arXiv  Detail & Related papers  (2024-09-09T21:12:03Z)
• MMAU: A Holistic Benchmark of Agent Capabilities Across Diverse Domains [54.117238759317004]
  Massive Multitask Agent Understanding (MMAU) benchmark features comprehensive offline tasks that eliminate the need for complex environment setups. It evaluates models across five domains, including Tool-use, Directed Acyclic Graph (DAG) QA, Data Science and Machine Learning coding, Contest-level programming and Mathematics. With a total of 20 meticulously designed tasks encompassing over 3K distinct prompts, MMAU provides a comprehensive framework for evaluating the strengths and limitations of LLM agents.
  arXiv  Detail & Related papers  (2024-07-18T00:58:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.