Zero-Shot Goal Recognition with Large Language Models

• URL: http://arxiv.org/abs/2605.15333v1
• Date: Thu, 14 May 2026 18:56:06 GMT
• Title: Zero-Shot Goal Recognition with Large Language Models
• Abstract summary: Large language models have recently reached near-parity with classical planners on well-known planning domains.<n>Goal recognition is a complementary abductive task structurally better suited to LLM strengths.<n>This paper provides the first systematic zero-shot evaluation of frontier LLMs as goal recognisers on key classical PDDL benchmarks.
• Score: 6.023276947115864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models have recently reached near-parity with classical planners on well-known planning domains, yet this competence relies on world-knowledge exploitation rather than genuine symbolic reasoning. Goal recognition is a complementary abductive task structurally better suited to LLM strengths: it consists of evaluating consistency with world knowledge rather than generating novel action sequences. This paper provides the first systematic zero-shot evaluation of frontier LLMs as goal recognisers on key classical PDDL benchmarks. Our results show that LLM competence on goal recognition is uneven: some models scale with evidence and approach landmark-based accuracy at full observations, while others remain anchored to world-knowledge priors regardless of how much evidence accumulates. Qualitative analysis of model reasoning traces reveals that this divergence reflects a fundamental difference in evidence integration rather than domain familiarity. These findings position goal recognition as a principled benchmark for the foundational planning knowledge of LLMs.

Related papers

• Listen to the Latents: Self-Correcting Speech Recognition in Large Audio Language Models Through Hidden-State Interactions [120.64235456332916]
  Speech recognition systems increasingly integrate large language models (LLMs) to leverage their semantic knowledge.<n>We refine warm-adapted LLM-based ASR models by leveraging their own pre-adaptation base LLMs.<n>We propose Hybrid Search, a targeted correction strategy motivated by two observations.
  arXiv  Detail & Related papers  (2026-08-31T15:46:48Z)
• Beyond Sally-Anne: Evaluating Theory of Mind in LLMs using Epistemic Schelling Points [0.8703469130213444]
  We introduce the Epistemic Asymmetry Schelling Task (EAST), a two-player dialogue game designed to benchmark robust ToM abilities.<n>We evaluate whether models can robustly apply ToM to achieve coordination.<n>Our results reveal a significant capability gap in functional social reasoning.
  arXiv  Detail & Related papers  (2026-07-13T10:27:22Z)
• LAF-Based Evaluation and UTTL-Based Learning Strategies with MIATTs [2.0885838435144963]
  In many real-world machine learning (ML) applications, the true target cannot be precisely defined due to ambiguity or subjectivity information.<n>Under the assumption that the true target for a given ML task is not assumed to exist objectively in the real world, the EL-MIATTs framework has been proposed.<n>We develop two complementary mechanisms: LAF-based evaluation algorithms and UTTL-based learning strategies with MIATTs.
  arXiv  Detail & Related papers  (2026-04-22T15:47:38Z)
• MM-OPERA: Benchmarking Open-ended Association Reasoning for Large Vision-Language Models [15.929002709503921]
  We aim to evaluate a fundamental yet underexplored intelligence: association.<n> MM-OPERA is a systematic benchmark with 11,497 instances across two open-ended tasks.<n>It challenges LVLMs to resemble the spirit of divergent thinking and convergent associative reasoning.
  arXiv  Detail & Related papers  (2025-10-30T18:49:06Z)
• Truly Assessing Fluid Intelligence of Large Language Models through Dynamic Reasoning Evaluation [106.17986469245302]
  Large language models (LLMs) have demonstrated impressive reasoning capacities that mirror human-like thinking.<n>Existing reasoning benchmarks either focus on domain-specific knowledge (crystallized intelligence) or lack interpretability.<n>We propose DRE-Bench, a dynamic reasoning evaluation benchmark grounded in a hierarchical cognitive framework.
  arXiv  Detail & Related papers  (2025-06-03T09:01:08Z)
• Latent Factor Models Meets Instructions: Goal-conditioned Latent Factor Discovery without Task Supervision [50.45597801390757]
  Instruct-LF is a goal-oriented latent factor discovery system.<n>It integrates instruction-following ability with statistical models to handle noisy datasets.
  arXiv  Detail & Related papers  (2025-02-21T02:03:08Z)
• Evaluating Human Alignment and Model Faithfulness of LLM Rationale [66.75309523854476]
  We study how well large language models (LLMs) explain their generations through rationales. We show that prompting-based methods are less "faithful" than attribution-based explanations.
  arXiv  Detail & Related papers  (2024-06-28T20:06:30Z)
• Fusing Domain-Specific Content from Large Language Models into Knowledge Graphs for Enhanced Zero Shot Object State Classification [1.1161827123148225]
  This study investigates the potential of Large Language Models (LLMs) in generating and providing domain-specific information.<n>To achieve this, an LLM is integrated into a pipeline that utilizes Knowledge Graphs and pre-trained semantic vectors.<n>Our findings reveal that the integration of LLM-based embeddings, in combination with general-purpose pre-trained embeddings, leads to substantial performance improvements.
  arXiv  Detail & Related papers  (2024-03-18T18:08:44Z)
• MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation [60.65820977963331]
  We introduce a novel evaluation paradigm for Large Language Models (LLMs) This paradigm shifts the emphasis from result-oriented assessments, which often neglect the reasoning process, to a more comprehensive evaluation. By applying this paradigm in the GSM8K dataset, we have developed the MR-GSM8K benchmark.
  arXiv  Detail & Related papers  (2023-12-28T15:49:43Z)
• EpiK-Eval: Evaluation for Language Models as Epistemic Models [16.485951373967502]
  We introduce EpiK-Eval, a novel question-answering benchmark tailored to evaluate LLMs' proficiency in formulating a coherent and consistent knowledge representation from segmented narratives. We argue that these shortcomings stem from the intrinsic nature of prevailing training objectives. The findings from this study offer insights for developing more robust and reliable LLMs.
  arXiv  Detail & Related papers  (2023-10-23T21:15:54Z)
• Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models [61.28463542324576]
  Vision-language models (VLMs) have recently demonstrated strong efficacy as visual assistants that can generate human-like outputs. We evaluate existing state-of-the-art VLMs and find that even the best-performing model is unable to demonstrate strong visual reasoning capabilities and consistency. We propose a two-stage training framework aimed at improving both the reasoning performance and consistency of VLMs.
  arXiv  Detail & Related papers  (2023-09-08T17:49:44Z)
• Improving Open Information Extraction with Large Language Models: A Study on Demonstration Uncertainty [52.72790059506241]
  Open Information Extraction (OIE) task aims at extracting structured facts from unstructured text. Despite the potential of large language models (LLMs) like ChatGPT as a general task solver, they lag behind state-of-the-art (supervised) methods in OIE tasks.
  arXiv  Detail & Related papers  (2023-09-07T01:35:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.