PACE: Towards Surfacing Hidden Conflicts in User Requests

• URL: http://arxiv.org/abs/2609.03293v1
• Date: Thu, 03 Sep 2026 02:40:25 GMT
• Title: PACE: Towards Surfacing Hidden Conflicts in User Requests
• Abstract summary: We introduce PACE, a dataset for evaluating whether models can identify latent constraints that render seemingly reasonable user requests inappropriate.<n>PACE pairs user requests grounded in well-defined personas with egocentric KB facts, requiring models to integrate contextual evidence to determine whether a request is conflicting.<n>Experiments on PACE evaluate both evidence retrieval quality and conflict decision accuracy, showing that PaceMaker consistently outperforms existing approaches.
• Score: 5.6582531294036755
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Personalized assistants should not only comply with user requests but also assess whether those requests are appropriate given the user's current circumstances. However, prior work has primarily focused on accurately executing requests, overlooking the need for assistants to account for context and engage in conflict-based refusal. Furthermore, while existing work on conflict or safety detection relies on explicitly provided factors, real-world scenarios often involve implicit factors that must be retrieved from a knowledge base (KB). To this end, we introduce Personalized Assistants for Conflict Evaluation (PACE), a dataset for evaluating whether models can identify latent constraints, expressed as egocentric knowledge or events, that render seemingly reasonable user requests inappropriate. PACE pairs user requests grounded in well-defined personas with egocentric KB facts, requiring models to integrate contextual evidence to determine whether a request is conflicting. This implicit retrieval setting hinders the direct association between user requests and conflict-inducing knowledge, making it difficult for existing models to identify relevant user-specific facts. To address this challenge, we further propose PaceMaker, a multi-agent framework in which specialized agents coordinate across query reformulation, multi-hop graph traversal, and conflict-aware filtering to retrieve contextually decisive evidence. Experiments on PACE evaluate both evidence retrieval quality and conflict decision accuracy, showing that PaceMaker consistently outperforms existing approaches.

Related papers

• CLAIM: Leading Open-domain Active Clarification of Large Language Models with Uncertainty Measurement [50.91045324738942]
  CLAIM is an uncertainty-driven framework for active clarification learning in open-domain settings.<n>We propose an entropy-driven synthetic data generation pipeline that integrates uncertainty estimation with semantic clustering and reasoning-based judgments.<n> Experimental results demonstrate that CLAIM can learn stable and generalizable clarification strategies without relying on manually labeled data.
  arXiv  Detail & Related papers  (2026-08-12T04:27:45Z)
• Personalized Deep Research Query Refinement with Graph-Scaffolded Evidence Grounding [9.25068659105141]
  G-STEER learns a clarification policy from graph-scaffolded trajectories spanning diverse factor dependencies and evidence conditions.<n>Experiments show that G-STEER achieves the strongest overall weighted target coverage and the highest downstream report personalization across both evaluated DRAs.
  arXiv  Detail & Related papers  (2026-08-06T10:58:17Z)
• SEER: A Self-Grounded Evidence Interface for Controlled Spatial Relation Classification [71.9783246097687]
  SEER hides candidate relations during pair localization, constructs a query-specific view with explicit subject/object roles, and retains the full image and sparse box geometry as complementary evidence.<n>For relation-choice protocols with exact inverse support, an optional refinement swaps the entity roles and changes the forward decision only when exactly one visual state obeys the corresponding inverse relation.<n>The unchanged protocol yields +4.35 to +11.79 on all 2,434 filtered EmbSpatial pair-relation questions across three models.
  arXiv  Detail & Related papers  (2026-08-04T13:16:15Z)
• From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents [56.187590188698955]
  Large Language Models have enabled increasingly capable autonomous agents, yet personalization remains critical for making such agents practically useful.<n>Recent benchmarks have begun evaluating personalization in agents, but they largely rely on static preference snapshots, fixed interaction logs, or question answering over predefined user profiles.<n>We introduce IBA-Bench, a benchmark for implicit behavioral alignment constructed from longitudinal interaction histories.<n>We propose IBA-Agent, an agent framework that reconciles conflicting priorities through broad retrieval and trajectory-level alignment.
  arXiv  Detail & Related papers  (2026-08-03T12:52:04Z)
• MMAgent-R$^2$: Learning to Rerank and Reject for Agentic mRAG [50.46218163777298]
  Knowledge-based Visual Question Answering (KB-VQA) requires models to retrieve visual entities matching the query image from large-scale encyclopedic knowledge bases and answer related questions.<n>We propose MMAgent-R$2$, an agentic mRAG framework that integrates visual reranking and active rejection as its internal verification mechanism.
  arXiv  Detail & Related papers  (2026-07-08T13:15:39Z)
• When Search Agents Should Ask: DiscoBench for Clarification-Aware Deep Search [6.384441153713447]
  We introduce DiscoBench, a benchmark for clarification-aware deep search.<n>It is designed to evaluate whether search agents can proactively identify ambiguity, ask effective clarification questions, and recover correct reasoning paths through user interaction.<n>Experiments show that ambiguity detection and effective clarification are distinct capabilities, and that repeatedly searching instead of asking for clarification often performs worse than direct guessing.
  arXiv  Detail & Related papers  (2026-06-26T02:57:15Z)
• REAL: Resolving Knowledge Conflicts in Knowledge-Intensive Visual Question Answering via Reasoning-Pivot Alignment [21.368211618743256]
  Knowledge-intensive Visual Question Answering (KI-VQA) frequently suffers from severe knowledge conflicts caused by the inherent limitations of open-domain retrieval.<n>We propose the REAL (Reasoning-Pivot Alignment) framework centered on the novel concept of the Reasoning-Pivot.<n>Our approach integrates Reasoning-Pivot Aware SFT (RPA-SFT) to train a generalizable discriminator by aligning conflicts with pivot extraction, and employs Reasoning-Pivot Guided Decoding (RPGD) for targeted conflict mitigation.
  arXiv  Detail & Related papers  (2026-02-15T09:29:53Z)
• Are We Asking the Right Questions? On Ambiguity in Natural Language Queries for Tabular Data Analysis [2.905751301655124]
  We develop a principled framework based on a shared responsibility of query specification between user and system.<n>Applying the framework to evaluations for question answering and analysis, we analyze the queries in 15 popular datasets.
  arXiv  Detail & Related papers  (2025-11-06T17:39:18Z)
• Towards Context-aware Reasoning-enhanced Generative Searching in E-commerce [61.03081096959132]
  We propose a context-aware reasoning-enhanced generative search framework for better textbfunderstanding the complicated context.<n>Our approach achieves superior performance compared with strong baselines, validating its effectiveness for search-based recommendation.
  arXiv  Detail & Related papers  (2025-10-19T16:46:11Z)
• Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Delivery [17.23286832909591]
  We propose a set of ten challenges in transmission of knowledge derived from the philosophical literature.<n>We find users develop workarounds to address each of the challenges.<n>For AI developers, the Epistemic Alignment Framework offers concrete guidance for supporting diverse approaches to knowledge.
  arXiv  Detail & Related papers  (2025-04-01T21:38:12Z)
• Tell Me What You Don't Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editing [53.44380921234324]
  We develop an evaluation benchmark that includes contextual knowledge conflicting requests, parametric knowledge conflicting requests, and non-conflicting requests.<n>We find that most RPAs behave significant performance gaps toward different conflict requests.<n>We introduce a lightweight representation editing approach that conveniently shifts conflicting requests to the rejection region.
  arXiv  Detail & Related papers  (2024-09-25T13:18:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.