Asking Clarifying Questions for Preference Elicitation With Large Language Models

• URL: http://arxiv.org/abs/2510.12015v1
• Date: Mon, 13 Oct 2025 23:32:31 GMT
• Title: Asking Clarifying Questions for Preference Elicitation With Large Language Models
• Authors: Ali Montazeralghaem, Guy Tennenholtz, Craig Boutilier, Ofer Meshi,
• Abstract summary: Large Language Models (LLMs) have made it possible for recommendation systems to interact with users in open-ended conversational interfaces.<n>We introduce a novel approach for training LLMs to ask sequential questions that reveal user preferences.
• Score: 19.809978521730855
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) have made it possible for recommendation systems to interact with users in open-ended conversational interfaces. In order to personalize LLM responses, it is crucial to elicit user preferences, especially when there is limited user history. One way to get more information is to present clarifying questions to the user. However, generating effective sequential clarifying questions across various domains remains a challenge. To address this, we introduce a novel approach for training LLMs to ask sequential questions that reveal user preferences. Our method follows a two-stage process inspired by diffusion models. Starting from a user profile, the forward process generates clarifying questions to obtain answers and then removes those answers step by step, serving as a way to add ``noise'' to the user profile. The reverse process involves training a model to ``denoise'' the user profile by learning to ask effective clarifying questions. Our results show that our method significantly improves the LLM's proficiency in asking funnel questions and eliciting user preferences effectively.

Related papers

• Towards Realistic Personalization: Evaluating Long-Horizon Preference Following in Personalized User-LLM Interactions [50.70965714314064]
  Large Language Models (LLMs) are increasingly serving as personal assistants, where users share complex and diverse preferences over extended interactions.<n>This work proposes RealPref, a benchmark for evaluating realistic preference-following in personalized user-LLM interactions.
  arXiv  Detail & Related papers  (2026-03-04T15:42:43Z)
• User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal [59.120335322495436]
  We analyze user feedback in the user-LLM conversation logs, providing insights into when and why such feedback occurs.<n>Second, we study harvesting learning signals from such implicit user feedback.
  arXiv  Detail & Related papers  (2025-07-30T23:33:29Z)
• Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale [51.9706400130481]
  Large Language Models (LLMs) have emerged as personalized assistants for users across a wide range of tasks.<n> PERSONAMEM features curated user profiles with over 180 simulated user-LLM interaction histories.<n>We evaluate LLM chatbots' ability to identify the most suitable response according to the current state of the user's profile.
  arXiv  Detail & Related papers  (2025-04-19T08:16:10Z)
• Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions [45.04582353648683]
  Large language models (LLMs) must often respond to highly ambiguous user requests.<n>Existing LLMs often respond by presupposing a single interpretation of such ambiguous requests, frustrating users who intended a different interpretation.<n>We propose preference labels by simulating their expected outcomes in future turns.<n>This allows LLMs to learn to ask clarifying questions when it can generate responses that are tailored to each user interpretation in future turns.
  arXiv  Detail & Related papers  (2024-10-17T17:29:04Z)
• Putting People in LLMs' Shoes: Generating Better Answers via Question Rewriter [17.736962215696366]
  We introduce single-round instance-level prompt optimization, referred to as question rewriter.<n>By enhancing the intelligibility of human questions for black-box LLMs, our question rewriter improves the quality of generated answers.<n>Experiments across multiple black-box LLMs and long-form question answering datasets demonstrate the efficacy of our method.
  arXiv  Detail & Related papers  (2024-08-20T06:24:47Z)
• Prompt Tuning as User Inherent Profile Inference Machine [68.16976932088708]
  We propose UserIP-Tuning, which uses prompt-tuning to infer user profiles.<n>UserIP-Tuning outperforms state-of-the-art recommendation algorithms.<n>The presented solution has been deployed in Huawei AppGallery's Explore page since May 2025.
  arXiv  Detail & Related papers  (2024-08-13T02:25:46Z)
• CLARINET: Augmenting Language Models to Ask Clarification Questions for Retrieval [52.134133938779776]
  We present CLARINET, a system that asks informative clarification questions by choosing questions whose answers would maximize certainty in the correct candidate. Our approach works by augmenting a large language model (LLM) to condition on a retrieval distribution, finetuning end-to-end to generate the question that would have maximized the rank of the true candidate at each turn.
  arXiv  Detail & Related papers  (2024-04-28T18:21:31Z)
• Relative Preference Optimization: Enhancing LLM Alignment through Contrasting Responses across Identical and Diverse Prompts [95.09994361995389]
  Relative Preference Optimization (RPO) is designed to discern between more and less preferred responses derived from both identical and related prompts. RPO has demonstrated a superior ability to align large language models with user preferences and to improve their adaptability during the training process.
  arXiv  Detail & Related papers  (2024-02-12T22:47:57Z)
• Active Preference Inference using Language Models and Probabilistic Reasoning [13.523369679010685]
  We introduce an inference-time algorithm that helps large language models infer user preferences. Our algorithm uses a probabilistic model whose conditional distributions are defined by prompting an LLM. Results in a simplified interactive web shopping setting with real product items show that an LLM equipped with our entropy reduction algorithm outperforms baselines.
  arXiv  Detail & Related papers  (2023-12-19T09:58:54Z)
• A New Dialogue Response Generation Agent for Large Language Models by Asking Questions to Detect User's Intentions [28.389176266764775]
  Large Language Models (LLMs) have been applied to various NLP tasks due to its open-domain generation capabilities. We propose a frameworkemphusing LLM to textbfEnhance dialogue response generation by asking questions to textbfDetect user's textbfImplicit intextbfTentions (textbfEDIT) Firstly, EDIT generates open questions related to the dialogue context as the potential user's intention; Then, EDIT answers those questions by interacting with LLMs and searching
  arXiv  Detail & Related papers  (2023-10-05T03:45:54Z)
• Generating Usage-related Questions for Preference Elicitation in Conversational Recommender Systems [19.950705852361565]
  We propose a novel approach to preference elicitation by asking implicit questions based on item usage.<n>We develop a high-quality labeled training dataset using crowdsourcing.<n>We show that our approaches are effective in generating elicitation questions, even with limited training data.
  arXiv  Detail & Related papers  (2021-11-26T12:23:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.