Zero-Shot and Efficient Clarification Need Prediction in Conversational Search

• URL: http://arxiv.org/abs/2503.00179v1
• Date: Fri, 28 Feb 2025 20:49:18 GMT
• Title: Zero-Shot and Efficient Clarification Need Prediction in Conversational Search
• Abstract summary: We propose a zero-shot and efficient CNP framework (Zef-CNP)<n>We first prompt large language models (LLMs) in a zero-shot manner to generate two sets of synthetic queries.<n>We then use the generated queries to train efficient CNP models.
• Score: 7.791642719869652
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Clarification need prediction (CNP) is a key task in conversational search, aiming to predict whether to ask a clarifying question or give an answer to the current user query. However, current research on CNP suffers from the issues of limited CNP training data and low efficiency. In this paper, we propose a zero-shot and efficient CNP framework (Zef-CNP), in which we first prompt large language models (LLMs) in a zero-shot manner to generate two sets of synthetic queries: ambiguous and specific (unambiguous) queries. We then use the generated queries to train efficient CNP models. Zef-CNP eliminates the need for human-annotated clarification-need labels during training and avoids the use of LLMs with high query latency at query time. To further improve the generation quality of synthetic queries, we devise a topic-, information-need-, and query-aware chain-of-thought (CoT) prompting strategy (TIQ-CoT). Moreover, we enhance TIQ-CoT with counterfactual query generation (CoQu), which guides LLMs first to generate a specific/ambiguous query and then sequentially generate its corresponding ambiguous/specific query. Experimental results show that Zef-CNP achieves superior CNP effectiveness and efficiency compared with zero- and few-shot LLM-based CNP predictors.

Related papers

• Efficient Sequential Evaluation of Large Language Models [10.520659921457629]
  We study the problem of sequentially evaluating a new large language model (LLM) on a fixed question set using historical performance data.<n>Our goal is to construct a confidence sequence (CS) for the model's capability on this question set and to design active querying rules that shrink the CS width as quickly as possible.
  arXiv  Detail & Related papers  (2026-07-19T21:01:34Z)
• ClarifyCodeBench: Evaluating LLMs on Clarifying Ambiguous Requirements for Code Generation [54.788849448970154]
  We introduce ClarifyCodeBench, a novel interactive benchmark for evaluating Large Language Models' capability in resolving requirement ambiguity.<n>We conduct a systematic evaluation of six state-of-the-art LLMs using ClarifyCodeBench.<n>Our empirical results yield three critical insights: 1) Capability Decoupling: Strong code generation performance does not inherently translate to effective requirement clarification; 2) The Reasoning Paradox: While increased computational thinking enhances code correctness, it yields marginal gains in identifying ambiguities.
  arXiv  Detail & Related papers  (2026-07-01T09:58:59Z)
• Adaptive Blockwise Search: Inference-Time Alignment for Large Language Models [13.368340836611075]
  In-time methods provide a flexible alternative to fine-tuning, but their uniform computational effort often yields suboptimal alignment.<n>We introduce AdaSearch, a novel blockwise search strategy.<n>It adaptively allocates a fixed computational budget using a sampling schedule, focusing search effort on critical tokens.
  arXiv  Detail & Related papers  (2025-10-27T13:48:59Z)
• Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models? [65.18157595903124]
  This work investigates iterative approximate evaluation for arbitrary prompts.<n>It introduces Model Predictive Prompt Selection (MoPPS), a Bayesian risk-predictive framework.<n>MoPPS reliably predicts prompt difficulty and accelerates training with significantly reduced rollouts.
  arXiv  Detail & Related papers  (2025-07-07T03:20:52Z)
• Conformal Information Pursuit for Interactively Guiding Large Language Models [64.39770942422288]
  This paper explores sequential querying strategies that aim to minimize the expected number of queries.<n>One such strategy is Information Pursuit (IP), a greedy algorithm that at each iteration selects the query that maximizes information gain or equivalently minimizes uncertainty.<n>We propose Conformal Information Pursuit (C-IP), an alternative approach to sequential information gain based on conformal prediction sets.
  arXiv  Detail & Related papers  (2025-07-04T03:55:39Z)
• AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learning [30.265984245328124]
  Chain-of-Thought prompting indiscriminately generates lengthy reasoning steps for all queries.<n>AdaCoT (Adaptive Chain-of-Thought) is a novel framework enabling LLMs to adaptively decide when to invoke CoT.<n>A key technical contribution is Selective Loss Masking (SLM), designed to counteract decision boundary collapse.
  arXiv  Detail & Related papers  (2025-05-17T08:27:00Z)
• Exploring Pseudo-Token Approaches in Transformer Neural Processes [0.0]
  We introduce the Induced Set Attentive Neural Processes (ISANPs) ISANPs perform competitively with Transformer Neural Processes (TNPs) and often surpass state-of-the-art models in 1D regression, image completion, contextual bandits, and Bayesian optimization. ISANPs offer a tunable balance between performance and computational complexity, which scale well to larger datasets.
  arXiv  Detail & Related papers  (2025-04-19T22:47:59Z)
• Order-agnostic Identifier for Large Language Model-based Generative Recommendation [94.37662915542603]
  Items are assigned identifiers for Large Language Models (LLMs) to encode user history and generate the next item.<n>Existing approaches leverage either token-sequence identifiers, representing items as discrete token sequences, or single-token identifiers, using ID or semantic embeddings.<n>We propose SETRec, which leverages semantic tokenizers to obtain order-agnostic multi-dimensional tokens.
  arXiv  Detail & Related papers  (2025-02-15T15:25:38Z)
• Neural-Symbolic Message Passing with Dynamic Pruning [22.485655410582375]
  We propose a Neural-Symbolic Message Passing (NSMP) framework based on pre-trained neural link predictors.<n>By introducing symbolic reasoning and fuzzy logic, NSMP can generalize to arbitrary existential first order logic queries without requiring training.<n>Compared to this approach, NSMP demonstrates faster inference times across all query types on benchmark datasets.
  arXiv  Detail & Related papers  (2025-01-24T17:30:17Z)
• Prompting Strategies for Enabling Large Language Models to Infer Causation from Correlation [68.58373854950294]
  We focus on causal reasoning and address the task of establishing causal relationships based on correlation information.<n>We introduce a prompting strategy for this problem that breaks the original task into fixed subquestions.<n>We evaluate our approach on an existing causal benchmark, Corr2Cause.
  arXiv  Detail & Related papers  (2024-12-18T15:32:27Z)
• In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement [71.60563181678323]
  Large language models (LLMs) have achieved great success across diverse tasks, and fine-tuning is sometimes needed to further enhance generation quality.<n>To handle these challenges, a direct solution is to generate high-confidence'' data from unsupervised downstream tasks.<n>We propose a novel approach, pseudo-supervised demonstrations aligned prompt optimization (PAPO) algorithm, which jointly refines both the prompt and the overall pseudo-supervision.
  arXiv  Detail & Related papers  (2024-10-04T03:39:28Z)
• Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generation [19.327008532572645]
  Question Generation over Knowledge Bases (KBQG) aims to convert a logical form into a natural language question. We propose Chain-of-Thought prompting, which is an in-context learning strategy for reasoning. We conduct extensive experiments over three public KBQG datasets.
  arXiv  Detail & Related papers  (2023-10-12T15:08:14Z)
• Rethinking Model Selection and Decoding for Keyphrase Generation with Pre-trained Sequence-to-Sequence Models [76.52997424694767]
  Keyphrase Generation (KPG) is a longstanding task in NLP with widespread applications. Seq2seq pre-trained language models (PLMs) have ushered in a transformative era for KPG, yielding promising performance improvements. This paper undertakes a systematic analysis of the influence of model selection and decoding strategies on PLM-based KPG.
  arXiv  Detail & Related papers  (2023-10-10T07:34:45Z)
• MinPrompt: Graph-based Minimal Prompt Data Augmentation for Few-shot Question Answering [64.6741991162092]
  We present MinPrompt, a minimal data augmentation framework for open-domain question answering. We transform the raw text into a graph structure to build connections between different factual sentences. We then apply graph algorithms to identify the minimal set of sentences needed to cover the most information in the raw text. We generate QA pairs based on the identified sentence subset and train the model on the selected sentences to obtain the final model.
  arXiv  Detail & Related papers  (2023-10-08T04:44:36Z)
• Python Code Generation by Asking Clarification Questions [57.63906360576212]
  In this work, we introduce a novel and more realistic setup for this task. We hypothesize that the under-specification of a natural language description can be resolved by asking clarification questions. We collect and introduce a new dataset named CodeClarQA containing pairs of natural language descriptions and code with created synthetic clarification questions and answers.
  arXiv  Detail & Related papers  (2022-12-19T22:08:36Z)
• Latent Bottlenecked Attentive Neural Processes [71.18817592128207]
  We present Latent Bottlenecked Attentive Neural Processes (LBANPs) LBANPs have a querying computational complexity independent of the number of context datapoints. We show LBANPs achieve results competitive with the state-of-the-art on meta-regression, image completion, and contextual multi-armed bandits.
  arXiv  Detail & Related papers  (2022-11-15T19:21:41Z)
• AdaPrompt: Adaptive Model Training for Prompt-based NLP [77.12071707955889]
  We propose AdaPrompt, adaptively retrieving external data for continual pretraining of PLMs. Experimental results on five NLP benchmarks show that AdaPrompt can improve over standard PLMs in few-shot settings. In zero-shot settings, our method outperforms standard prompt-based methods by up to 26.35% relative error reduction.
  arXiv  Detail & Related papers  (2022-02-10T04:04:57Z)
• Message Passing Neural Processes [3.0969191504482247]
  We introduce Message Passing Neural Processes (MPNPs), which explicitly makes use of relational structure within the model. MPNPs thrive at lower sampling rates, on existing benchmarks and newly-proposed CA and Cora-Branched tasks. We report strong generalisation over density-based CA rulesets and significant gains in challenging arbitrary-labelling and few-shot learning setups.
  arXiv  Detail & Related papers  (2020-09-29T09:40:09Z)
• Efficient Neural Query Auto Completion [17.58784759652327]
  Three major challenges are observed for a query auto completion system. Traditional QAC systems rely on handcrafted features such as the query candidate frequency in search logs. We propose an efficient neural QAC system with effective context modeling to overcome these challenges.
  arXiv  Detail & Related papers  (2020-08-06T21:28:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.