Asking Questions the Right Way: A Multi-Agent Conversational System for Prompt Formulation in Complex Task Resolution

• URL: http://arxiv.org/abs/2608.01366v2
• Date: Tue, 04 Aug 2026 05:28:54 GMT
• Title: Asking Questions the Right Way: A Multi-Agent Conversational System for Prompt Formulation in Complex Task Resolution
• Abstract summary: Large language models (LLMs) are integral to complex intellectual tasks, yet output quality remains constrained by user-provided prompts.<n>We present PAWNI, an agentic conversational interface of eight agents that transforms unstructured queries into structured prompts.<n>We also propose a three-tier framework of 18 prompt elements across Essential, Enhancement, and Elevation categories.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are integral to complex intellectual tasks, yet output quality remains constrained by user-provided prompts. Iterative multi-turn prompting often leads to context degradation and diminishing cognitive returns. We present PAWNI (Prompt Architecture Wizard using Neural Intelligence), an agentic conversational interface of eight agents that transforms unstructured queries into structured prompts through guided question-and-answer dialogue informed by a self-evolving knowledge base. Rather than optimising the model's response, PAWNI optimises the question itself by front-loading intent clarification. We also propose a three-tier framework of 18 prompt elements across Essential, Enhancement, and Elevation categories. To evaluate system behaviour and validate a measurement protocol, we conducted an exploratory within-subjects study (N=4) across four complex tasks, integrating 32-channel EEG, NASA-TLX workload, and behavioural metrics. Participants produced more structurally complete prompts with PAWNI (42% to 91% of assessed elements), rated LLM outputs higher across all quality dimensions, and reported lower workload (39.6 vs. 21.7 NASA-TLX). Every participant reached satisfactory output in a single turn, compared to 1-12 turns unaided. While effect sizes are unstable due to sample size, direction consistency supports the hypothesis that optimising prompt formulation front-end is a critical lever for human-AI collaboration.

Related papers

• Improving Answer Extraction in Context-based Question Answering Systems Using LLMs [0.25489046505746704]
  We propose a question answering system based on large language models, where the input consists of a textual context and a corresponding question, and the output is a concise and accurate answer.<n>Our methodology involves fine-tuning a pre-trained LLM on a benchmark QA dataset to improve its contextual comprehension and answer extraction capabilities.<n> Experimental results show that the fine-tuned Roberta-base model achieves the highest performance, attaining a ROUGE-L score of 86.84%, a BLEU score of 28.24%, and a BERTScore of 95.38%.
  arXiv  Detail & Related papers  (2026-06-04T14:04:11Z)
• AMATA: Adaptive Multi-Agent Trajectory Alignment for Knowledge-Intensive Question Answering [59.145044878490815]
  AMATA is an Adaptive Multi-Agent Trajectory Alignment framework.<n>It dynamically integrates external knowledge to improve response interpretability and factual grounding.<n>We introduce two principal innovations: (1) Intra-Trajectory Preference Learning, which learns objective-oriented preferences to prioritize critical agents, and (2) Inter-Agent Dependency Learning, which captures cross-agent tool dependencies.
  arXiv  Detail & Related papers  (2026-05-17T09:45:24Z)
• GISA: A Benchmark for General Information-Seeking Assistant [102.30831921333755]
  GISA is a benchmark for General Information-Seeking Assistants comprising 373 human-crafted queries.<n>It integrates both deep reasoning and broad information aggregation within unified tasks, and includes a live subset with periodically updated answers to resist memorization.<n>Experiments on mainstream LLMs and commercial search products reveal that even the best-performing model achieves only 19.30% exact match score.
  arXiv  Detail & Related papers  (2026-02-09T11:44:15Z)
• Prism: Towards Lowering User Cognitive Load in LLMs via Complex Intent Understanding [8.863937298785347]
  Large Language Models are rapidly emerging as web-native interfaces to social platforms.<n>Existing approaches attempt to clarify user intents through sequential or parallel questioning.<n>Inspired by the Cognitive Load Theory, we propose Prism, a novel framework for complex intent understanding.
  arXiv  Detail & Related papers  (2026-01-13T15:30:48Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• First Ask Then Answer: A Framework Design for AI Dialogue Based on Supplementary Questioning with Large Language Models [0.0]
  First Ask Then Answer (FATA) generates multidimensional supplementary questions for users prior to response generation.<n>In contrast to existing clarification approaches, FATA emphasizes completeness and user participation.<n> Experimental results show that FATA outperforms B-Prompt by approximately 40% in aggregate metrics.
  arXiv  Detail & Related papers  (2025-08-08T13:39:47Z)
• Teaching Language Models To Gather Information Proactively [53.85419549904644]
  Large language models (LLMs) are increasingly expected to function as collaborative partners.<n>In this work, we introduce a new task paradigm: proactive information gathering.<n>We design a scalable framework that generates partially specified, real-world tasks, masking key information.<n>Within this setup, our core innovation is a reinforcement finetuning strategy that rewards questions that elicit genuinely new, implicit user information.
  arXiv  Detail & Related papers  (2025-07-28T23:50:09Z)
• The Future of MLLM Prompting is Adaptive: A Comprehensive Experimental Evaluation of Prompt Engineering Methods for Robust Multimodal Performance [0.393259574660092]
  Multimodal Large Language Models (MLLMs) are set to transform how machines process and generate human-like responses.<n>We present a comprehensive experimental evaluation of seven prompt engineering methods applied to 13 open-source MLLMs over 24 tasks.
  arXiv  Detail & Related papers  (2025-04-14T12:31:39Z)
• Self-Taught Agentic Long Context Understanding [47.186303525057475]
  AgenticLU is a framework designed to integrate targeted self-clarification with contextual grounding.<n>AgenticLU achieves 97.8% answer recall on NarrativeQA with a search depth of up to three and a branching factor of eight.
  arXiv  Detail & Related papers  (2025-02-21T20:29:36Z)
• Self-Convinced Prompting: Few-Shot Question Answering with Repeated Introspection [13.608076739368949]
  We introduce a novel framework that harnesses the potential of large-scale pre-trained language models. Our framework processes the output of a typical few-shot chain-of-thought prompt, assesses the correctness of the response, scrutinizes the answer, and ultimately produces a new solution.
  arXiv  Detail & Related papers  (2023-10-08T06:36:26Z)
• PICK: Polished & Informed Candidate Scoring for Knowledge-Grounded Dialogue Systems [59.1250765143521]
  Current knowledge-grounded dialogue systems often fail to align the generated responses with human-preferred qualities. We propose Polished & Informed Candidate Scoring (PICK), a generation re-scoring framework. We demonstrate the effectiveness of PICK in generating responses that are more faithful while keeping them relevant to the dialogue history.
  arXiv  Detail & Related papers  (2023-09-19T08:27:09Z)
• Using Textual Interface to Align External Knowledge for End-to-End Task-Oriented Dialogue Systems [53.38517204698343]
  We propose a novel paradigm that uses a textual interface to align external knowledge and eliminate redundant processes. We demonstrate our paradigm in practice through MultiWOZ-Remake, including an interactive textual interface built for the MultiWOZ database.
  arXiv  Detail & Related papers  (2023-05-23T05:48:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.