DiscoTrace: Representing and Comparing Answering Strategies of Humans and LLMs in Information-Seeking Question Answering

• URL: http://arxiv.org/abs/2604.15140v1
• Date: Thu, 16 Apr 2026 15:17:36 GMT
• Title: DiscoTrace: Representing and Comparing Answering Strategies of Humans and LLMs in Information-Seeking Question Answering
• Abstract summary: We introduce DiscoTrace, a method to identify the rhetorical strategies that answerers use when responding to information-seeking questions.<n>Applying DiscoTrace to answers from nine different human communities reveals that communities have diverse preferences for answer construction.
• Score: 28.365849533156023
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We introduce DiscoTrace, a method to identify the rhetorical strategies that answerers use when responding to information-seeking questions. DiscoTrace represents answers as a sequence of question-related discourse acts paired with interpretations of the original question, annotated on top of rhetorical structure theory parses. Applying DiscoTrace to answers from nine different human communities reveals that communities have diverse preferences for answer construction. In contrast, LLMs do not exhibit rhetorical diversity in their answers, even when prompted to mimic specific human community answering guidelines. LLMs also systematically opt for breadth, addressing interpretations of questions that human answerers choose not to address. Our findings can guide the development of pragmatic LLM answerers that consider a range of strategies informed by context in QA.

Related papers

• Not What, But How: A Framework for Auditing LLM Responses across Positioning, Generalization, Anthromorphism, and Maxims [54.31707479192195]
  Large language models (LLMs) are being increasingly used to answer subjective, information-seeking questions.<n>Existing evaluations for subjective cultural queries largely focus on factual correctness, ignoring how the response is framed.<n>We introduce FRANZ, an automated FRAmework for respoNse characteriZation to conduct communicative audit of LLM responses along four dimensions.
  arXiv  Detail & Related papers  (2026-06-01T17:01:31Z)
• Multimodal QUD: Inquisitive Questions from Scientific Figures [63.41049609329304]
  We generate inquisitive questions that reach the depth of questions humans generate when engaging with scientific papers.<n>We extend the linguistic theory of Questions Under Discussion (QUD) from being text-only to multimodal.<n>We show that fine-tuning a VLM on MQUD shifts the model from generating generic low-level visual questions to content-specific grounding.
  arXiv  Detail & Related papers  (2026-04-26T14:25:23Z)
• Rhetorical Questions in LLM Representations: A Linear Probing Study [5.165491129638819]
  We analyze rhetorical questions in LLM representations using linear probes on two social-media datasets with different discourse contexts.<n>We find that rhetorical signals emerge early and are most stably captured by last-token representations.
  arXiv  Detail & Related papers  (2026-04-15T17:50:56Z)
• Can LLMs Ask Good Questions? [45.54763954234726]
  We evaluate questions generated by large language models (LLMs) from context.<n>We compare them to human-authored questions across six dimensions: question type, question length, context coverage, answerability, uncommonness, and required answer length.
  arXiv  Detail & Related papers  (2025-01-07T03:21:17Z)
• Enhancing Answer Attribution for Faithful Text Generation with Large Language Models [5.065947993017158]
  We propose new methods for producing more independent and contextualized claims for better retrieval and attribution. New methods are evaluated and shown to improve the performance of answer attribution components.
  arXiv  Detail & Related papers  (2024-10-22T15:37:46Z)
• ELOQ: Resources for Enhancing LLM Detection of Out-of-Scope Questions [52.33835101586687]
  We study out-of-scope questions, where the retrieved document appears semantically similar to the question but lacks the necessary information to answer it.<n>We propose a guided hallucination-based approach ELOQ to automatically generate a diverse set of out-of-scope questions from post-cutoff documents.
  arXiv  Detail & Related papers  (2024-10-18T16:11:29Z)
• Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions [45.04582353648683]
  Large language models (LLMs) must often respond to highly ambiguous user requests.<n>Existing LLMs often respond by presupposing a single interpretation of such ambiguous requests, frustrating users who intended a different interpretation.<n>We propose preference labels by simulating their expected outcomes in future turns.<n>This allows LLMs to learn to ask clarifying questions when it can generate responses that are tailored to each user interpretation in future turns.
  arXiv  Detail & Related papers  (2024-10-17T17:29:04Z)
• Open Domain Question Answering with Conflicting Contexts [55.739842087655774]
  We find that as much as 25% of unambiguous, open domain questions can lead to conflicting contexts when retrieved using Google Search.<n>We ask our annotators to provide explanations for their selections of correct answers.
  arXiv  Detail & Related papers  (2024-10-16T07:24:28Z)
• GenSco: Can Question Decomposition based Passage Alignment improve Question Answering? [1.5776201492893507]
  "GenSco" is a novel approach of selecting passages based on the predicted decomposition of the multi-hop questions. We evaluate on three broadly established multi-hop question answering datasets.
  arXiv  Detail & Related papers  (2024-07-14T15:25:08Z)
• Reasoning in Conversation: Solving Subjective Tasks through Dialogue Simulation for Large Language Models [56.93074140619464]
  We propose RiC (Reasoning in Conversation), a method that focuses on solving subjective tasks through dialogue simulation. The motivation of RiC is to mine useful contextual information by simulating dialogues instead of supplying chain-of-thought style rationales. We evaluate both API-based and open-source LLMs including GPT-4, ChatGPT, and OpenChat across twelve tasks.
  arXiv  Detail & Related papers  (2024-02-27T05:37:10Z)
• Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves [57.974103113675795]
  We present a method named Rephrase and Respond' (RaR) which allows Large Language Models to rephrase and expand questions posed by humans. RaR serves as a simple yet effective prompting method for improving performance. We show that RaR is complementary to the popular Chain-of-Thought (CoT) methods, both theoretically and empirically.
  arXiv  Detail & Related papers  (2023-11-07T18:43:34Z)
• What should I Ask: A Knowledge-driven Approach for Follow-up Questions Generation in Conversational Surveys [63.51903260461746]
  We propose a novel task for knowledge-driven follow-up question generation in conversational surveys. We constructed a new human-annotated dataset of human-written follow-up questions with dialogue history and labeled knowledge. We then propose a two-staged knowledge-driven model for the task, which generates informative and coherent follow-up questions.
  arXiv  Detail & Related papers  (2022-05-23T00:57:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.