DayDreamer at CQs-Gen 2025: Generating Critical Questions through Argument Scheme Completion

• URL: http://arxiv.org/abs/2505.15554v1
• Date: Wed, 21 May 2025 14:15:49 GMT
• Title: DayDreamer at CQs-Gen 2025: Generating Critical Questions through Argument Scheme Completion
• Authors: Wendi Zhou, Ameer Saadat-Yazdi, Nadin Kökciyan,
• Abstract summary: We present our system for the Critical Questions Generation (CQs-Gen) Shared Task at ArgMining 2025.<n>Our approach leverages large language models (LLMs) with chain-of-thought prompting to generate critical questions guided by Walton's argumentation schemes.<n>Our pipeline achieves competitive performance in the final test set, showing its potential to foster critical thinking given argumentative text and detect missing or uninformed claims.
• Score: 3.740062413173584
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Critical questions are essential resources to provoke critical thinking when encountering an argumentative text. We present our system for the Critical Questions Generation (CQs-Gen) Shared Task at ArgMining 2025. Our approach leverages large language models (LLMs) with chain-of-thought prompting to generate critical questions guided by Walton's argumentation schemes. For each input intervention, we conversationally prompt LLMs to instantiate the corresponding argument scheme template to first obtain structured arguments, and then generate relevant critical questions. Following this, we rank all the available critical questions by prompting LLMs to select the top 3 most helpful questions based on the original intervention text. This combination of structured argumentation theory and step-by-step reasoning enables the generation of contextually relevant and diverse critical questions. Our pipeline achieves competitive performance in the final test set, showing its potential to foster critical thinking given argumentative text and detect missing or uninformed claims. Code available at \href{https://git.ecdf.ed.ac.uk/s2236454/DayDreamer-CQs-Gen}{DayDreamer}.

Related papers

• ELLIS Alicante at CQs-Gen 2025: Winning the critical thinking questions shared task: LLM-based question generation and selection [7.152439554068969]
  This study is part of a shared task of the 12th Workshop on Argument Mining, co-located with ACL 2025.<n>We propose a two-step framework involving two small-scale open source language models: a Questioner that generates multiple candidate questions and a Judge that selects the most relevant ones.
  arXiv  Detail & Related papers  (2025-06-17T10:10:51Z)
• Benchmarking Critical Questions Generation: A Challenging Reasoning Task for Large Language Models [6.0158981171030685]
  Critical Questions Generation (CQs-Gen) aims to foster critical thinking by enabling systems to generate questions that expose underlying assumptions.<n>Despite growing interest in this area, progress has been hindered by the lack of suitable datasets and automatic evaluation standards.<n>This paper presents a comprehensive approach to support the development and benchmarking of systems for this task.
  arXiv  Detail & Related papers  (2025-05-16T15:08:04Z)
• SQuARE: Sequential Question Answering Reasoning Engine for Enhanced Chain-of-Thought in Large Language Models [4.328173053224842]
  This paper introduces SQuARE, a novel prompting technique designed to improve reasoning through a self-interrogation paradigm.<n>Building upon CoT frameworks, SQuARE prompts models to generate and resolve multiple auxiliary questions before tackling the main query.<n>Our evaluations, conducted with Llama 3 and GPT-4o models across multiple question-answering datasets, demonstrate that SQuARE significantly surpasses traditional CoT prompts and existing rephrase-and-respond methods.
  arXiv  Detail & Related papers  (2025-02-13T15:07:20Z)
• AGENT-CQ: Automatic Generation and Evaluation of Clarifying Questions for Conversational Search with LLMs [53.6200736559742]
  AGENT-CQ consists of two stages: a generation stage and an evaluation stage. CrowdLLM simulates human crowdsourcing judgments to assess generated questions and answers. Experiments on the ClariQ dataset demonstrate CrowdLLM's effectiveness in evaluating question and answer quality.
  arXiv  Detail & Related papers  (2024-10-25T17:06:27Z)
• Critical Questions Generation: Motivation and Challenges [6.0158981171030685]
  We propose a new task, consisting of processing an argumentative text to generate the critical questions raised by it. In argumentation theory CQs are tools designed to lay bare the blind spots of an argument by pointing at the information it could be missing. Research on CQs Generation using LLMs requires a reference dataset for large scale experimentation.
  arXiv  Detail & Related papers  (2024-10-18T09:46:38Z)
• Qsnail: A Questionnaire Dataset for Sequential Question Generation [76.616068047362]
  We present the first dataset specifically constructed for the questionnaire generation task, which comprises 13,168 human-written questionnaires. We conduct experiments on Qsnail, and the results reveal that retrieval models and traditional generative models do not fully align with the given research topic and intents. Despite enhancements through the chain-of-thought prompt and finetuning, questionnaires generated by language models still fall short of human-written questionnaires.
  arXiv  Detail & Related papers  (2024-02-22T04:14:10Z)
• QUDEVAL: The Evaluation of Questions Under Discussion Discourse Parsing [87.20804165014387]
  Questions Under Discussion (QUD) is a versatile linguistic framework in which discourse progresses as continuously asking questions and answering them. This work introduces the first framework for the automatic evaluation of QUD parsing. We present QUDeval, a dataset of fine-grained evaluation of 2,190 QUD questions generated from both fine-tuned systems and LLMs.
  arXiv  Detail & Related papers  (2023-10-23T03:03:58Z)
• Improving Question Generation with Multi-level Content Planning [70.37285816596527]
  This paper addresses the problem of generating questions from a given context and an answer, specifically focusing on questions that require multi-hop reasoning across an extended context. We propose MultiFactor, a novel QG framework based on multi-level content planning. Specifically, MultiFactor includes two components: FA-model, which simultaneously selects key phrases and generates full answers, and Q-model which takes the generated full answer as an additional input to generate questions.
  arXiv  Detail & Related papers  (2023-10-20T13:57:01Z)
• Inquisitive Question Generation for High Level Text Comprehension [60.21497846332531]
  We introduce INQUISITIVE, a dataset of 19K questions that are elicited while a person is reading through a document. We show that readers engage in a series of pragmatic strategies to seek information. We evaluate question generation models based on GPT-2 and show that our model is able to generate reasonable questions.
  arXiv  Detail & Related papers  (2020-10-04T19:03:39Z)
• Reinforced Multi-task Approach for Multi-hop Question Generation [47.15108724294234]
  We take up Multi-hop question generation, which aims at generating relevant questions based on supporting facts in the context. We employ multitask learning with the auxiliary task of answer-aware supporting fact prediction to guide the question generator. We demonstrate the effectiveness of our approach through experiments on the multi-hop question answering dataset, HotPotQA.
  arXiv  Detail & Related papers  (2020-04-05T10:16:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.