Assessing the Sufficiency of Arguments through Conclusion Generation

• URL: http://arxiv.org/abs/2110.13495v1
• Date: Tue, 26 Oct 2021 08:47:53 GMT
• Title: Assessing the Sufficiency of Arguments through Conclusion Generation
• Abstract summary: An argument whose premises make its conclusion rationally worthy is called sufficient in argument quality research. In this paper, we hypothesize that the conclusion of a sufficient argument can be generated from its premises. Our best model achieves variant an F1-score of.885, outperforming the previous state-of-the-art and being on par with human experts.
• Score: 15.69811781628757
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The premises of an argument give evidence or other reasons to support a conclusion. However, the amount of support required depends on the generality of a conclusion, the nature of the individual premises, and similar. An argument whose premises make its conclusion rationally worthy to be drawn is called sufficient in argument quality research. Previous work tackled sufficiency assessment as a standard text classification problem, not modeling the inherent relation of premises and conclusion. In this paper, we hypothesize that the conclusion of a sufficient argument can be generated from its premises. To study this hypothesis, we explore the potential of assessing sufficiency based on the output of large-scale pre-trained language models. Our best model variant achieves an F1-score of .885, outperforming the previous state-of-the-art and being on par with human experts. While manual evaluation reveals the quality of the generated conclusions, their impact remains low ultimately.

Related papers

• Before the Action: Benchmarking LLMs on Prospective Hypothesis Discovery [95.44432473367836]
  Large language models (LLMs) excel at answering pre-specified questions, yet their ability to navigate the open-ended, pre-conclusion stage of discovery remains largely unmeasured.<n>We introduce Prospective Hypothesis Discovery (PHD), which asks models to autonomously construct grounded, discnative, and testable hypothesis spaces from inconclusive evidence.<n>We introduce HypoArena, comprising HypoData, a benchmark of 988 cases across six scientific and analytical domains, and HypoEval, an evaluation framework for open-ended hypothesis sets.
  arXiv  Detail & Related papers  (2026-07-17T08:56:43Z)
• Neurosymbolic Learning for Inference-Time Argumentation [18.461907013598424]
  Claim verification is an important problem in high-stakes settings, including health and finance.<n>We introduce inference-time argumentation (ITA), a trainable neurosymbolic framework for ternary claim verification.
  arXiv  Detail & Related papers  (2026-05-19T16:49:29Z)
• WarrantScore: Modeling Warrants between Claims and Evidence for Substantiation Evaluation in Peer Reviews [19.27332072329702]
  The scientific peer-review process is facing a shortage of human resources due to the rapid growth in the number of submitted papers.<n>We propose a new evaluation metric for scientific review comments that assesses the logical inference between claims and evidence.
  arXiv  Detail & Related papers  (2026-01-24T08:54:25Z)
• CLATTER: Comprehensive Entailment Reasoning for Hallucination Detection [60.98964268961243]
  We propose that guiding models to perform a systematic and comprehensive reasoning process allows models to execute much finer-grained and accurate entailment decisions.<n>We define a 3-step reasoning process, consisting of (i) claim decomposition, (ii) sub-claim attribution and entailment classification, and (iii) aggregated classification, showing that such guided reasoning indeed yields improved hallucination detection.
  arXiv  Detail & Related papers  (2025-06-05T17:02:52Z)
• Causality can systematically address the monsters under the bench(marks) [64.36592889550431]
  Benchmarks are plagued by various biases, artifacts, or leakage. Models may behave unreliably due to poorly explored failure modes. causality offers an ideal framework to systematically address these challenges.
  arXiv  Detail & Related papers  (2025-02-07T17:01:37Z)
• Conceptual and Unbiased Reasoning in Language Models [98.90677711523645]
  We propose a novel conceptualization framework that forces models to perform conceptual reasoning on abstract questions. We show that existing large language models fall short on conceptual reasoning, dropping 9% to 28% on various benchmarks. We then discuss how models can improve since high-level abstract reasoning is key to unbiased and generalizable decision-making.
  arXiv  Detail & Related papers  (2024-03-30T00:53:53Z)
• CASA: Causality-driven Argument Sufficiency Assessment [79.13496878681309]
  We propose CASA, a zero-shot causality-driven argument sufficiency assessment framework. PS measures how likely introducing the premise event would lead to the conclusion when both the premise and conclusion events are absent. Experiments on two logical fallacy detection datasets demonstrate that CASA accurately identifies insufficient arguments.
  arXiv  Detail & Related papers  (2024-01-10T16:21:18Z)
• Advancing Counterfactual Inference through Nonlinear Quantile Regression [77.28323341329461]
  We propose a framework for efficient and effective counterfactual inference implemented with neural networks. The proposed approach enhances the capacity to generalize estimated counterfactual outcomes to unseen data. Empirical results conducted on multiple datasets offer compelling support for our theoretical assertions.
  arXiv  Detail & Related papers  (2023-06-09T08:30:51Z)
• Conclusion-based Counter-Argument Generation [26.540485804067536]
  In real-world debates, the most common way to counter an argument is to reason against its main point, that is, its conclusion. We propose a multitask approach that jointly learns to generate both the conclusion and the counter of an input argument.
  arXiv  Detail & Related papers  (2023-01-24T10:49:01Z)
• MetaLogic: Logical Reasoning Explanations with Fine-Grained Structure [129.8481568648651]
  We propose a benchmark to investigate models' logical reasoning capabilities in complex real-life scenarios. Based on the multi-hop chain of reasoning, the explanation form includes three main components. We evaluate the current best models' performance on this new explanation form.
  arXiv  Detail & Related papers  (2022-10-22T16:01:13Z)
• Analyzing and Evaluating Faithfulness in Dialogue Summarization [67.07947198421421]
  We first perform the fine-grained human analysis on the faithfulness of dialogue summaries and observe that over 35% of generated summaries are faithfully inconsistent respective the source dialogues. We present a new model-level faithfulness evaluation method. It examines generation models with multi-choice questions created by rule-based transformations.
  arXiv  Detail & Related papers  (2022-10-21T07:22:43Z)
• Comparing and extending the use of defeasible argumentation with quantitative data in real-world contexts [5.482532589225552]
  A non-monotonic formalism is one that allows the retraction of previous conclusions or claims, from premises, in light of new evidence. This study contributes to the body of knowledge through the exploitation of defeasible argumentation and its comparison to similar approaches.
  arXiv  Detail & Related papers  (2022-06-28T12:28:47Z)
• Logical Satisfiability of Counterfactuals for Faithful Explanations in NLI [60.142926537264714]
  We introduce the methodology of Faithfulness-through-Counterfactuals. It generates a counterfactual hypothesis based on the logical predicates expressed in the explanation. It then evaluates if the model's prediction on the counterfactual is consistent with that expressed logic.
  arXiv  Detail & Related papers  (2022-05-25T03:40:59Z)
• Interactive Model with Structural Loss for Language-based Abductive Reasoning [36.02450824915494]
  The abductive natural language inference task ($alpha$NLI) is proposed to infer the most plausible explanation between the cause and the event. We name this new model for $alpha$NLI: Interactive Model with Structural Loss (IMSL) Our IMSL has achieved the highest performance on the RoBERTa-large pretrained model, with ACC and AUC results increased by about 1% and 5% respectively.
  arXiv  Detail & Related papers  (2021-12-01T05:21:07Z)
• The Role of Pragmatic and Discourse Context in Determining Argument Impact [39.70446357000737]
  This paper presents a new dataset to initiate the study of this aspect of argumentation. It consists of a diverse collection of arguments covering 741 controversial topics and comprising over 47,000 claims. We propose predictive models that incorporate the pragmatic and discourse context of argumentative claims and show that they outperform models that rely on claim-specific linguistic features for predicting the perceived impact of individual claims within a particular line of argument.
  arXiv  Detail & Related papers  (2020-04-06T23:00:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.