Dynamic Knowledge Integration for Evidence-Driven Counter-Argument Generation with Large Language Models

• URL: http://arxiv.org/abs/2503.05328v1
• Date: Fri, 07 Mar 2025 11:13:33 GMT
• Title: Dynamic Knowledge Integration for Evidence-Driven Counter-Argument Generation with Large Language Models
• Authors: Anar Yeginbergen, Maite Oronoz, Rodrigo Agerri,
• Abstract summary: This paper investigates the role of dynamic external knowledge integration in improving counter-argument generation using Large Language Models (LLMs)<n>We introduce a new manually curated dataset of argument and counter-argument pairs designed to balance argumentative complexity with evaluative feasibility.<n>Our experimental results demonstrate that integrating dynamic external knowledge from the web significantly improves the quality of generated counter-arguments.
• Score: 5.735035463793008
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper investigates the role of dynamic external knowledge integration in improving counter-argument generation using Large Language Models (LLMs). While LLMs have shown promise in argumentative tasks, their tendency to generate lengthy, potentially unfactual responses highlights the need for more controlled and evidence-based approaches. We introduce a new manually curated dataset of argument and counter-argument pairs specifically designed to balance argumentative complexity with evaluative feasibility. We also propose a new LLM-as-a-Judge evaluation methodology that shows a stronger correlation with human judgments compared to traditional reference-based metrics. Our experimental results demonstrate that integrating dynamic external knowledge from the web significantly improves the quality of generated counter-arguments, particularly in terms of relatedness, persuasiveness, and factuality. The findings suggest that combining LLMs with real-time external knowledge retrieval offers a promising direction for developing more effective and reliable counter-argumentation systems.

Related papers

• Improving Multilingual Retrieval-Augmented Language Models through Dialectic Reasoning Argumentations [65.11348389219887]
  We introduce Dialectic-RAG (DRAG), a modular approach that evaluates retrieved information by comparing, contrasting, and resolving conflicting perspectives. We show the impact of our framework both as an in-context learning strategy and for constructing demonstrations to instruct smaller models.
  arXiv  Detail & Related papers  (2025-04-07T06:55:15Z)
• Enhancing Multi-Hop Fact Verification with Structured Knowledge-Augmented Large Language Models [26.023148371263012]
  We propose a novel Structured Knowledge-Augmented LLM-based Network (LLM-SKAN) for multi-hop fact verification. Specifically, we utilize an LLM-driven Knowledge Extractor to capture fine-grained information, including entities and their complicated relations. The experimental results on four common-used datasets demonstrate the effectiveness and superiority of our model.
  arXiv  Detail & Related papers  (2025-03-11T14:47:24Z)
• Exploring and Evaluating Multimodal Knowledge Reasoning Consistency of Multimodal Large Language Models [52.569132872560814]
  multimodal large language models (MLLMs) have achieved significant breakthroughs, enhancing understanding across text and vision.<n>However, current MLLMs still face challenges in effectively integrating knowledge across these modalities during multimodal knowledge reasoning.<n>We analyze and compare the extent of consistency degradation in multimodal knowledge reasoning within MLLMs.
  arXiv  Detail & Related papers  (2025-03-03T09:01:51Z)
• Your Language Model May Think Too Rigidly: Achieving Reasoning Consistency with Symmetry-Enhanced Training [66.48331530995786]
  We propose syMmetry-ENhanceD (MEND) Data Augmentation, a data-centric approach that improves the model's ability to extract useful information from context.<n>Unlike existing methods that emphasize reasoning chain augmentation, our approach improves model robustness at the knowledge extraction stage.<n>Experiments on both logical and arithmetic reasoning tasks show that MEND enhances reasoning performance across diverse query variations.
  arXiv  Detail & Related papers  (2025-02-25T03:03:35Z)
• Calling a Spade a Heart: Gaslighting Multimodal Large Language Models via Negation [65.92001420372007]
  This paper systematically evaluates state-of-the-art Multimodal Large Language Models (MLLMs) across diverse benchmarks.<n>We show significant performance drops when negation arguments are introduced to initially correct responses.
  arXiv  Detail & Related papers  (2025-01-31T10:37:48Z)
• Learning to Generate Research Idea with Dynamic Control [21.30777644522451]
  Large language models (LLMs) have shown promise in generating hypotheses and research ideas. We introduce a novel framework that employs a two-stage approach combiningSupervised Fine-Tuning (SFT) and controllable Reinforcement Learning (RL) Our framework provides a balanced approach to research ideation, achieving high-quality outcomes by dynamically navigating the trade-offs among novelty, feasibility, and effectiveness.
  arXiv  Detail & Related papers  (2024-12-19T08:28:18Z)
• Parenting: Optimizing Knowledge Selection of Retrieval-Augmented Language Models with Parameter Decoupling and Tailored Tuning [14.403812623299027]
  Retrieval-Augmented Generation (RAG) offers an effective solution to the issues faced by Large Language Models (LLMs) in hallucination generation and knowledge obsolescence. We propose Parenting, a novel framework that decouples, identifies, and purposefully optimize parameter subspaces related to adherence and robustness.
  arXiv  Detail & Related papers  (2024-10-14T10:26:57Z)
• GIVE: Structured Reasoning of Large Language Models with Knowledge Graph Inspired Veracity Extrapolation [108.2008975785364]
  Graph Inspired Veracity Extrapolation (GIVE) is a novel reasoning method that merges parametric and non-parametric memories to improve accurate reasoning with minimal external input.<n>GIVE guides the LLM agent to select the most pertinent expert data (observe), engage in query-specific divergent thinking (reflect), and then synthesize this information to produce the final output (speak)
  arXiv  Detail & Related papers  (2024-10-11T03:05:06Z)
• The Role of Deductive and Inductive Reasoning in Large Language Models [37.430396755248104]
  We propose the Deductive and InDuctive(DID) method to enhance Large Language Models (LLMs) reasoning.<n>DID implements a dual-metric complexity evaluation system that combines Littlestone dimension and information entropy.<n>Our results demonstrate significant improvements in reasoning quality and solution accuracy.
  arXiv  Detail & Related papers  (2024-10-03T18:30:47Z)
• Improving Retrieval Augmented Language Model with Self-Reasoning [20.715106330314605]
  We propose a novel self-reasoning framework aimed at improving the reliability and traceability of RALMs.<n>The framework involves constructing self-reason trajectories with three processes: a relevance-aware process, an evidence-aware selective process, and a trajectory analysis process.<n>We have evaluated our framework across four public datasets to demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2024-07-29T09:05:10Z)
• Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication [76.04373033082948]
  Large Language Models (LLMs) have recently made significant strides in complex reasoning tasks through the Chain-of-Thought technique. We propose Exchange-of-Thought (EoT), a novel framework that enables cross-model communication during problem-solving.
  arXiv  Detail & Related papers  (2023-12-04T11:53:56Z)
• EpiK-Eval: Evaluation for Language Models as Epistemic Models [16.485951373967502]
  We introduce EpiK-Eval, a novel question-answering benchmark tailored to evaluate LLMs' proficiency in formulating a coherent and consistent knowledge representation from segmented narratives. We argue that these shortcomings stem from the intrinsic nature of prevailing training objectives. The findings from this study offer insights for developing more robust and reliable LLMs.
  arXiv  Detail & Related papers  (2023-10-23T21:15:54Z)
• Improving Open Information Extraction with Large Language Models: A Study on Demonstration Uncertainty [52.72790059506241]
  Open Information Extraction (OIE) task aims at extracting structured facts from unstructured text. Despite the potential of large language models (LLMs) like ChatGPT as a general task solver, they lag behind state-of-the-art (supervised) methods in OIE tasks.
  arXiv  Detail & Related papers  (2023-09-07T01:35:24Z)
• Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models [115.7508325840751]
  The recent success of large language models (LLMs) has shown great potential to develop more powerful conversational recommender systems (CRSs) In this paper, we embark on an investigation into the utilization of ChatGPT for conversational recommendation, revealing the inadequacy of the existing evaluation protocol. We propose an interactive Evaluation approach based on LLMs named iEvaLM that harnesses LLM-based user simulators.
  arXiv  Detail & Related papers  (2023-05-22T15:12:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.