DS@GT ARC at Touché: Large Language Models for Retrieval-Augmented Debate

• URL: http://arxiv.org/abs/2608.08143v1
• Date: Sat, 08 Aug 2026 14:04:12 GMT
• Title: DS@GT ARC at Touché: Large Language Models for Retrieval-Augmented Debate
• Abstract summary: We extend the DS@GT ARC working-note submission to the Touché 2025 Retrieval-Augmented Debate task.<n>The task has two subtasks: generating the next utterance in a simulated debate, and evaluating debate responses.<n>We summarize the results from the working paper and explore whether multi-LLM evaluator agreement is a reliable proxy for official evaluation performance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We extend the DS@GT ARC working-note submission to the Touché 2025 Retrieval-Augmented Debate task. The task has two subtasks: generating the next utterance in a simulated debate, and evaluating debate responses according to the Gricean maxims of Quantity, Quality, Relation, and Manner. The DS@GT ARC submission consisted of six leading LLMs from three providers through a retrieval-augmented prompting pipeline. We summarize the results from the working paper and explore whether multi-LLM evaluator agreement is a reliable proxy for official evaluation performance. The analysis shows that frontier LLM systems are strong response generators, and as evaluators they agree strongly within model families. However this consensus does not reliably track the official evaluation target, with the largest gap on the Quality maxim. The accompanying source code for this paper is located at https://github.com/dsgt-arc/touche-2025-rad and https://github.com/dsgt-arc/touche-2025-rad-analysis.

Related papers

• RepoReasoner: Evaluating Repository-Level Code Reasoning Ability of Long-Context Language Models [54.53236295237077]
  We introduce RepoReasoner, a benchmark for evaluating repository-level code reasoning.<n> Output Prediction measures fine-grained, stateful execution reasoning across files.<n>Call Chain Prediction evaluates high-level architectural dependency understanding under noisy context.
  arXiv  Detail & Related papers  (2026-07-28T17:12:41Z)
• DS@GT ARC at LongEval: Citation Integrity and Factual Grounding in Scientific QA [0.0]
  We examine a divergence between traditional natural language evaluation metrics and citation integrity as applied to RAG QA systems.<n>We evaluate a corrective pipeline using Corrective RAG (CRAG) and CiteFix against baseline and frontier model benchmark RAG QA scores.<n>We propose that evaluation of trustworthy RAG QA requires metrics that reward strict answer grounding.
  arXiv  Detail & Related papers  (2026-07-15T22:35:02Z)
• DS@GT at Touché: Large Language Models for Retrieval-Augmented Debate [0.0]
  We deploy six leading publicly available models for the Retrieval-Augmented Debate and Evaluation.<n>The evaluation is performed by measuring four key metrics: Quality, Quantity, Manner, and Relation.<n>Although LLMs perform well in debates when given related arguments, they tend to be verbose in responses yet consistent in evaluation.
  arXiv  Detail & Related papers  (2025-07-12T00:20:00Z)
• RAG-Zeval: Towards Robust and Interpretable Evaluation on RAG Responses through End-to-End Rule-Guided Reasoning [64.46921169261852]
  RAG-Zeval is a novel end-to-end framework that formulates faithfulness and correctness evaluation as a rule-guided reasoning task.<n>Our approach trains evaluators with reinforcement learning, facilitating compact models to generate comprehensive and sound assessments.<n>Experiments demonstrate RAG-Zeval's superior performance, achieving the strongest correlation with human judgments.
  arXiv  Detail & Related papers  (2025-05-28T14:55:33Z)
• GLIDER: Grading LLM Interactions and Decisions using Explainable Ranking [0.9614204956530676]
  We introduce GLIDER, a powerful 3B evaluator LLM that can score any text input and associated context on arbitrary user defined criteria.<n>GLIDER shows higher Pearson's correlation than GPT-4o on FLASK and greatly outperforms prior evaluation models.<n>It supports fine-grained scoring, multilingual reasoning, span highlighting and was trained on 685 domains and 183 criteria.
  arXiv  Detail & Related papers  (2024-12-18T18:41:12Z)
• ConQRet: Benchmarking Fine-Grained Evaluation of Retrieval Augmented Argumentation with LLM Judges [23.179246872272362]
  Computational argumentation has become increasingly important in today's polarized environment.<n>We propose a new benchmark featuring long and complex human-authored arguments on debated topics, grounded in real-world websites.<n>Our proposed LLM Judges and the ConQRet benchmark can enable rapid progress in computational argumentation.
  arXiv  Detail & Related papers  (2024-12-06T17:35:52Z)
• Argue with Me Tersely: Towards Sentence-Level Counter-Argument Generation [62.069374456021016]
  We present the ArgTersely benchmark for sentence-level counter-argument generation. We also propose Arg-LlaMA for generating high-quality counter-argument.
  arXiv  Detail & Related papers  (2023-12-21T06:51:34Z)
• RADE: Reference-Assisted Dialogue Evaluation for Open-Domain Dialogue [37.82954848948347]
  We propose the Reference-Assisted Dialogue Evaluation (RADE) approach under the multi-task learning framework. RADE explicitly compares reference and the candidate response to predict their overall scores. Experiments on our three datasets and two existing benchmarks demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2023-09-15T04:47:19Z)
• Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models [115.7508325840751]
  The recent success of large language models (LLMs) has shown great potential to develop more powerful conversational recommender systems (CRSs) In this paper, we embark on an investigation into the utilization of ChatGPT for conversational recommendation, revealing the inadequacy of the existing evaluation protocol. We propose an interactive Evaluation approach based on LLMs named iEvaLM that harnesses LLM-based user simulators.
  arXiv  Detail & Related papers  (2023-05-22T15:12:43Z)
• Multimodal Chain-of-Thought Reasoning in Language Models [94.70184390935661]
  We propose Multimodal-CoT that incorporates language (text) and vision (images) modalities into a two-stage framework. Experimental results on ScienceQA and A-OKVQA benchmark datasets show the effectiveness of our proposed approach.
  arXiv  Detail & Related papers  (2023-02-02T07:51:19Z)
• Don't Copy the Teacher: Data and Model Challenges in Embodied Dialogue [92.01165203498299]
  Embodied dialogue instruction following requires an agent to complete a complex sequence of tasks from a natural language exchange. This paper argues that imitation learning (IL) and related low-level metrics are actually misleading and do not align with the goals of embodied dialogue research.
  arXiv  Detail & Related papers  (2022-10-10T05:51:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.