Chain-of-Discussion: A Multi-Model Framework for Complex Evidence-Based Question Answering

• URL: http://arxiv.org/abs/2402.16313v3
• Date: Mon, 16 Dec 2024 07:11:59 GMT
• Title: Chain-of-Discussion: A Multi-Model Framework for Complex Evidence-Based Question Answering
• Authors: Mingxu Tao, Dongyan Zhao, Yansong Feng,
• Abstract summary: We propose a novel Chain-ofDiscussion framework to leverage the synergy among open-source Large Language Models. Our experiments show that discussions among multiple LLMs play a vital role in enhancing the quality of answers.
• Score: 55.295699268654545
• License:
• Abstract: Open-ended question answering requires models to find appropriate evidence to form wellreasoned, comprehensive and helpful answers. In practical applications, models also need to engage in extended discussions on potential scenarios closely relevant to the question. With augmentation of retrieval module, open-source Large Language Models (LLMs) can produce coherent answers often with different focuses, but are still sub-optimal in terms of reliable evidence selection and in-depth question analysis. In this paper, we propose a novel Chain-ofDiscussion framework to leverage the synergy among multiple open-source LLMs aiming to provide more correct and more comprehensive answers for open-ended QA, although they are not strong enough individually. Our experiments show that discussions among multiple LLMs play a vital role in enhancing the quality of answers.

Related papers

• LRQ-Fact: LLM-Generated Relevant Questions for Multimodal Fact-Checking [14.647261841209767]
  We propose a fully-automated framework, LRQ-Fact, for multimodal fact-checking. It generates comprehensive questions and answers for probing multimodal content. It then evaluates both the original content and the generated questions and answers to assess the overall veracity.
  arXiv  Detail & Related papers  (2024-10-06T20:33:22Z)
• AQA: Adaptive Question Answering in a Society of LLMs via Contextual Multi-Armed Bandit [59.10281630985958]
  In question answering (QA), different questions can be effectively addressed with different answering strategies. We develop a dynamic method that adaptively selects the most suitable QA strategy for each question. Our experiments show that the proposed solution is viable for adaptive orchestration of a QA system with multiple modules.
  arXiv  Detail & Related papers  (2024-09-20T12:28:18Z)
• Multi-LLM QA with Embodied Exploration [55.581423861790945]
  We investigate the use of Multi-Embodied LLM Explorers (MELE) for question-answering in an unknown environment. Multiple LLM-based agents independently explore and then answer queries about a household environment. We analyze different aggregation methods to generate a single, final answer for each query.
  arXiv  Detail & Related papers  (2024-06-16T12:46:40Z)
• Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoning [49.3242278912771]
  We introduce a novel multimodal RAG framework named RMR (Retrieval Meets Reasoning) The RMR framework employs a bi-modal retrieval module to identify the most relevant question-answer pairs. It significantly boosts the performance of various vision-language models across a spectrum of benchmark datasets.
  arXiv  Detail & Related papers  (2024-05-31T14:23:49Z)
• Crafting Interpretable Embeddings by Asking LLMs Questions [89.49960984640363]
  Large language models (LLMs) have rapidly improved text embeddings for a growing array of natural-language processing tasks. We introduce question-answering embeddings (QA-Emb), embeddings where each feature represents an answer to a yes/no question asked to an LLM. We use QA-Emb to flexibly generate interpretable models for predicting fMRI voxel responses to language stimuli.
  arXiv  Detail & Related papers  (2024-05-26T22:30:29Z)
• Enhancing Answer Selection in Community Question Answering with Pre-trained and Large Language Models [0.9065034043031668]
  We first propose the Question-Answer cross attention networks (QAN) with pre-trained models for answer selection. We then utilize large language model (LLM) to perform answer selection with knowledge augmentation. Experiments show that the QAN model state-of-the-art performance on two datasets, SemEval2015 and SemEval 2017.
  arXiv  Detail & Related papers  (2023-11-29T10:24:50Z)
• Self-prompted Chain-of-Thought on Large Language Models for Open-domain Multi-hop Reasoning [70.74928578278957]
  In open-domain question-answering (ODQA), most existing questions require single-hop reasoning on commonsense. Large language models (LLMs) have found significant utility in facilitating ODQA without external corpus. We propose Self-prompted Chain-of-Thought (SP-CoT), an automated framework to mass-produce high quality CoTs.
  arXiv  Detail & Related papers  (2023-10-20T14:51:10Z)
• Federated Prompting and Chain-of-Thought Reasoning for Improving LLMs Answering [13.735277588793997]
  We investigate how to enhance answer precision in frequently asked questions posed by distributed users using cloud-based Large Language Models (LLMs) Our study focuses on a typical situations where users ask similar queries that involve identical mathematical reasoning steps and problem-solving procedures. We propose to improve the distributed synonymous questions using Self-Consistency (SC) and Chain-of-Thought (CoT) techniques.
  arXiv  Detail & Related papers  (2023-04-27T01:48:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.