Retrieval-Augmented Thought Process as Sequential Decision Making
- URL: http://arxiv.org/abs/2402.07812v1
- Date: Mon, 12 Feb 2024 17:17:50 GMT
- Title: Retrieval-Augmented Thought Process as Sequential Decision Making
- Authors: Thomas Pouplin, Hao Sun, Samuel Holt, Mihaela van der Schaar
- Abstract summary: We introduce the Retrieval-Augmented Thought Process (RATP)
RATP formulates the thought generation of Large Language Models as a multiple-step decision process.
It achieves a 50% improvement over existing in-context retrieval-augmented language models.
- Score: 58.87539195379386
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Large Language Models (LLMs) have demonstrated their strong ability to assist
people and show "sparks of intelligence". However, several open challenges
hinder their wider application: such as concerns over privacy, tendencies to
produce hallucinations, and difficulties in handling long contexts. In this
work, we address those challenges by introducing the Retrieval-Augmented
Thought Process (RATP). Given access to external knowledge, RATP formulates the
thought generation of LLMs as a multiple-step decision process. To optimize
such a thought process, RATP leverages Monte-Carlo Tree Search, and learns a
Q-value estimator that permits cost-efficient inference. In addressing the task
of question-answering with private data, where ethical and security concerns
limit LLM training methods, RATP achieves a 50% improvement over existing
in-context retrieval-augmented language models.
Related papers
- Reasoning with Large Language Models, a Survey [2.831296564800826]
This paper reviews the rapidly expanding field of prompt-based reasoning with LLMs.
Our taxonomy identifies different ways to generate, evaluate, and control multi-step reasoning.
We find that self-improvement, self-reflection, and some meta abilities of the reasoning processes are possible through the judicious use of prompts.
arXiv Detail & Related papers (2024-07-16T08:49:35Z) - MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time [51.5039731721706]
MindStar is a purely inference-based searching method for large language models.
It formulates reasoning tasks as searching problems and proposes two search ideas to identify the optimal reasoning paths.
It significantly enhances the reasoning abilities of open-source models, such as Llama-2-13B and Mistral-7B, and achieves comparable performance to GPT-3.5 and Grok-1.
arXiv Detail & Related papers (2024-05-25T15:07:33Z) - Probing the Multi-turn Planning Capabilities of LLMs via 20 Question
Games [14.063311955315077]
Large language models (LLMs) are effective at answering questions that are clearly asked.
When faced with ambiguous queries they can act unpredictably and produce incorrect outputs.
This underscores the need for the development of intelligent agents capable of asking clarification questions to resolve ambiguities effectively.
arXiv Detail & Related papers (2023-10-02T16:55:37Z) - Furthest Reasoning with Plan Assessment: Stable Reasoning Path with
Retrieval-Augmented Large Language Models [10.04323204974924]
Multi-Hop Question Answering (MHQA) stands as a widely discussed category.
Existing methods employ Large Language Models (LLMs) to generate reasoning paths and plans.
We propose a novel pipeline for MHQA called Furthest-Reasoning-with-Plan-Assessment (FuRePA)
arXiv Detail & Related papers (2023-09-22T10:15:13Z) - LaGR-SEQ: Language-Guided Reinforcement Learning with Sample-Efficient
Querying [71.86163159193327]
Large language models (LLMs) have recently demonstrated their impressive ability to provide context-aware responses via text.
This ability could potentially be used to predict plausible solutions in sequential decision making tasks pertaining to pattern completion.
We introduce LaGR, which uses this predictive ability of LLMs to propose solutions to tasks that have been partially completed by a primary reinforcement learning (RL) agent.
arXiv Detail & Related papers (2023-08-21T02:07:35Z) - RRAML: Reinforced Retrieval Augmented Machine Learning [10.94680155282906]
We propose a novel framework called Reinforced Retrieval Augmented Machine Learning (RRAML)
RRAML integrates the reasoning capabilities of large language models with supporting information retrieved by a purpose-built retriever from a vast user-provided database.
We believe that the research agenda outlined in this paper has the potential to profoundly impact the field of AI.
arXiv Detail & Related papers (2023-07-24T13:51:19Z) - Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate [85.89346248535922]
We propose a Multi-Agent Debate (MAD) framework, in which multiple agents express their arguments in the state of "tit for tat" and a judge manages the debate process to obtain a final solution.
Our framework encourages divergent thinking in LLMs which would be helpful for tasks that require deep levels of contemplation.
arXiv Detail & Related papers (2023-05-30T15:25:45Z) - Check Your Facts and Try Again: Improving Large Language Models with
External Knowledge and Automated Feedback [127.75419038610455]
Large language models (LLMs) are able to generate human-like, fluent responses for many downstream tasks.
This paper proposes a LLM-Augmenter system, which augments a black-box LLM with a set of plug-and-play modules.
arXiv Detail & Related papers (2023-02-24T18:48:43Z) - Rethinking with Retrieval: Faithful Large Language Model Inference [91.66406351103484]
We propose a novel post-processing approach, rethinking with retrieval (RR)
RR retrieves relevant external knowledge based on the reasoning steps obtained from the chain-of-thought prompting.
We evaluate the effectiveness of RR through extensive experiments with GPT-3 on three complex reasoning tasks.
arXiv Detail & Related papers (2022-12-31T22:35:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.