An Interactive Query Generation Assistant using LLM-based Prompt
Modification and User Feedback
- URL: http://arxiv.org/abs/2311.11226v1
- Date: Sun, 19 Nov 2023 04:42:24 GMT
- Title: An Interactive Query Generation Assistant using LLM-based Prompt
Modification and User Feedback
- Authors: Kaustubh D. Dhole, Ramraj Chandradevan, Eugene Agichtein
- Abstract summary: The proposed interface is a novel search interface which supports automatic and interactive query generation over a mono-linguial or multi-lingual document collection.
The interface enables the users to refine the queries generated by different LLMs, to provide feedback on the retrieved documents or passages, and is able to incorporate the users' feedback as prompts to generate more effective queries.
- Score: 9.461978375200102
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: While search is the predominant method of accessing information, formulating
effective queries remains a challenging task, especially for situations where
the users are not familiar with a domain, or searching for documents in other
languages, or looking for complex information such as events, which are not
easily expressible as queries. Providing example documents or passages of
interest, might be easier for a user, however, such query-by-example scenarios
are prone to concept drift, and are highly sensitive to the query generation
method. This demo illustrates complementary approaches of using LLMs
interactively, assisting and enabling the user to provide edits and feedback at
all stages of the query formulation process. The proposed Query Generation
Assistant is a novel search interface which supports automatic and interactive
query generation over a mono-linguial or multi-lingual document collection.
Specifically, the proposed assistive interface enables the users to refine the
queries generated by different LLMs, to provide feedback on the retrieved
documents or passages, and is able to incorporate the users' feedback as
prompts to generate more effective queries. The proposed interface is a
valuable experimental tool for exploring fine-tuning and prompting of LLMs for
query generation to qualitatively evaluate the effectiveness of retrieval and
ranking models, and for conducting Human-in-the-Loop (HITL) experiments for
complex search tasks where users struggle to formulate queries without such
assistance.
Related papers
- Synthetic Query Generation using Large Language Models for Virtual Assistants [7.446599238906526]
We explore the use of Large Language Models (LLMs) to generate synthetic queries that are complementary to template-based methods.
We find that LLMs generate more verbose queries, compared to template-based methods, and reference aspects specific to the entity.
arXiv Detail & Related papers (2024-06-10T18:50:57Z) - Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars [66.823588073584]
Large language models (LLMs) have shown impressive capabilities in real-world applications.
The quality of these exemplars in the prompt greatly impacts performance.
Existing methods fail to adequately account for the impact of exemplar ordering on the performance.
arXiv Detail & Related papers (2024-05-25T08:23:05Z) - Leveraging Large Language Models for Multimodal Search [0.6249768559720121]
This paper introduces a novel multimodal search model that achieves a new performance milestone on the Fashion200K dataset.
We also propose a novel search interface integrating Large Language Models (LLMs) to facilitate natural language interaction.
arXiv Detail & Related papers (2024-04-24T10:30:42Z) - Q-PEFT: Query-dependent Parameter Efficient Fine-tuning for Text Reranking with Large Language Models [28.105271954633682]
We introduce a query-dependent parameter efficient fine-tuning (Q-PEFT) approach for text reranking to leak information to Large Language Models (LLMs)
We utilize the query to extract the top-$k$ tokens from input documents, serving as contextual clues.
We further augment Q-PEFT by substituting the retrieval mechanism with a multi-head attention layer to achieve end-to-end training and cover all the tokens in the documents.
arXiv Detail & Related papers (2024-04-06T06:44:41Z) - Improving Topic Relevance Model by Mix-structured Summarization and LLM-based Data Augmentation [16.170841777591345]
In most social search scenarios such as Dianping, modeling search relevance always faces two challenges.
We first take queryd with the query-based summary and the document summary without query as the input of topic relevance model.
Then, we utilize the language understanding and generation abilities of large language model (LLM) to rewrite and generate query from queries and documents in existing training data.
arXiv Detail & Related papers (2024-04-03T10:05:47Z) - Effective and Efficient Conversation Retrieval for Dialogue State Tracking with Implicit Text Summaries [48.243879779374836]
Few-shot dialogue state tracking (DST) with Large Language Models (LLM) relies on an effective and efficient conversation retriever to find similar in-context examples for prompt learning.
Previous works use raw dialogue context as search keys and queries, and a retriever is fine-tuned with annotated dialogues to achieve superior performance.
We handle the task of conversation retrieval based on text summaries of the conversations.
A LLM-based conversation summarizer is adopted for query and key generation, which enables effective maximum inner product search.
arXiv Detail & Related papers (2024-02-20T14:31:17Z) - Social Commonsense-Guided Search Query Generation for Open-Domain
Knowledge-Powered Conversations [66.16863141262506]
We present a novel approach that focuses on generating internet search queries guided by social commonsense.
Our proposed framework addresses passive user interactions by integrating topic tracking, commonsense response generation and instruction-driven query generation.
arXiv Detail & Related papers (2023-10-22T16:14:56Z) - Decomposing Complex Queries for Tip-of-the-tongue Retrieval [72.07449449115167]
Complex queries describe content elements (e.g., book characters or events), information beyond the document text.
This retrieval setting, called tip of the tongue (TOT), is especially challenging for models reliant on lexical and semantic overlap between query and document text.
We introduce a simple yet effective framework for handling such complex queries by decomposing the query into individual clues, routing those as sub-queries to specialized retrievers, and ensembling the results.
arXiv Detail & Related papers (2023-05-24T11:43:40Z) - Query Rewriting for Retrieval-Augmented Large Language Models [139.242907155883]
Large Language Models (LLMs) play powerful, black-box readers in the retrieve-then-read pipeline.
This work introduces a new framework, Rewrite-Retrieve-Read instead of the previous retrieve-then-read for the retrieval-augmented LLMs.
arXiv Detail & Related papers (2023-05-23T17:27:50Z) - Graph Enhanced BERT for Query Understanding [55.90334539898102]
query understanding plays a key role in exploring users' search intents and facilitating users to locate their most desired information.
In recent years, pre-trained language models (PLMs) have advanced various natural language processing tasks.
We propose a novel graph-enhanced pre-training framework, GE-BERT, which can leverage both query content and the query graph.
arXiv Detail & Related papers (2022-04-03T16:50:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.