Passage Query Methods for Retrieval and Reranking in Conversational Agents

• URL: http://arxiv.org/abs/2503.00238v1
• Date: Fri, 28 Feb 2025 22:53:56 GMT
• Title: Passage Query Methods for Retrieval and Reranking in Conversational Agents
• Authors: Victor De Lima, Grace Hui Yang,
• Abstract summary: This paper presents our approach to the TREC Interactive Knowledge Assistance Track (iKAT)<n>We develop passage queries that align with the target document's expected format to improve query-document matching during retrieval.<n>Findings suggest that PQs improve semantic alignment with target documents and demonstrate their potential to improve multi-turn dialogue systems.
• Score: 7.780766187171572
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents our approach to the TREC Interactive Knowledge Assistance Track (iKAT), which focuses on improving conversational information-seeking (CIS) systems. While recent advancements in CIS have improved conversational agents' ability to assist users, significant challenges remain in understanding context and retrieving relevant documents across domains and dialogue turns. To address these issues, we extend the Generate-Retrieve-Generate pipeline by developing passage queries (PQs) that align with the target document's expected format to improve query-document matching during retrieval. We propose two variations of this approach: Weighted Reranking and Short and Long Passages. Each method leverages a Meta Llama model for context understanding and generating queries and responses. Passage ranking evaluation results show that the Short and Long Passages approach outperformed the organizers' baselines, performed best among Llama-based systems in the track, and achieved results comparable to GPT-4-based systems. These results indicate that the method effectively balances efficiency and performance. Findings suggest that PQs improve semantic alignment with target documents and demonstrate their potential to improve multi-turn dialogue systems.

Related papers

• IRLab@iKAT24: Learned Sparse Retrieval with Multi-aspect LLM Query Generation for Conversational Search [6.974395116689502]
  iKAT 2024 focuses on advancing conversational assistants, able to adapt their interaction and responses from personalized user knowledge. The track incorporates a Personal Textual Knowledge Base (PTKB) alongside Conversational AI tasks, such as passage ranking and response generation.
  arXiv  Detail & Related papers  (2024-11-22T05:18:35Z)
• AdaCQR: Enhancing Query Reformulation for Conversational Search via Sparse and Dense Retrieval Alignment [16.62505706601199]
  We present a novel framework AdaCQR for conversational search reformulation.<n>By aligning reformulation models with both term-based and semantic-based retrieval systems, AdaCQR enhances the generalizability of information-seeking queries.<n> Experimental results on the TopiOCQA and QReCC datasets demonstrate that AdaCQR outperforms the existing methods in a more efficient framework.
  arXiv  Detail & Related papers  (2024-07-02T05:50:16Z)
• Leveraging Translation For Optimal Recall: Tailoring LLM Personalization With User Profiles [0.0]
  This paper explores a novel technique for improving recall in cross-language information retrieval systems. The proposed methodology combines multi-level translation, semantic embedding-based expansion, and user profile-centered augmentation. Experiments on news and Twitter datasets demonstrate superior performance over baseline BM25 ranking.
  arXiv  Detail & Related papers  (2024-02-21T03:25:14Z)
• Strong and Efficient Baselines for Open Domain Conversational Question Answering [2.773656427800412]
  We study the State-of-the-Art (SotA) Dense Passage Retrieval (DPR) retriever and Fusion-in-Decoder (FiD) reader pipeline. We propose and evaluate strong yet simple and efficient baselines, by introducing a fast reranking component between the retriever and the reader. Experiments on two ODConvQA tasks, namely TopiOCQA and OR-QuAC, show that our method improves the SotA results, while reducing reader's latency by 60%.
  arXiv  Detail & Related papers  (2023-10-23T08:48:14Z)
• Modeling Uncertainty and Using Post-fusion as Fallback Improves Retrieval Augmented Generation with LLMs [80.74263278847063]
  The integration of retrieved passages and large language models (LLMs) has significantly contributed to improving open-domain question answering. This paper investigates different methods of combining retrieved passages with LLMs to enhance answer generation.
  arXiv  Detail & Related papers  (2023-08-24T05:26:54Z)
• Query Performance Prediction: From Ad-hoc to Conversational Search [55.37199498369387]
  Query performance prediction (QPP) is a core task in information retrieval. Research has shown the effectiveness and usefulness of QPP for ad-hoc search. Despite its potential, QPP for conversational search has been little studied.
  arXiv  Detail & Related papers  (2023-05-18T12:37:01Z)
• FCC: Fusing Conversation History and Candidate Provenance for Contextual Response Ranking in Dialogue Systems [53.89014188309486]
  We present a flexible neural framework that can integrate contextual information from multiple channels. We evaluate our model on the MSDialog dataset widely used for evaluating conversational response ranking tasks.
  arXiv  Detail & Related papers  (2023-03-31T23:58:28Z)
• Better Retrieval May Not Lead to Better Question Answering [59.1892787017522]
  A popular approach to improve the system's performance is to improve the quality of the retrieved context from the IR stage. We show that for StrategyQA, a challenging open-domain QA dataset that requires multi-hop reasoning, this common approach is surprisingly ineffective.
  arXiv  Detail & Related papers  (2022-05-07T16:59:38Z)
• Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters [52.725200145600624]
  We propose KnowExpert to bypass the retrieval process by injecting prior knowledge into the pre-trained language models with lightweight adapters. Experimental results show that KnowExpert performs comparably with the retrieval-based baselines.
  arXiv  Detail & Related papers  (2021-05-13T12:33:23Z)
• Multi-Stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting [56.268862325167575]
  We tackle conversational passage retrieval (ConvPR) with query reformulation integrated into a multi-stage ad-hoc IR system. We propose two conversational query reformulation (CQR) methods: (1) term importance estimation and (2) neural query rewriting. For the former, we expand conversational queries using important terms extracted from the conversational context with frequency-based signals. For the latter, we reformulate conversational queries into natural, standalone, human-understandable queries with a pretrained sequence-tosequence model.
  arXiv  Detail & Related papers  (2020-05-05T14:30:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.