Counterfactual Learning To Rank for Utility-Maximizing Query Autocompletion

• URL: http://arxiv.org/abs/2204.10936v1
• Date: Fri, 22 Apr 2022 21:40:51 GMT
• Title: Counterfactual Learning To Rank for Utility-Maximizing Query Autocompletion
• Authors: Adam Block, Rahul Kidambi, Daniel N. Hill, Thorsten Joachims, and Inderjit S. Dhillon
• Abstract summary: We propose a new approach that explicitly optimize the query suggestions for downstream retrieval performance. We formulate this as a problem of ranking a set of rankings, where each query suggestion is represented by the downstream item ranking it produces. We then present a learning method that ranks query suggestions by the quality of their item rankings.
• Score: 40.31426350180036
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conventional methods for query autocompletion aim to predict which completed query a user will select from a list. A shortcoming of this approach is that users often do not know which query will provide the best retrieval performance on the current information retrieval system, meaning that any query autocompletion methods trained to mimic user behavior can lead to suboptimal query suggestions. To overcome this limitation, we propose a new approach that explicitly optimizes the query suggestions for downstream retrieval performance. We formulate this as a problem of ranking a set of rankings, where each query suggestion is represented by the downstream item ranking it produces. We then present a learning method that ranks query suggestions by the quality of their item rankings. The algorithm is based on a counterfactual learning approach that is able to leverage feedback on the items (e.g., clicks, purchases) to evaluate query suggestions through an unbiased estimator, thus avoiding the assumption that users write or select optimal queries. We establish theoretical support for the proposed approach and provide learning-theoretic guarantees. We also present empirical results on publicly available datasets, and demonstrate real-world applicability using data from an online shopping store.

Related papers

• Unsupervised Query Routing for Retrieval Augmented Generation [64.47987041500966]
  We introduce a novel unsupervised method that constructs the "upper-bound" response to evaluate the quality of retrieval-augmented responses. This evaluation enables the decision of the most suitable search engine for a given query. By eliminating manual annotations, our approach can automatically process large-scale real user queries and create training data.
  arXiv  Detail & Related papers  (2025-01-14T02:27:06Z)
• Maybe you are looking for CroQS: Cross-modal Query Suggestion for Text-to-Image Retrieval [15.757140563856675]
  This work introduces a novel task that focuses on suggesting minimal textual modifications needed to explore visually consistent subsets of the collection. To facilitate the evaluation and development of methods, we present a tailored benchmark named CroQS. Baseline methods from related fields, such as image captioning and content summarization, are adapted for this task to provide reference performance scores.
  arXiv  Detail & Related papers  (2024-12-18T13:24:09Z)
• Counterfactual Learning-Driven Representation Disentanglement for Search-Enhanced Recommendation [19.76299850698492]
  We propose a Counterfactual learning-driven representation disentanglement framework for search-enhanced recommendation. We leverage search queries to construct counterfactual signals to disentangle item representations, isolating only query-independent general features. Experiments on real datasets demonstrate ClardRec is effective in both collaborative filtering and sequential recommendation scenarios.
  arXiv  Detail & Related papers  (2024-11-14T09:51:50Z)
• ExcluIR: Exclusionary Neural Information Retrieval [74.08276741093317]
  We present ExcluIR, a set of resources for exclusionary retrieval. evaluation benchmark includes 3,452 high-quality exclusionary queries. training set contains 70,293 exclusionary queries, each paired with a positive document and a negative document.
  arXiv  Detail & Related papers  (2024-04-26T09:43:40Z)
• The Surprising Effectiveness of Rankers Trained on Expanded Queries [4.874071145951159]
  We improve the ranking performance of hard or difficult queries without compromising the performance of other queries. We combine relevance scores from the specialized ranker and the base ranker, along with a query performance score estimated for each query. In our experiments on the DL-Hard dataset, we find that a principled query performance based scoring method offers a significant improvement of up to 25% on the passage ranking task.
  arXiv  Detail & Related papers  (2024-04-03T09:12:22Z)
• Online Learning of Optimally Diverse Rankings [63.62764375279861]
  We propose an algorithm that efficiently learns the optimal list based on users' feedback only. We show that after $T$ queries, the regret of LDR scales as $O((N-L)log(T))$ where $N$ is the number of all items.
  arXiv  Detail & Related papers  (2021-09-13T12:13:20Z)
• Conditional Sequential Slate Optimization [15.10459152219771]
  A search ranking system typically orders the results by independent query-document scores to produce a slate of search results. We introduce conditional sequential slate optimization (CSSO), which jointly learns to optimize for traditional ranking metrics as well as prescribed distribution criteria of documents within the slate. The proposed method can be applied to practical real world problems such as enforcing diversity in e-commerce search results, mitigating bias in top results and personalization of results.
  arXiv  Detail & Related papers  (2021-08-12T09:14:46Z)
• High Quality Related Search Query Suggestions using Deep Reinforcement Learning [0.15229257192293202]
  "High Quality Related Search Query Suggestions" task aims at recommending search queries which are real, accurate, diverse, relevant and engaging. We train a Deep Reinforcement Learning model to predict the query a user would enter next. The reward signal is composed of long-term session-based user feedback, syntactic relatedness and estimated naturalness of generated query.
  arXiv  Detail & Related papers  (2021-08-10T05:22:32Z)
• Session-Aware Query Auto-completion using Extreme Multi-label Ranking [61.753713147852125]
  We take the novel approach of modeling session-aware query auto-completion as an e Multi-Xtreme Ranking (XMR) problem. We adapt a popular XMR algorithm for this purpose by proposing several modifications to the key steps in the algorithm. Our approach meets the stringent latency requirements for auto-complete systems while leveraging session information in making suggestions.
  arXiv  Detail & Related papers  (2020-12-09T17:56:22Z)
• Optimal Clustering from Noisy Binary Feedback [75.17453757892152]
  We study the problem of clustering a set of items from binary user feedback. We devise an algorithm with a minimal cluster recovery error rate. For adaptive selection, we develop an algorithm inspired by the derivation of the information-theoretical error lower bounds.
  arXiv  Detail & Related papers  (2019-10-14T09:18:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.