Counterfactual Learning To Rank for Utility-Maximizing Query Autocompletion

• URL: http://arxiv.org/abs/2204.10936v1
• Date: Fri, 22 Apr 2022 21:40:51 GMT
• Title: Counterfactual Learning To Rank for Utility-Maximizing Query Autocompletion
• Authors: Adam Block, Rahul Kidambi, Daniel N. Hill, Thorsten Joachims, and Inderjit S. Dhillon
• Abstract summary: We propose a new approach that explicitly optimize the query suggestions for downstream retrieval performance. We formulate this as a problem of ranking a set of rankings, where each query suggestion is represented by the downstream item ranking it produces. We then present a learning method that ranks query suggestions by the quality of their item rankings.
• Score: 40.31426350180036
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conventional methods for query autocompletion aim to predict which completed query a user will select from a list. A shortcoming of this approach is that users often do not know which query will provide the best retrieval performance on the current information retrieval system, meaning that any query autocompletion methods trained to mimic user behavior can lead to suboptimal query suggestions. To overcome this limitation, we propose a new approach that explicitly optimizes the query suggestions for downstream retrieval performance. We formulate this as a problem of ranking a set of rankings, where each query suggestion is represented by the downstream item ranking it produces. We then present a learning method that ranks query suggestions by the quality of their item rankings. The algorithm is based on a counterfactual learning approach that is able to leverage feedback on the items (e.g., clicks, purchases) to evaluate query suggestions through an unbiased estimator, thus avoiding the assumption that users write or select optimal queries. We establish theoretical support for the proposed approach and provide learning-theoretic guarantees. We also present empirical results on publicly available datasets, and demonstrate real-world applicability using data from an online shopping store.

Related papers

• ExcluIR: Exclusionary Neural Information Retrieval [74.08276741093317]
  We present ExcluIR, a set of resources for exclusionary retrieval. evaluation benchmark includes 3,452 high-quality exclusionary queries. training set contains 70,293 exclusionary queries, each paired with a positive document and a negative document.
  arXiv  Detail & Related papers  (2024-04-26T09:43:40Z)
• The Surprising Effectiveness of Rankers Trained on Expanded Queries [4.874071145951159]
  We improve the ranking performance of hard or difficult queries without compromising the performance of other queries. We combine relevance scores from the specialized ranker and the base ranker, along with a query performance score estimated for each query. In our experiments on the DL-Hard dataset, we find that a principled query performance based scoring method offers a significant improvement of up to 25% on the passage ranking task.
  arXiv  Detail & Related papers  (2024-04-03T09:12:22Z)
• Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL [62.824464372594576]
  We aim to enhance arithmetic reasoning ability of Large Language Models (LLMs) through zero-shot prompt optimization. We identify a previously overlooked objective of query dependency in such optimization. We introduce Prompt-OIRL, which harnesses offline inverse reinforcement learning to draw insights from offline prompting demonstration data.
  arXiv  Detail & Related papers  (2023-09-13T01:12:52Z)
• Improving Sequential Query Recommendation with Immediate User Feedback [6.925738064847176]
  We propose an algorithm for next query recommendation in interactive data exploration settings. We conduct a large-scale experimental study using log files from a popular online literature discovery service.
  arXiv  Detail & Related papers  (2022-05-12T18:19:24Z)
• Online Learning of Optimally Diverse Rankings [63.62764375279861]
  We propose an algorithm that efficiently learns the optimal list based on users' feedback only. We show that after $T$ queries, the regret of LDR scales as $O((N-L)log(T))$ where $N$ is the number of all items.
  arXiv  Detail & Related papers  (2021-09-13T12:13:20Z)
• Conditional Sequential Slate Optimization [15.10459152219771]
  A search ranking system typically orders the results by independent query-document scores to produce a slate of search results. We introduce conditional sequential slate optimization (CSSO), which jointly learns to optimize for traditional ranking metrics as well as prescribed distribution criteria of documents within the slate. The proposed method can be applied to practical real world problems such as enforcing diversity in e-commerce search results, mitigating bias in top results and personalization of results.
  arXiv  Detail & Related papers  (2021-08-12T09:14:46Z)
• High Quality Related Search Query Suggestions using Deep Reinforcement Learning [0.15229257192293202]
  "High Quality Related Search Query Suggestions" task aims at recommending search queries which are real, accurate, diverse, relevant and engaging. We train a Deep Reinforcement Learning model to predict the query a user would enter next. The reward signal is composed of long-term session-based user feedback, syntactic relatedness and estimated naturalness of generated query.
  arXiv  Detail & Related papers  (2021-08-10T05:22:32Z)
• Session-Aware Query Auto-completion using Extreme Multi-label Ranking [61.753713147852125]
  We take the novel approach of modeling session-aware query auto-completion as an e Multi-Xtreme Ranking (XMR) problem. We adapt a popular XMR algorithm for this purpose by proposing several modifications to the key steps in the algorithm. Our approach meets the stringent latency requirements for auto-complete systems while leveraging session information in making suggestions.
  arXiv  Detail & Related papers  (2020-12-09T17:56:22Z)
• Optimal Clustering from Noisy Binary Feedback [75.17453757892152]
  We study the problem of clustering a set of items from binary user feedback. We devise an algorithm with a minimal cluster recovery error rate. For adaptive selection, we develop an algorithm inspired by the derivation of the information-theoretical error lower bounds.
  arXiv  Detail & Related papers  (2019-10-14T09:18:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.