Towards Group-aware Search Success

• URL: http://arxiv.org/abs/2404.17313v2
• Date: Sun, 23 Jun 2024 03:13:57 GMT
• Title: Towards Group-aware Search Success
• Authors: Haolun Wu, Bhaskar Mitra, Nick Craswell,
• Abstract summary: We introduce a novel metric, named Group-aware Search Success (GA-SS) GA-SS redefines search success to ensure that all demographic groups achieve satisfaction from search outcomes. We empirically validate our metric and approach with two real-world datasets.
• Score: 12.281168800322458
• License:
• Abstract: Traditional measures of search success often overlook the varying information needs of different demographic groups. To address this gap, we introduce a novel metric, named Group-aware Search Success (GA-SS). GA-SS redefines search success to ensure that all demographic groups achieve satisfaction from search outcomes. We introduce a comprehensive mathematical framework to calculate GA-SS, incorporating both static and stochastic ranking policies and integrating user browsing models for a more accurate assessment. In addition, we have proposed Group-aware Most Popular Completion (gMPC) ranking model to account for demographic variances in user intent, aligning more closely with the diverse needs of all user groups. We empirically validate our metric and approach with two real-world datasets: one focusing on query auto-completion and the other on movie recommendations, where the results highlight the impact of stochasticity and the complex interplay among various search success metrics. Our findings advocate for a more inclusive approach in measuring search success, as well as inspiring future investigations into the quality of service of search.

Related papers

• Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximization [21.115495457454365]
  This paper investigates the design of a unified search engine to serve multiple retrieval-augmented generation (RAG) agents. We introduce an iterative approach where the search engine generates retrieval results for these RAG agents and gathers feedback on the quality of the retrieved documents during an offline phase. We adapt this approach to an online setting, allowing the search engine to refine its behavior based on real-time individual agents feedback.
  arXiv  Detail & Related papers  (2024-10-13T17:53:50Z)
• DiscoveryBench: Towards Data-Driven Discovery with Large Language Models [50.36636396660163]
  We present DiscoveryBench, the first comprehensive benchmark that formalizes the multi-step process of data-driven discovery. Our benchmark contains 264 tasks collected across 6 diverse domains, such as sociology and engineering. Our benchmark, thus, illustrates the challenges in autonomous data-driven discovery and serves as a valuable resource for the community to make progress.
  arXiv  Detail & Related papers  (2024-07-01T18:58:22Z)
• Evaluating Ensemble Methods for News Recommender Systems [50.90330146667386]
  This paper demonstrates how ensemble methods can be used to combine many diverse state-of-the-art algorithms to achieve superior results on the Microsoft News dataset (MIND) Our findings demonstrate that a combination of NRS algorithms can outperform individual algorithms, provided that the base learners are sufficiently diverse.
  arXiv  Detail & Related papers  (2024-06-23T13:40:50Z)
• A structured regression approach for evaluating model performance across intersectional subgroups [53.91682617836498]
  Disaggregated evaluation is a central task in AI fairness assessment, where the goal is to measure an AI system's performance across different subgroups. We introduce a structured regression approach to disaggregated evaluation that we demonstrate can yield reliable system performance estimates even for very small subgroups.
  arXiv  Detail & Related papers  (2024-01-26T14:21:45Z)
• Overview of the TREC 2023 Product Product Search Track [70.56592126043546]
  This is the first year of the TREC Product search track. The focus was the creation of a reusable collection. We leverage the new product search corpus, which includes contextual metadata.
  arXiv  Detail & Related papers  (2023-11-14T02:25:18Z)
• Que2Engage: Embedding-based Retrieval for Relevant and Engaging Products at Facebook Marketplace [15.054431410052851]
  We present Que2Engage, a search EBR system built towards bridging the gap between retrieval and ranking for end-to-end optimizations. We show the effectiveness of our approach via a multitask evaluation framework and thorough baseline comparisons and ablation studies.
  arXiv  Detail & Related papers  (2023-02-21T23:10:16Z)
• Ranking-based Group Identification via Factorized Attention on Social Tripartite Graph [68.08590487960475]
  We propose a novel GNN-based framework named Contextualized Factorized Attention for Group identification (CFAG) We devise tripartite graph convolution layers to aggregate information from different types of neighborhoods among users, groups, and items. To cope with the data sparsity issue, we devise a novel propagation augmentation layer, which is based on our proposed factorized attention mechanism.
  arXiv  Detail & Related papers  (2022-11-02T01:42:20Z)
• Keyword Targeting Optimization in Sponsored Search Advertising: Combining Selection and Matching [0.0]
  An optimal keyword targeting strategy guarantees reaching the right population effectively. This paper aims to address the keyword targeting problem, which is a challenging task because of the incomplete information of historical advertising performance indices. Experimental results show that, (a) BB-KSM outperforms seven baselines in terms of profit; (b) BB-KSM shows its superiority as the budget increases.
  arXiv  Detail & Related papers  (2022-10-19T03:37:32Z)
• Advertising Media and Target Audience Optimization via High-dimensional Bandits [2.5137859989323537]
  We present a data-driven algorithm that advertisers can use to automate their digital ad-campaigns at online publishers. The algorithm enables the advertiser to search across available target audiences and ad-media to find the best possible combination for its campaign via online experimentation.
  arXiv  Detail & Related papers  (2022-09-17T21:00:53Z)
• Semi-Supervised Domain Generalization with Stochastic StyleMatch [90.98288822165482]
  In real-world applications, we might have only a few labels available from each source domain due to high annotation cost. In this work, we investigate semi-supervised domain generalization, a more realistic and practical setting. Our proposed approach, StyleMatch, is inspired by FixMatch, a state-of-the-art semi-supervised learning method based on pseudo-labeling.
  arXiv  Detail & Related papers  (2021-06-01T16:00:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.