FAIR-QR: Enhancing Fairness-aware Information Retrieval through Query Refinement

• URL: http://arxiv.org/abs/2503.21092v1
• Date: Thu, 27 Mar 2025 02:10:19 GMT
• Title: FAIR-QR: Enhancing Fairness-aware Information Retrieval through Query Refinement
• Authors: Fumian Chen, Hui Fang,
• Abstract summary: We propose a novel framework that refines query keywords to retrieve documents from underrepresented groups and achieve group fairness.<n>Our method not only shows promising retrieval results regarding relevance and fairness but also interpretability by showing refined keywords used at each iteration.
• Score: 1.8577028544235155
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Information retrieval systems such as open web search and recommendation systems are ubiquitous and significantly impact how people receive and consume online information. Previous research has shown the importance of fairness in information retrieval systems to combat the issue of echo chambers and mitigate the rich-get-richer effect. Therefore, various fairness-aware information retrieval methods have been proposed. Score-based fairness-aware information retrieval algorithms, focusing on statistical parity, are interpretable but could be mathematically infeasible and lack generalizability. In contrast, learning-to-rank-based fairness-aware information retrieval algorithms using fairness-aware loss functions demonstrate strong performance but lack interpretability. In this study, we proposed a novel and interpretable framework that recursively refines query keywords to retrieve documents from underrepresented groups and achieve group fairness. Retrieved documents using refined queries will be re-ranked to ensure relevance. Our method not only shows promising retrieval results regarding relevance and fairness but also preserves interpretability by showing refined keywords used at each iteration.

Related papers

• Improving RAG Retrieval via Propositional Content Extraction: a Speech Act Theory Approach [0.0]
  This paper investigates whether extracting the underlying propositional content from user utterances can improve retrieval quality in Retrieval-Augmented Generation systems.<n>We propose a practical method for automatically transforming queries into their propositional equivalents before embedding.
  arXiv  Detail & Related papers  (2025-03-07T20:15:40Z)
• Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generation [53.285436927963865]
  We present the first comprehensive study of RAG systems that incorporate fairness-aware rankings.<n>We find that fairness-aware retrieval frequently retains or even improves ranking effectiveness and generation quality.<n>Our results underscore the importance of item-side fairness throughout both retrieval and generation phases.
  arXiv  Detail & Related papers  (2024-09-17T23:10:04Z)
• Beyond Relevance: Evaluate and Improve Retrievers on Perspective Awareness [56.42192735214931]
  retrievers are expected to not only rely on the semantic relevance between the documents and the queries but also recognize the nuanced intents or perspectives behind a user query. In this work, we study whether retrievers can recognize and respond to different perspectives of the queries. We show that current retrievers have limited awareness of subtly different perspectives in queries and can also be biased toward certain perspectives.
  arXiv  Detail & Related papers  (2024-05-04T17:10:00Z)
• ExcluIR: Exclusionary Neural Information Retrieval [74.08276741093317]
  We present ExcluIR, a set of resources for exclusionary retrieval. evaluation benchmark includes 3,452 high-quality exclusionary queries. training set contains 70,293 exclusionary queries, each paired with a positive document and a negative document.
  arXiv  Detail & Related papers  (2024-04-26T09:43:40Z)
• Evaluating Generative Ad Hoc Information Retrieval [58.800799175084286]
  generative retrieval systems often directly return a grounded generated text as a response to a query. Quantifying the utility of the textual responses is essential for appropriately evaluating such generative ad hoc retrieval.
  arXiv  Detail & Related papers  (2023-11-08T14:05:00Z)
• The Role of Relevance in Fair Ranking [1.5469452301122177]
  We argue that relevance scores should satisfy a set of desired criteria in order to guide fairness interventions. We then empirically show that not all of these criteria are met in a case study of relevance inferred from biased user click data. Our analyses and results surface the pressing need for new approaches to relevance collection and generation.
  arXiv  Detail & Related papers  (2023-05-09T16:58:23Z)
• Algorithmic Fairness Datasets: the Story so Far [68.45921483094705]
  Data-driven algorithms are studied in diverse domains to support critical decisions, directly impacting people's well-being. A growing community of researchers has been investigating the equity of existing algorithms and proposing novel ones, advancing the understanding of risks and opportunities of automated decision-making for historically disadvantaged populations. Progress in fair Machine Learning hinges on data, which can be appropriately used only if adequately documented. Unfortunately, the algorithmic fairness community suffers from a collective data documentation debt caused by a lack of information on specific resources (opacity) and scatteredness of available information (sparsity)
  arXiv  Detail & Related papers  (2022-02-03T17:25:46Z)
• Exposing Query Identification for Search Transparency [69.06545074617685]
  We explore the feasibility of approximate exposing query identification (EQI) as a retrieval task by reversing the role of queries and documents in two classes of search systems. We derive an evaluation metric to measure the quality of a ranking of exposing queries, as well as conducting an empirical analysis focusing on various practical aspects of approximate EQI.
  arXiv  Detail & Related papers  (2021-10-14T20:19:27Z)
• Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback [29.719150565643965]
  This paper proposes ANCE-PRF, a new query encoder that uses pseudo relevance feedback (PRF) to improve query representations for dense retrieval. ANCE-PRF uses a BERT encoder that consumes the query and the top retrieved documents from a dense retrieval model, ANCE, and it learns to produce better query embeddings directly from relevance labels. Analysis shows that the PRF encoder effectively captures the relevant and complementary information from PRF documents, while ignoring the noise with its learned attention mechanism.
  arXiv  Detail & Related papers  (2021-08-30T18:10:26Z)
• Overview of the TREC 2019 Fair Ranking Track [65.15263872493799]
  The goal of the TREC Fair Ranking track was to develop a benchmark for evaluating retrieval systems in terms of fairness to different content providers. This paper presents an overview of the track, including the task definition, descriptions of the data and the annotation process.
  arXiv  Detail & Related papers  (2020-03-25T21:34:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.