RADio -- Rank-Aware Divergence Metrics to Measure Normative Diversity in News Recommendations

• URL: http://arxiv.org/abs/2209.13520v1
• Date: Sat, 17 Sep 2022 07:33:05 GMT
• Title: RADio -- Rank-Aware Divergence Metrics to Measure Normative Diversity in News Recommendations
• Authors: Sanne Vrijenhoek, Gabriel B\'en\'edict, Mateo Gutierrez Granada, Daan Odijk, Maarten de Rijke
• Abstract summary: In traditional recommender system literature, diversity is often seen as the opposite of similarity, and typically defined as the distance between identified topics, categories or word models. We introduce RADio, a versatile metrics framework to evaluate recommendations according to news organization's norms and values. We find that RADio provides insightful estimates that can potentially be used to inform news recommender system design.
• Score: 42.3075454542449
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In traditional recommender system literature, diversity is often seen as the opposite of similarity, and typically defined as the distance between identified topics, categories or word models. However, this is not expressive of the social science's interpretation of diversity, which accounts for a news organization's norms and values and which we here refer to as normative diversity. We introduce RADio, a versatile metrics framework to evaluate recommendations according to these normative goals. RADio introduces a rank-aware Jensen Shannon (JS) divergence. This combination accounts for (i) a user's decreasing propensity to observe items further down a list and (ii) full distributional shifts as opposed to point estimates. We evaluate RADio's ability to reflect five normative concepts in news recommendations on the Microsoft News Dataset and six (neural) recommendation algorithms, with the help of our metadata enrichment pipeline. We find that RADio provides insightful estimates that can potentially be used to inform news recommender system design.

Related papers

• Bayesian-Guided Diversity in Sequential Sampling for Recommender Systems [1.675857332621569]
  We propose a novel framework that leverages a multi-objective, contextual sequential sampling strategy.<n>Item selection is guided by Bayesian updates that dynamically adjust scores to optimize diversity.<n> Experiments on a real-world dataset show that our approach significantly improves diversity without sacrificing relevance.
  arXiv  Detail & Related papers  (2025-06-22T19:36:02Z)
• Evaluating the Evaluation of Diversity in Commonsense Generation [28.654890118684957]
  We conduct a systematic meta-evaluation of diversity metrics for commonsense generation.<n>We find that form-based diversity metrics tend to consistently overestimate the diversity in sentence sets.<n>We show that content-based diversity evaluation metrics consistently outperform the form-based counterparts.
  arXiv  Detail & Related papers  (2025-05-31T11:18:26Z)
• Scalable Ensemble Diversification for OOD Generalization and Detection [68.8982448081223]
  SED identifies hard training samples on the fly and encourages the ensemble members to disagree on these. We show how to avoid the expensive computations in existing methods of exhaustive pairwise disagreements across models. For OOD generalization, we observe large benefits from the diversification in multiple settings including output-space (classical) ensembles and weight-space ensembles (model soups)
  arXiv  Detail & Related papers  (2024-09-25T10:30:24Z)
• Relevance meets Diversity: A User-Centric Framework for Knowledge Exploration through Recommendations [15.143224593682012]
  We propose a novel recommendation strategy that combines relevance and diversity by a copula function. We use diversity as a surrogate of the amount of knowledge obtained by the user while interacting with the system. Our strategy outperforms several state-of-the-art competitors.
  arXiv  Detail & Related papers  (2024-08-07T13:48:24Z)
• Knowledge Graph Context-Enhanced Diversified Recommendation [53.3142545812349]
  This research explores the realm of diversified RecSys within the intricate context of knowledge graphs (KG) Our contributions include introducing an innovative metric, Entity Coverage, and Relation Coverage, which effectively quantifies diversity within the KG domain. In tandem with this, we introduce a novel technique named Conditional Alignment and Uniformity (CAU) which encodes KG item embeddings while preserving contextual integrity.
  arXiv  Detail & Related papers  (2023-10-20T03:18:57Z)
• Ordinal Graph Gamma Belief Network for Social Recommender Systems [54.9487910312535]
  We develop a hierarchical Bayesian model termed ordinal graph factor analysis (OGFA), which jointly models user-item and user-user interactions. OGFA not only achieves good recommendation performance, but also extracts interpretable latent factors corresponding to representative user preferences. We extend OGFA to ordinal graph gamma belief network, which is a multi-stochastic-layer deep probabilistic model.
  arXiv  Detail & Related papers  (2022-09-12T09:19:22Z)
• A Closer Look at Debiased Temporal Sentence Grounding in Videos: Dataset, Metric, and Approach [53.727460222955266]
  Temporal Sentence Grounding in Videos (TSGV) aims to ground a natural language sentence in an untrimmed video. Recent studies have found that current benchmark datasets may have obvious moment annotation biases. We introduce a new evaluation metric "dR@n,IoU@m" that discounts the basic recall scores to alleviate the inflating evaluation caused by biased datasets.
  arXiv  Detail & Related papers  (2022-03-10T08:58:18Z)
• Choosing the Best of Both Worlds: Diverse and Novel Recommendations through Multi-Objective Reinforcement Learning [68.45370492516531]
  We introduce Scalarized Multi-Objective Reinforcement Learning (SMORL) for the Recommender Systems (RS) setting. SMORL agent augments standard recommendation models with additional RL layers that enforce it to simultaneously satisfy three principal objectives: accuracy, diversity, and novelty of recommendations. Our experimental results on two real-world datasets reveal a substantial increase in aggregate diversity, a moderate increase in accuracy, reduced repetitiveness of recommendations, and demonstrate the importance of reinforcing diversity and novelty as complementary objectives.
  arXiv  Detail & Related papers  (2021-10-28T13:22:45Z)
• SERAB: A multi-lingual benchmark for speech emotion recognition [12.579936838293387]
  Recent developments in speech emotion recognition (SER) often leverage deep neural networks (DNNs) We present the Speech Emotion Recognition Adaptation Benchmark (SERAB), a framework for evaluating the performance and generalization capacity of different approaches for utterance-level SER.
  arXiv  Detail & Related papers  (2021-10-07T13:01:34Z)
• Random Walks with Erasure: Diversifying Personalized Recommendations on Social and Information Networks [4.007832851105161]
  We develop a novel recommendation framework with a goal of improving information diversity using a modified random walk exploration of the user-item graph. For recommending political content on social networks, we first propose a new model to estimate the ideological positions for both users and the content they share. Based on these estimated positions, we generate diversified personalized recommendations using our new random-walk based recommendation algorithm.
  arXiv  Detail & Related papers  (2021-02-18T21:53:32Z)
• Operationalizing Framing to Support Multiperspective Recommendations of Opinion Pieces [1.3286165491120467]
  We operationalize the notion of framing, adopted from communication science. We apply this notion to a re-ranking of topic-relevant recommended lists. Our offline evaluation indicates that the proposed method is capable of enhancing the viewpoint diversity of recommendation lists.
  arXiv  Detail & Related papers  (2021-01-15T14:40:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.