RADio -- Rank-Aware Divergence Metrics to Measure Normative Diversity in
News Recommendations
- URL: http://arxiv.org/abs/2209.13520v1
- Date: Sat, 17 Sep 2022 07:33:05 GMT
- Title: RADio -- Rank-Aware Divergence Metrics to Measure Normative Diversity in
News Recommendations
- Authors: Sanne Vrijenhoek, Gabriel B\'en\'edict, Mateo Gutierrez Granada, Daan
Odijk, Maarten de Rijke
- Abstract summary: In traditional recommender system literature, diversity is often seen as the opposite of similarity, and typically defined as the distance between identified topics, categories or word models.
We introduce RADio, a versatile metrics framework to evaluate recommendations according to news organization's norms and values.
We find that RADio provides insightful estimates that can potentially be used to inform news recommender system design.
- Score: 42.3075454542449
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: In traditional recommender system literature, diversity is often seen as the
opposite of similarity, and typically defined as the distance between
identified topics, categories or word models. However, this is not expressive
of the social science's interpretation of diversity, which accounts for a news
organization's norms and values and which we here refer to as normative
diversity. We introduce RADio, a versatile metrics framework to evaluate
recommendations according to these normative goals. RADio introduces a
rank-aware Jensen Shannon (JS) divergence. This combination accounts for (i) a
user's decreasing propensity to observe items further down a list and (ii) full
distributional shifts as opposed to point estimates. We evaluate RADio's
ability to reflect five normative concepts in news recommendations on the
Microsoft News Dataset and six (neural) recommendation algorithms, with the
help of our metadata enrichment pipeline. We find that RADio provides
insightful estimates that can potentially be used to inform news recommender
system design.
Related papers
- Knowledge Graph Context-Enhanced Diversified Recommendation [53.3142545812349]
This research explores the realm of diversified RecSys within the intricate context of knowledge graphs (KG)
Our contributions include introducing an innovative metric, Entity Coverage, and Relation Coverage, which effectively quantifies diversity within the KG domain.
In tandem with this, we introduce a novel technique named Conditional Alignment and Uniformity (CAU) which encodes KG item embeddings while preserving contextual integrity.
arXiv Detail & Related papers (2023-10-20T03:18:57Z) - Ordinal Graph Gamma Belief Network for Social Recommender Systems [54.9487910312535]
We develop a hierarchical Bayesian model termed ordinal graph factor analysis (OGFA), which jointly models user-item and user-user interactions.
OGFA not only achieves good recommendation performance, but also extracts interpretable latent factors corresponding to representative user preferences.
We extend OGFA to ordinal graph gamma belief network, which is a multi-stochastic-layer deep probabilistic model.
arXiv Detail & Related papers (2022-09-12T09:19:22Z) - A Closer Look at Debiased Temporal Sentence Grounding in Videos:
Dataset, Metric, and Approach [53.727460222955266]
Temporal Sentence Grounding in Videos (TSGV) aims to ground a natural language sentence in an untrimmed video.
Recent studies have found that current benchmark datasets may have obvious moment annotation biases.
We introduce a new evaluation metric "dR@n,IoU@m" that discounts the basic recall scores to alleviate the inflating evaluation caused by biased datasets.
arXiv Detail & Related papers (2022-03-10T08:58:18Z) - Choosing the Best of Both Worlds: Diverse and Novel Recommendations
through Multi-Objective Reinforcement Learning [68.45370492516531]
We introduce Scalarized Multi-Objective Reinforcement Learning (SMORL) for the Recommender Systems (RS) setting.
SMORL agent augments standard recommendation models with additional RL layers that enforce it to simultaneously satisfy three principal objectives: accuracy, diversity, and novelty of recommendations.
Our experimental results on two real-world datasets reveal a substantial increase in aggregate diversity, a moderate increase in accuracy, reduced repetitiveness of recommendations, and demonstrate the importance of reinforcing diversity and novelty as complementary objectives.
arXiv Detail & Related papers (2021-10-28T13:22:45Z) - D2RLIR : an improved and diversified ranking function in interactive
recommendation systems based on deep reinforcement learning [0.3058685580689604]
This paper proposes a deep reinforcement learning based recommendation system by utilizing Actor-Critic architecture.
The proposed model is able to generate a diverse while relevance recommendation list based on the user's preferences.
arXiv Detail & Related papers (2021-10-28T13:11:29Z) - SERAB: A multi-lingual benchmark for speech emotion recognition [12.579936838293387]
Recent developments in speech emotion recognition (SER) often leverage deep neural networks (DNNs)
We present the Speech Emotion Recognition Adaptation Benchmark (SERAB), a framework for evaluating the performance and generalization capacity of different approaches for utterance-level SER.
arXiv Detail & Related papers (2021-10-07T13:01:34Z) - Random Walks with Erasure: Diversifying Personalized Recommendations on
Social and Information Networks [4.007832851105161]
We develop a novel recommendation framework with a goal of improving information diversity using a modified random walk exploration of the user-item graph.
For recommending political content on social networks, we first propose a new model to estimate the ideological positions for both users and the content they share.
Based on these estimated positions, we generate diversified personalized recommendations using our new random-walk based recommendation algorithm.
arXiv Detail & Related papers (2021-02-18T21:53:32Z) - Operationalizing Framing to Support Multiperspective Recommendations of
Opinion Pieces [1.3286165491120467]
We operationalize the notion of framing, adopted from communication science.
We apply this notion to a re-ranking of topic-relevant recommended lists.
Our offline evaluation indicates that the proposed method is capable of enhancing the viewpoint diversity of recommendation lists.
arXiv Detail & Related papers (2021-01-15T14:40:34Z) - Simultaneous Relevance and Diversity: A New Recommendation Inference
Approach [81.44167398308979]
We propose a new approach, which extends the general collaborative filtering (CF) by introducing a new way of CF inference, negative-to-positive.
Our approach is applicable to a wide range of recommendation scenarios/use-cases at various sophistication levels.
Our analysis and experiments on public datasets and real-world production data show that our approach outperforms existing methods on relevance and diversity simultaneously.
arXiv Detail & Related papers (2020-09-27T22:20:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.