Revisiting N2DCG: An Empirically Grounded Reformulation of Carousel Recommendation Evaluation

• URL: http://arxiv.org/abs/2608.21877v1
• Date: Sat, 22 Aug 2026 09:40:10 GMT
• Title: Revisiting N2DCG: An Empirically Grounded Reformulation of Carousel Recommendation Evaluation
• Abstract summary: We propose a reformulation of N2DCG that normalizes appropriately by respecting constraints and uses an empirically grounded discount function.<n>We validate the proposed metric, showing that it better reflects users' empirical behavior on real-world eye-tracking data.
• Score: 56.85622784396052
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Carousel interfaces have been widely used in video and music streaming services, yet it remains unclear how to properly evaluate recommender systems in these two-dimensional layouts. N2DCG has been proposed to address this gap by adapting NDCG to carousel-based recommendation, but it relies on unverified assumptions borrowed from the single-list web-search setting that do not transfer well to two-dimensional carousel layouts. We identify two substantial limitations of N2DCG: its ideal ranking, used for normalization, violates carousel constraints, and its discount function does not reflect user browsing behavior observed in empirical data. To address both limitations, we propose a reformulation of N2DCG that normalizes appropriately by respecting constraints and uses an empirically grounded discount function. We validate the proposed metric, showing that it better reflects users' empirical behavior on real-world eye-tracking data and better predicts the comparison results of carousel layouts simulated based on empirical examination patterns.

Related papers

• Verifiable Reasoning for LLM-based Generative Recommendation [106.7765000777685]
  Reasoning in Large Language Models (LLMs) has recently shown strong potential in enhancing generative recommendation.<n>We propose a novel textbftextitreason-verify-recommend paradigm, which interleaves reasoning with verification to provide reliable feedback.<n>Experiments on four real-world datasets demonstrate that VRec substantially enhances recommendation effectiveness and scalability without compromising efficiency.
  arXiv  Detail & Related papers  (2026-03-08T16:55:45Z)
• Embed Progressive Implicit Preference in Unified Space for Deep Collaborative Filtering [13.24227546548424]
  Generalized Neural Ordinal Logistic Regression (GNOLR) is proposed to capture the structured progression of user engagement.<n>GNOLR enhances predictive accuracy, captures the progression of user engagement, and simplifies the retrieval process.<n>Experiments on ten real-world datasets show that GNOLR significantly outperforms state-of-the-art methods in efficiency and adaptability.
  arXiv  Detail & Related papers  (2025-05-27T08:43:35Z)
• A Reproducibility Study of PLAID [25.86500025007641]
  We compare PLAID with an important baseline missing from the paper: re-ranking a lexical system. We find that applying ColBERTv2 as a re-ranker atop an initial pool of BM25 results provides better efficiency-effectiveness trade-offs in low-latency settings. We find that recently proposed modifications to re-ranking that pull in the neighbors of top-scoring documents overcome this limitation.
  arXiv  Detail & Related papers  (2024-04-23T12:46:53Z)
• Aligning GPTRec with Beyond-Accuracy Goals with Reinforcement Learning [67.71952251641545]
  GPTRec is an alternative to the Top-K model for item-by-item recommendations. We show that GPTRec offers a better tradeoff between accuracy and secondary metrics than classic greedy re-ranking techniques. Our experiments on two datasets show that GPTRec's Next-K generation approach offers a better tradeoff between accuracy and secondary metrics than classic greedy re-ranking techniques.
  arXiv  Detail & Related papers  (2024-03-07T19:47:48Z)
• Sequential Recommendation with Controllable Diversification: Representation Degeneration and Diversity [59.24517649169952]
  We argue that the representation degeneration issue is the root cause of insufficient recommendation diversity in existing SR methods. We propose a novel Singular sPectrum sMoothing regularization for Recommendation (SPMRec), which acts as a controllable surrogate to alleviate the degeneration.
  arXiv  Detail & Related papers  (2023-06-21T02:42:37Z)
• A Methodology for the Offline Evaluation of Recommender Systems in a User Interface with Multiple Carousels [7.8851236034886645]
  Video-on-demand and music streaming services provide the user with a page consisting of several recommendation lists. Finding efficient strategies to select which carousels to display is an active research topic of great industrial interest. We propose an offline evaluation protocol for a carousel setting in which the recommendation quality of a model is measured by how much it improves upon that of an already available set of carousels.
  arXiv  Detail & Related papers  (2021-05-13T13:14:59Z)
• Canny-VO: Visual Odometry with RGB-D Cameras based on Geometric 3D-2D Edge Alignment [85.32080531133799]
  This paper reviews the classical problem of free-form curve registration and applies it to an efficient RGBD visual odometry system called Canny-VO. Two replacements for the distance transformation commonly used in edge registration are proposed: Approximate Nearest Neighbour Fields and Oriented Nearest Neighbour Fields. 3D2D edge alignment benefits from these alternative formulations in terms of both efficiency and accuracy.
  arXiv  Detail & Related papers  (2020-12-15T11:42:17Z)
• CRACT: Cascaded Regression-Align-Classification for Robust Visual Tracking [97.84109669027225]
  We introduce an improved proposal refinement module, Cascaded Regression-Align- Classification (CRAC) CRAC yields new state-of-the-art performances on many benchmarks. In experiments on seven benchmarks including OTB-2015, UAV123, NfS, VOT-2018, TrackingNet, GOT-10k and LaSOT, our CRACT exhibits very promising results in comparison with state-of-the-art competitors.
  arXiv  Detail & Related papers  (2020-11-25T02:18:33Z)
• Convolutional Gaussian Embeddings for Personalized Recommendation with Uncertainty [17.258674767363345]
  Most existing embedding based recommendation models use embeddings corresponding to a single fixed point in low-dimensional space. We propose a unified deep recommendation framework employing Gaussian embeddings, which are proven adaptive to uncertain preferences. Our framework adopts Monte-Carlo sampling and convolutional neural networks to compute the correlation between the objective user and the candidate item.
  arXiv  Detail & Related papers  (2020-06-19T02:10:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.