Why is Normalization Necessary for Linear Recommenders?

• URL: http://arxiv.org/abs/2504.05805v2
• Date: Mon, 28 Apr 2025 04:42:17 GMT
• Title: Why is Normalization Necessary for Linear Recommenders?
• Authors: Seongmin Park, Mincheol Yoon, Hye-young Kim, Jongwuk Lee,
• Abstract summary: We propose a versatile normalization solution, called Data- Normalization (DAN), which flexibly controls the popularity and neighborhood biases.<n> Experimental results show that DAN-equipped LAEs consistently improve existing LAE-based models across six benchmark datasets.
• Score: 10.843794863154391
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Despite their simplicity, linear autoencoder (LAE)-based models have shown comparable or even better performance with faster inference speed than neural recommender models. However, LAEs face two critical challenges: (i) popularity bias, which tends to recommend popular items, and (ii) neighborhood bias, which overly focuses on capturing local item correlations. To address these issues, this paper first analyzes the effect of two existing normalization methods for LAEs, i.e., random-walk and symmetric normalization. Our theoretical analysis reveals that normalization highly affects the degree of popularity and neighborhood biases among items. Inspired by this analysis, we propose a versatile normalization solution, called Data-Adaptive Normalization (DAN), which flexibly controls the popularity and neighborhood biases by adjusting item- and user-side normalization to align with unique dataset characteristics. Owing to its model-agnostic property, DAN can be easily applied to various LAE-based models. Experimental results show that DAN-equipped LAEs consistently improve existing LAE-based models across six benchmark datasets, with significant gains of up to 128.57% and 12.36% for long-tail items and unbiased evaluations, respectively. Refer to our code in https://github.com/psm1206/DAN.

Related papers

• Rethinking Relation Extraction: Beyond Shortcuts to Generalization with a Debiased Benchmark [53.876493664396506]
  Benchmarks are crucial for evaluating machine learning algorithm performance, facilitating comparison and identifying superior solutions.<n>This paper addresses the issue of entity bias in relation extraction tasks, where models tend to rely on entity mentions rather than context.<n>We propose a debiased relation extraction benchmark DREB that breaks the pseudo-correlation between entity mentions and relation types through entity replacement.<n>To establish a new baseline on DREB, we introduce MixDebias, a debiasing method combining data-level and model training-level techniques.
  arXiv  Detail & Related papers  (2025-01-02T17:01:06Z)
• Few-shot Steerable Alignment: Adapting Rewards and LLM Policies with Neural Processes [50.544186914115045]
  Large language models (LLMs) are increasingly embedded in everyday applications. Ensuring their alignment with the diverse preferences of individual users has become a critical challenge. We present a novel framework for few-shot steerable alignment.
  arXiv  Detail & Related papers  (2024-12-18T16:14:59Z)
• RAZOR: Sharpening Knowledge by Cutting Bias with Unsupervised Text Rewriting [16.633948320306832]
  biases prevalent in manually constructed datasets can introduce spurious correlations between tokens and labels.<n>Existing debiasing methods often rely on prior knowledge of specific dataset biases.<n>We propose RAZOR, a novel, unsupervised, and data-focused debiasing approach based on text rewriting for shortcut mitigation.
  arXiv  Detail & Related papers  (2024-12-10T17:02:58Z)
• Aligning Large Language Models via Fine-grained Supervision [20.35000061196631]
  Pre-trained large-scale language models (LLMs) excel at producing coherent articles, yet their outputs may be untruthful, toxic, or fail to align with user expectations. Current approaches focus on using reinforcement learning with human feedback to improve model alignment. We propose a method to enhance LLM alignment through fine-grained token-level supervision.
  arXiv  Detail & Related papers  (2024-06-04T20:21:45Z)
• Preference Learning Algorithms Do Not Learn Preference Rankings [62.335733662381884]
  We study the conventional wisdom that preference learning trains models to assign higher likelihoods to more preferred outputs than less preferred outputs. We find that most state-of-the-art preference-tuned models achieve a ranking accuracy of less than 60% on common preference datasets.
  arXiv  Detail & Related papers  (2024-05-29T21:29:44Z)
• Self-Exploring Language Models: Active Preference Elicitation for Online Alignment [88.56809269990625]
  We propose a bilevel objective optimistically biased towards potentially high-reward responses to actively explore out-of-distribution regions. Our experimental results demonstrate that when fine-tuned on Zephyr-7B-SFT and Llama-3-8B-Instruct models, Self-Exploring Language Models (SELM) significantly boosts the performance on instruction-following benchmarks.
  arXiv  Detail & Related papers  (2024-05-29T17:59:07Z)
• Going Beyond Popularity and Positivity Bias: Correcting for Multifactorial Bias in Recommender Systems [74.47680026838128]
  Two typical forms of bias in user interaction data with recommender systems (RSs) are popularity bias and positivity bias. We consider multifactorial selection bias affected by both item and rating value factors. We propose smoothing and alternating gradient descent techniques to reduce variance and improve the robustness of its optimization.
  arXiv  Detail & Related papers  (2024-04-29T12:18:21Z)
• Countering Mainstream Bias via End-to-End Adaptive Local Learning [17.810760161534247]
  Collaborative filtering (CF) based recommendations suffer from mainstream bias. We propose a novel end-To-end Adaptive Local Learning framework to provide high-quality recommendations to both mainstream and niche users.
  arXiv  Detail & Related papers  (2024-04-13T03:17:33Z)
• IBADR: an Iterative Bias-Aware Dataset Refinement Framework for Debiasing NLU models [52.03761198830643]
  We propose IBADR, an Iterative Bias-Aware dataset Refinement framework. We first train a shallow model to quantify the bias degree of samples in the pool. Then, we pair each sample with a bias indicator representing its bias degree, and use these extended samples to train a sample generator. In this way, this generator can effectively learn the correspondence relationship between bias indicators and samples.
  arXiv  Detail & Related papers  (2023-11-01T04:50:38Z)
• Bilateral Self-unbiased Learning from Biased Implicit Feedback [10.690479112143658]
  We propose a novel unbiased recommender learning model, namely BIlateral SElf-unbiased Recommender (BISER) BISER consists of two key components: (i) self-inverse propensity weighting (SIPW) to gradually mitigate the bias of items without incurring high computational costs; and (ii) bilateral unbiased learning (BU) to bridge the gap between two complementary models in model predictions. Extensive experiments show that BISER consistently outperforms state-of-the-art unbiased recommender models over several datasets.
  arXiv  Detail & Related papers  (2022-07-26T05:17:42Z)
• General Greedy De-bias Learning [163.65789778416172]
  We propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model like gradient descent in functional space. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
  arXiv  Detail & Related papers  (2021-12-20T14:47:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.