Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation

• URL: http://arxiv.org/abs/2605.23191v1
• Date: Fri, 22 May 2026 03:17:29 GMT
• Title: Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation
• Abstract summary: Scaling recommendation models is a central challenge in recommender systems.<n>RankMixer suffers from textitembedding collapse, where learned representations have low effective rank.<n>We propose RankElastor, a novel architecture that produces spectrum-robust representations with provable collapse mitigation.
• Score: 21.08394922908602
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scaling recommendation models is a central challenge in recommender systems. Recently, RankMixer has emerged as an effective solution, operating on a unified token representation and alternating between token mixing and per-token feedforward networks (P-FFNs) to achieve scalable performance. However, RankMixer suffers from \textit{embedding collapse}, where learned representations have low effective rank, limiting expressivity and underutilizing the expanded representation space. Through empirical analysis and theoretical insights, we identify rigid token mixing and P-FFN modules as the primary causes of this phenomenon, jointly inducing a \textbf{damped oscillatory trajectory} in effective-rank evolution across layers. To address it, we propose RankElastor, a novel architecture that produces spectrum-robust representations with provable collapse mitigation. RankElastor introduces two components: (i) \textbf{parameterized full mixing}, which enables expressive token mixing with improved spectral robustness; and (ii) \textbf{GLU-improved P-FFNs}, which stabilize representation spectra through GLU-style FFN modules. Extensive experiments on large-scale industrial datasets demonstrate that RankElastor consistently improves recommendation performance, mitigates embedding collapse, and exhibits robust scaling behavior. Code is available at this GitHub repository: https://github.com/vasile-paskardlgm/RankElastor

Related papers

• Beyond Noisy Signals: Dual-Level Denoising for Multi-modal Sequential Recommendation [6.902443075506667]
  We propose textbfDual-level textbfDenoising textbfMulti-modal textbfSequential textbfRecommendation framework.
  arXiv  Detail & Related papers  (2026-07-21T07:06:06Z)
• $S^{2}$-FracMix: Label-Preserving Self-Saliency Mixup Augmentation [41.90458736034228]
  Self-Saliency ($S2$) Mixup and FracMix ($S2$-FracMix) are presented.<n>Self-Saliency constructs challenging yet label-consistent samples by extracting multi-scale salient patches.<n>FracMix injects self-similarity patterns into salient regions using adaptive ratios.
  arXiv  Detail & Related papers  (2026-06-24T13:00:39Z)
• Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation [58.33732779555114]
  We argue that the key bottleneck is a representation-level failure caused by two coupled heterogeneities.<n>We propose Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation (SpectraMB)<n>SpectraMB achieves the best results in most evaluation settings and exhibits improved robustness under noisy interactions.
  arXiv  Detail & Related papers  (2026-06-01T15:58:13Z)
• RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems [21.830994984868212]
  We propose textbfRankUp, an architecture designed to mitigate representation collapse and enhance expressive capacity.<n>RankUp has been fully deployed in large-scale production across Weixin Video Accounts, Official Accounts and Moments, yielding GMV improvements of 3.41%, 4.81% and 2.21%, respectively.
  arXiv  Detail & Related papers  (2026-04-20T06:40:27Z)
• UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systems [22.268247447161134]
  We propose a unified scaling architecture for recommendation systems, namely textbfUniMixer, to improve scaling efficiency.<n>We establish a unified scaling module design framework for recommender systems, which bridges the connections among attention-based, TokenMixer-based, and factorization-machine-based methods.
  arXiv  Detail & Related papers  (2026-04-01T07:57:40Z)
• ConsistentRFT: Reducing Visual Hallucinations in Flow-based Reinforcement Fine-Tuning [85.20505958752928]
  Reinforcement Fine-Tuning (RFT) on flow-based models is crucial for preference alignment.<n>RFT often introduce visual hallucinations like over-optimized details and semantic misalignment.<n>This work preliminarily explores why visual hallucinations arise and how to reduce them.
  arXiv  Detail & Related papers  (2026-02-03T11:49:46Z)
• DynaPURLS: Dynamic Refinement of Part-aware Representations for Skeleton-based Zero-Shot Action Recognition [51.80782323686666]
  We introduce textbfDynaPURLS, a unified framework that establishes robust, multi-scale visual-semantic correspondences.<n>Our framework leverages a large language model to generate hierarchical textual descriptions that encompass both global movements and local body-part dynamics.<n>Experiments on three large-scale benchmark datasets, including NTU RGB+D 60/120 and PKU-MMD, demonstrate that DynaPURLS significantly outperforms prior art.
  arXiv  Detail & Related papers  (2025-12-12T10:39:10Z)
• HiLoMix: Robust High- and Low-Frequency Graph Learning Framework for Mixing Address Association [4.848214568017272]
  Mixing services are increasingly being exploited by malicious actors for illicit transactions.<n>We propose HiLoMix, a graph-based learning framework specifically designed for mixing address association.<n> Experimental results demonstrate that HiLoMix outperforms existing methods in mixing address association.
  arXiv  Detail & Related papers  (2025-11-11T02:19:00Z)
• MergeMix: A Unified Augmentation Paradigm for Visual and Multi-Modal Understanding [23.96717124380285]
  MergeMix is a training-time augmentation paradigm that bridges SFT and RL.<n>It first applies an attention-aware image mixing via token merge with more cluster representation and spatial context.<n>It then presents a preference-driven training paradigm for MLLMs by building preference pairs with mixed images and raw images, and optimizing via SimPO loss.
  arXiv  Detail & Related papers  (2025-10-27T16:12:40Z)
• Coefficients-Preserving Sampling for Reinforcement Learning with Flow Matching [6.238027696245818]
  Reinforcement Learning (RL) has emerged as a powerful technique for improving image and video generation in Diffusion and Flow Matching models.<n>Our investigation reveals a significant drawback to this approach: SDE-based sampling introduces pronounced noise artifacts in the generated images.<n>Our proposed method, Coefficients-Preserving Sampling (CPS) eliminates these noise artifacts.
  arXiv  Detail & Related papers  (2025-09-07T07:25:00Z)
• Rectified Diffusion Guidance for Conditional Generation [94.83538269086613]
  We revisit the theory behind CFG and rigorously confirm that the improper combination coefficients (textiti.e.) brings about expectation shift the generative distribution.<n>We show that our approach enjoys a textbftextitform solution given the strength.<n> Empirical evidence on real-world data demonstrate the compatibility of our design with existing state-of-the-art diffusion models.
  arXiv  Detail & Related papers  (2024-10-24T13:41:32Z)
• Hybrid Convolutional and Attention Network for Hyperspectral Image Denoising [54.110544509099526]
  Hyperspectral image (HSI) denoising is critical for the effective analysis and interpretation of hyperspectral data. We propose a hybrid convolution and attention network (HCANet) to enhance HSI denoising. Experimental results on mainstream HSI datasets demonstrate the rationality and effectiveness of the proposed HCANet.
  arXiv  Detail & Related papers  (2024-03-15T07:18:43Z)
• Chasing Fairness in Graphs: A GNN Architecture Perspective [73.43111851492593]
  We propose textsfFair textsfMessage textsfPassing (FMP) designed within a unified optimization framework for graph neural networks (GNNs) In FMP, the aggregation is first adopted to utilize neighbors' information and then the bias mitigation step explicitly pushes demographic group node presentation centers together. Experiments on node classification tasks demonstrate that the proposed FMP outperforms several baselines in terms of fairness and accuracy on three real-world datasets.
  arXiv  Detail & Related papers  (2023-12-19T18:00:15Z)
• Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers [55.90468016961356]
  We propose an efficient token mixer that learns to mix in the Fourier domain. AFNO is based on a principled foundation of operator learning. It can handle a sequence size of 65k and outperforms other efficient self-attention mechanisms.
  arXiv  Detail & Related papers  (2021-11-24T05:44:31Z)
• Gaussian MRF Covariance Modeling for Efficient Black-Box Adversarial Attacks [86.88061841975482]
  We study the problem of generating adversarial examples in a black-box setting, where we only have access to a zeroth order oracle. We use this setting to find fast one-step adversarial attacks, akin to a black-box version of the Fast Gradient Sign Method(FGSM) We show that the method uses fewer queries and achieves higher attack success rates than the current state of the art.
  arXiv  Detail & Related papers  (2020-10-08T18:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.