Efficient Learning of Sparse Representations from Interactions

• URL: http://arxiv.org/abs/2602.09935v1
• Date: Tue, 10 Feb 2026 16:09:58 GMT
• Title: Efficient Learning of Sparse Representations from Interactions
• Authors: Vojtěch Vančura, Martin Spišák, Rodrigo Alves, Ladislav Peška,
• Abstract summary: We propose a training strategy for learning high-dimensional sparse embedding layers in place of conventional dense ones.<n>We modified the production-grade collaborative filtering autoencoder ELSA, achieving up to 10x reduction in embedding size with no loss of recommendation accuracy.
• Score: 9.381985901356922
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Behavioral patterns captured in embeddings learned from interaction data are pivotal across various stages of production recommender systems. However, in the initial retrieval stage, practitioners face an inherent tradeoff between embedding expressiveness and the scalability and latency of serving components, resulting in the need for representations that are both compact and expressive. To address this challenge, we propose a training strategy for learning high-dimensional sparse embedding layers in place of conventional dense ones, balancing efficiency, representational expressiveness, and interpretability. To demonstrate our approach, we modified the production-grade collaborative filtering autoencoder ELSA, achieving up to 10x reduction in embedding size with no loss of recommendation accuracy, and up to 100x reduction with only a 2.5% loss. Moreover, the active embedding dimensions reveal an interpretable inverted-index structure that segments items in a way directly aligned with the model's latent space, thereby enabling integration of segment-level recommendation functionality (e.g., 2D homepage layouts) within the candidate retrieval model itself. Source codes, additional results, as well as a live demo are available at https://github.com/zombak79/compressed_elsa

Related papers

• Knowledge-Informed Neural Network for Complex-Valued SAR Image Recognition [51.03674130115878]
  We introduce the Knowledge-Informed Neural Network (KINN), a lightweight framework built upon a novel "compression-aggregation-compression" architecture.<n>KINN establishes a state-of-the-art in parameter-efficient recognition, offering exceptional generalization in data-scarce and out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2025-10-23T07:12:26Z)
• SMEC: Rethinking Matryoshka Representation Learning for Retrieval Embedding Compression [15.655201854308396]
  We propose a novel training framework named Sequential Matryoshka Embedding Compression (SMEC)<n>This framework introduces the Sequential Matryoshka Representation Learning(SMRL) method to mitigate gradient variance during training, the Adaptive Dimension Selection (ADS) module to reduce information degradation during dimension pruning, and the Selectable Cross-batch Memory (S-XBM) module to enhance unsupervised learning between high- and low-dimensional embeddings.<n> Experiments on image, text, and multimodal datasets demonstrate that SMEC achieves significant dimensionality reduction while maintaining performance.
  arXiv  Detail & Related papers  (2025-10-14T13:04:22Z)
• Learning Hyperspectral Images with Curated Text Prompts for Efficient Multimodal Alignment [1.7188280334580195]
  We make an attempt to optimize a Vision-Language Model (VLM) for hyperspectral scene understanding by exploiting a CLIP-style contrastive training framework.<n>Our framework maps voxel-level embeddings from a vision backbone onto the latent space of a frozen large embedding model.<n>It is seen that the proposed method updates only 0.07 percent of the total parameters, yet yields state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-09-20T23:23:04Z)
• Revisiting Efficient Semantic Segmentation: Learning Offsets for Better Spatial and Class Feature Alignment [67.94226713255732]
  We propose a dual-branch offset learning paradigm that learns feature and class offsets to dynamically refine both class representations and spatial image features.<n>Based on the proposed paradigm, we construct an efficient semantic segmentation network, OffSeg.<n>Experiments on four datasets, including ADE20K, Cityscapes, COCO-Stuff-164K, and Pascal Context, demonstrate consistent improvements with negligible parameters.
  arXiv  Detail & Related papers  (2025-08-12T10:10:10Z)
• USDRL: Unified Skeleton-Based Dense Representation Learning with Multi-Grained Feature Decorrelation [24.90512145836643]
  We introduce a Unified Skeleton-based Dense Representation Learning framework based on feature decorrelation.<n>We show that our approach significantly outperforms the current state-of-the-art (SOTA) approaches.
  arXiv  Detail & Related papers  (2024-12-12T12:20:27Z)
• Anti-Collapse Loss for Deep Metric Learning Based on Coding Rate Metric [99.19559537966538]
  DML aims to learn a discriminative high-dimensional embedding space for downstream tasks like classification, clustering, and retrieval. To maintain the structure of embedding space and avoid feature collapse, we propose a novel loss function called Anti-Collapse Loss. Comprehensive experiments on benchmark datasets demonstrate that our proposed method outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-03T13:44:20Z)
• Dual-scale Enhanced and Cross-generative Consistency Learning for Semi-supervised Medical Image Segmentation [49.57907601086494]
  Medical image segmentation plays a crucial role in computer-aided diagnosis. We propose a novel Dual-scale Enhanced and Cross-generative consistency learning framework for semi-supervised medical image (DEC-Seg)
  arXiv  Detail & Related papers  (2023-12-26T12:56:31Z)
• Instance Adaptive Prototypical Contrastive Embedding for Generalized Zero Shot Learning [11.720039414872296]
  Generalized zero-shot learning aims to classify samples from seen and unseen labels, assuming unseen labels are not accessible during training. Recent advancements in GZSL have been expedited by incorporating contrastive-learning-based embedding in generative networks.
  arXiv  Detail & Related papers  (2023-09-13T14:26:03Z)
• Open-vocabulary Panoptic Segmentation with Embedding Modulation [71.15502078615587]
  Open-vocabulary image segmentation is attracting increasing attention due to its critical applications in the real world. Traditional closed-vocabulary segmentation methods are not able to characterize novel objects, whereas several recent open-vocabulary attempts obtain unsatisfactory results. We propose OPSNet, an omnipotent and data-efficient framework for Open-vocabulary Panopticon.
  arXiv  Detail & Related papers  (2023-03-20T17:58:48Z)
• Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognition [54.23513799338309]
  We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition. Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
  arXiv  Detail & Related papers  (2022-09-21T02:33:07Z)
• TLDR: Twin Learning for Dimensionality Reduction [25.373435473381356]
  Dimensionality reduction methods learn low-dimensional spaces where some properties of the initial space, typically the notion of "neighborhood", are preserved. We propose a dimensionality reduction method for generic input spaces that is porting the simple self-supervised learning framework of Barlow Twins to a setting where it is hard or impossible to define an appropriate set of distortions by hand.
  arXiv  Detail & Related papers  (2021-10-18T16:46:12Z)
• Adversarial Feature Augmentation and Normalization for Visual Recognition [109.6834687220478]
  Recent advances in computer vision take advantage of adversarial data augmentation to ameliorate the generalization ability of classification models. Here, we present an effective and efficient alternative that advocates adversarial augmentation on intermediate feature embeddings. We validate the proposed approach across diverse visual recognition tasks with representative backbone networks.
  arXiv  Detail & Related papers  (2021-03-22T20:36:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.