Monosemanticity in Recommender Systems

• URL: http://arxiv.org/abs/2606.29341v2
• Date: Thu, 02 Jul 2026 05:28:08 GMT
• Title: Monosemanticity in Recommender Systems
• Abstract summary: We train a large-scale matrix factorization recommender system on the Amazon Fashion dataset and apply a Matryoshka Sparse Autoencoder to the learned embeddings.<n>We analyze the resulting latent features through metadata alignment and LLM-generated labeling to assess semantic coherence and disentanglement.<n>Our findings suggest that collaborative filtering embeddings contain recoverable hierarchical structure, and that Matryoshka training provides a principled mechanism for exposing interpretable latent factors.
• Score: 0.0
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Latent factor models such as matrix factorization are widely used in recommender systems, yet the learned embedding dimensions typically lack explicit semantic interpretation. This opacity limits transparency, explainability, and principled intervention in recommendation behavior. While sparse autoencoders (SAEs) have recently been used to extract monosemantic features from dense neural representations, standard SAEs suffer from scaling pathologies including feature splitting, feature absorption, and feature composition, which degrade interpretability as dictionary size increases. In this work, we investigate whether hierarchical sparse representations can reveal interpretable structure in collaborative filtering embeddings. We train a large-scale matrix factorization recommender system on the Amazon Fashion dataset and apply a Matryoshka Sparse Autoencoder (MSAE) to the learned embeddings. We analyze the resulting latent features through metadata alignment and LLM-generated labeling to assess semantic coherence and disentanglement. Finally, we show an intervention on a subset of gender associated latent neurons that emerged from the analysis. Our findings suggest that collaborative filtering embeddings contain recoverable hierarchical structure, and that Matryoshka training provides a principled mechanism for exposing interpretable latent factors in interaction-driven recommendation models.

Related papers

• Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach [58.15427952222424]
  General-purpose text embeddings are widely deployed for such tasks, but broad semantic similarity can place semantically similar yet task-distinct examples.<n>We introduce Prototype-Guided Contrastive Learning (PGCL), a prototype-guided geometric regularization module built on top of frozen text embeddings.
  arXiv  Detail & Related papers  (2026-08-15T13:19:25Z)
• Influence-Guided Symbolic Regression: Scientific Discovery via LLM-Driven Equation Search with Granular Feedback [56.69850045068714]
  We introduce textitInfluence-Guided Regression (IGSR), a method that frames equation discovery as an iterative two-step process.<n>We demonstrate IGSRs effectiveness on a diverse suite of benchmarks, including LLM-SRBench, pharmacological PKPD models, an epidemiological simulation, and real-world genomic data.
  arXiv  Detail & Related papers  (2026-05-27T23:48:01Z)
• K-Models: a Flexible and Interpretable Method for Ordinal Clustering with Application to Antigen-Antibody Interaction Profiles [65.21921642886367]
  K-Models is a novel framework that integrates ordinal constraints and estimates key underlying elements of the random process generating the observed functional profiles.<n>The proposed method is evaluated through simulations and real-world applications.
  arXiv  Detail & Related papers  (2026-05-14T13:35:44Z)
• Attention-Based Multimodal Survival Prediction with Cross-Modal Bilinear Fusion [1.4203297056262238]
  We propose a novel deep learning framework for patient-level survival prediction.<n>It integrates whole-slide histology features, RNA-seq expression profiles, and clinical variables.<n>Our architecture combines an ABMIL moduleciteilse2018attention for slide-level representation with feedforward encoders for RNA and clinical data.
  arXiv  Detail & Related papers  (2026-05-12T13:09:25Z)
• Weight-Informed Self-Explaining Clustering for Mixed-Type Tabular Data [63.62853416081748]
  WISE is a framework that unifies representation, feature weighting, clustering, and interpretation.<n>It produces faithful, human-interpretable explanations grounded in the same primitives that drive clustering.
  arXiv  Detail & Related papers  (2026-04-07T13:18:31Z)
• SIGMA: Scalable Spectral Insights for LLM Collapse [51.863164847253366]
  We introduce SIGMA (Spectral Inequalities for Gram Matrix Analysis), a unified framework for model collapse.<n>By utilizing benchmarks that deriving and deterministic bounds on the matrix's spectrum, SIGMA provides a mathematically grounded metric to track the contraction of the representation space.<n>We demonstrate that SIGMA effectively captures the transition towards states, offering both theoretical insights into the mechanics of collapse.
  arXiv  Detail & Related papers  (2026-01-06T19:47:11Z)
• POEMS: Product of Experts for Interpretable Multi-omic Integration using Sparse Decoding [10.520179127805187]
  We introduce POEMS: Product Of Experts for Interpretable Multiomics Integration using Sparse Decoding.<n> POEMS provides interpretability without linearizing any part of the network by 1) mapping features to latent factors using sparse connections.<n>In a cancer subtyping case study, POEMS achieves competitive clustering and classification performance.
  arXiv  Detail & Related papers  (2025-11-05T13:39:28Z)
• Unveiling Decision-Making in LLMs for Text Classification : Extraction of influential and interpretable concepts with Sparse Autoencoders [0.0]
  We present a novel SAE-based architecture tailored for text classification.<n>We benchmark this architecture against established methods such as ConceptShap, Independent Component Analysis, and other SAE-based concept extraction techniques.<n>Our empirical results show that our architecture improves both the causality and interpretability of the extracted features.
  arXiv  Detail & Related papers  (2025-06-30T15:18:50Z)
• The Complexity of Learning Sparse Superposed Features with Feedback [2.4140387101794283]
  We investigate whether the underlying learned features of a model can be efficiently retrieved through feedback from an agent.<n>We analyze the feedback complexity associated with learning a feature matrix in sparse settings.<n>Our results establish tight bounds when the agent is permitted to construct activations and demonstrate strong upper bounds in sparse scenarios.
  arXiv  Detail & Related papers  (2025-02-08T01:54:23Z)
• Enhancing the conformal predictability of context-aware recommendation systems by using Deep Autoencoders [4.3012765978447565]
  We introduce a framework that combines neural contextual matrix factorization with autoencoders to predict user ratings for items.<n>We conduct experiments on various real-world datasets and compare the results against state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-11-30T18:24:42Z)
• Enhancing Interpretability in Deep Reinforcement Learning through Semantic Clustering [4.859171801301904]
  semantic clustering refers to the ability of neural networks to cluster inputs based on their semantic similarity in the feature space.<n>We propose a DRL architecture that incorporates a novel semantic clustering module that combines feature dimensionality reduction with online clustering.<n>We experimentally validate the effectiveness of the proposed module and demonstrate its ability to reveal semantic clustering properties within DRL.
  arXiv  Detail & Related papers  (2024-09-25T22:48:14Z)
• ER: Equivariance Regularizer for Knowledge Graph Completion [107.51609402963072]
  We propose a new regularizer, namely, Equivariance Regularizer (ER) ER can enhance the generalization ability of the model by employing the semantic equivariance between the head and tail entities. The experimental results indicate a clear and substantial improvement over the state-of-the-art relation prediction methods.
  arXiv  Detail & Related papers  (2022-06-24T08:18:05Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.