ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers

• URL: http://arxiv.org/abs/2609.06155v1
• Date: Sat, 05 Sep 2026 15:55:32 GMT
• Title: ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers
• Abstract summary: ExpertLens is a post-hoc explainability framework for Mixture-of-Experts (MoE)-enhanced dense retrievers.<n>We show that ExpertLens consistently improves embedding space structure and retrieval effectiveness.
• Score: 45.67070112128053
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural models, including dense retrievers, have been widely adopted in Information Retrieval (IR), often delivering state-of-the-art performance. Despite their effectiveness, these models operate as black boxes, limiting the interpretability of their ranking decisions. Existing post-hoc explainability methods for neural rankers primarily focus on feature-level attributions, which can be insufficient to capture the complexity of learned embedding spaces. In this work, we propose ExpertLens, a post-hoc explainability framework for Mixture-of-Experts (MoE)-enhanced dense retrievers that shifts focus from local scalar feature importance to representation-level global interpretability. ExpertLens leverages discriminative embedding space visualizations jointly with automatically extracted Concept Activation Vectors to reveal how expert routing drives embedding space formulation and retrieval effectiveness. Experiments across five IR benchmarks and two MoE-enhanced dense retrievers show that expert routing consistently improves embedding space structure, positioning queries and their relevant documents into better-defined geometric neighborhoods. Analysis of expert subspaces further reveals general-purpose dominant experts, along with minority experts exhibiting distinct linguistic specialization, with subspaces arranged according to multi-semantic concept similarity. Our code is publicly available.

Related papers

• Towards a Statistical Understanding of Mixture-of-Experts [9.70213785111328]
  Mixture-of-experts (MoE) architectures increase model capacity by combining a collection of expert predictors through input-dependent routing.<n>We view MoE as a form of localized aggregation and show how this localization reshapes the approximation-estimation-computation tradeoff.<n>We also interpret gating through the geometry of input space, relating routing performance to regions of local expert advantage, and show how shared experts, as adopted in architectures such as DeepSeekMoE, can extract common predictive structure.
  arXiv  Detail & Related papers  (2026-09-03T08:03:16Z)
• Unleashing Multimodal Large Language Models for Training-free HOI Detection in the Wild [49.77540427384148]
  We present AgentHOI, a training-free, agentic framework that transfers the generalist multimodal reasoning capabilities of foundation models to HOI detection in the wild.<n>To address the challenges of incomplete interaction discovery and ambiguous localization in complex scenes, we introduce two key mechanisms.<n>AgentHOI achieves superior performance over state-of-the-art supervised and weakly supervised methods in real-world settings.
  arXiv  Detail & Related papers  (2026-07-15T14:30:20Z)
• Beyond Routing: Characterising Expert Tuning and Representation in Vision Mixture-of-Experts [0.0]
  Mixture-of-Experts (MoE) models are often interpreted by analysing categories which are routed to which experts.<n>We train sparsely-gated convolutional MoE models with a contrastive objective on natural images and expert specialisation.<n>We measure per-expert category separability, and per-expert tuning using the most exciting inputs.
  arXiv  Detail & Related papers  (2026-05-20T01:55:40Z)
• Spatial-VLN: Zero-Shot Vision-and-Language Navigation With Explicit Spatial Perception and Exploration [16.651645602449577]
  Vision-and-Language Navigation (VLN) agents leveraging Large Language Models (LLMs) excel in generalization but suffer from insufficient spatial perception.<n>We present Spatial-VLN, a perception-guided exploration framework designed to overcome these challenges.
  arXiv  Detail & Related papers  (2026-01-19T06:53:02Z)
• Visual Exploration of Feature Relationships in Sparse Autoencoders with Curated Concepts [8.768503486874623]
  We propose a focused exploration framework that prioritizes curated concepts and their corresponding SAE features over attempts to visualize all available features simultaneously.<n>We present an interactive visualization system that combines topology-based visual encoding with dimensionality reduction to faithfully represent both local and global relationships among selected features.
  arXiv  Detail & Related papers  (2025-11-08T15:36:57Z)
• Beyond Redundancy: Diverse and Specialized Multi-Expert Sparse Autoencoder [59.89996751196727]
  Sparse autoencoders (SAEs) have emerged as a powerful tool for interpreting large language models.<n>SAEs' hidden layers have high dimensionality to satisfy sparsity constraints, resulting in prohibitive training and inference costs.<n>Recent Mixture of Experts (MoE) approaches attempt to address this by SAEs into narrower expert networks with gated activation.<n>We propose two key innovations: (1) Multiple Expert Activation that simultaneously engages semantically weighted expert subsets to encourage specialization, and (2) Feature Scaling that enhances diversity through adaptive high-frequency scaling.
  arXiv  Detail & Related papers  (2025-11-07T22:19:34Z)
• SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning [48.64616652818972]
  We propose a novel framework that transforms raw depth data into structured, interpretable textual rationales.<n>These textual rationales serve as meaningful intermediate representations to significantly enhance spatial reasoning capabilities.<n>We introduce a new dataset named SSR-CoT, a million-scale visual-language reasoning dataset enriched with intermediate spatial reasoning annotations.
  arXiv  Detail & Related papers  (2025-05-18T14:40:16Z)
• Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition [57.08108545219043]
  Recognizing novel sub-categories with scarce samples is an essential and challenging research topic in computer vision. Existing literature addresses this challenge by employing local-based representation approaches. This article proposes a novel model, Robust Saliency-aware Distillation (RSaD), for few-shot fine-grained visual recognition.
  arXiv  Detail & Related papers  (2023-05-12T00:13:17Z)
• MoEC: Mixture of Expert Clusters [93.63738535295866]
  Sparsely Mixture of Experts (MoE) has received great interest due to its promising scaling capability with affordable computational overhead. MoE converts dense layers into sparse experts, and utilizes a gated routing network to make experts conditionally activated. However, as the number of experts grows, MoE with outrageous parameters suffers from overfitting and sparse data allocation.
  arXiv  Detail & Related papers  (2022-07-19T06:09:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.