Shared Semantics, Divergent Mechanisms: Unsupervised Feature Discovery by Aligning Semantics and Mechanisms

• URL: http://arxiv.org/abs/2606.08236v1
• Date: Sat, 06 Jun 2026 15:46:54 GMT
• Title: Shared Semantics, Divergent Mechanisms: Unsupervised Feature Discovery by Aligning Semantics and Mechanisms
• Abstract summary: Circuit analysis is a central approach in mechanistic interpretability, but it is typically target-conditioned.<n>We introduce distribution-level unsupervised feature discovery.<n>Across clustering and steering analyses, the discovered clusters expose continuation modes that single-view baselines miss.
• Score: 6.233211865365017
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As large language models are increasingly deployed in high-stakes settings, there is a growing need for tools that audit not only model outputs but also the internal computations that produce them. Circuit analysis is a central approach in mechanistic interpretability, but it is typically target-conditioned, explaining a single prompt paired with a chosen completion. This target-conditioned setup can obscure heterogeneity across a model's continuation distribution. We introduce distribution-level unsupervised feature discovery, which clusters sampled continuations using both semantic content and sequence-level mechanistic attributions, without manually specifying target outputs. Our method represents each continuation with a semantic embedding and a prefix-to-continuation attribution signature, then optimizes a rate-distortion objective that trades off semantic coherence, mechanistic consistency, and cluster granularity. Across clustering and steering analyses, the discovered clusters expose continuation modes that single-view baselines miss and provide interventional evidence that cluster signatures correspond to actionable mechanistic factors. Overall, our approach complements circuit analysis and behavioral evaluation by providing a scalable audit of the mechanisms underlying a model's continuation distribution.

Related papers

• Imputation Meets Clustering: Exploiting Latent Subgroup Structure for Missing Data Recovery [52.29866804948133]
  We propose CAGI (Cluster-Aware Generative Imputation), a framework that reformulates clustering and imputation as a mutually reinforcing co-optimization process.<n>Experiments on 14 benchmark datasets with 15 representative baselines demonstrate the superiority of CAGI.
  arXiv  Detail & Related papers  (2026-07-08T02:50:02Z)
• Straight-Path Flow Matching for Incomplete Multi-View Clustering [7.71004905661976]
  Incomplete Multi-View Clustering addresses the problem of clustering multi-modal data when certain views are missing.<n>Recent approaches leverage diffusion models to recover missing views via noise-to-data trajectories.<n>We develop an end-to-end IMVC architecture that integrates flow-matching view completion with cluster-level and entropy-based alignment.
  arXiv  Detail & Related papers  (2026-07-07T13:47:51Z)
• Weight-Informed Self-Explaining Clustering for Mixed-Type Tabular Data [63.62853416081748]
  WISE is a framework that unifies representation, feature weighting, clustering, and interpretation.<n>It produces faithful, human-interpretable explanations grounded in the same primitives that drive clustering.
  arXiv  Detail & Related papers  (2026-04-07T13:18:31Z)
• Explainable cluster analysis: a bagging approach [3.5173697454104844]
  Methods rarely provide insight into which features drive the grouping of similar observations.<n>We propose an ensemble-based clustering framework that integrates bagging and feature dropout to generate feature importance scores.<n>Its effectiveness is demonstrated on multiple simulated and real-world datasets.
  arXiv  Detail & Related papers  (2026-03-20T10:34:04Z)
• Cluster-R1: Large Reasoning Models Are Instruction-following Clustering Agents [81.79110139097297]
  General-purpose embedding models excel at recognizing semantic similarities but fail to capture the characteristics of texts specified by user instructions.<n>We reframe instruction-following clustering as a generative task and train large reasoning models as autonomous clustering agents.<n>Our reasoning-driven training pipeline enables LRMs to interpret high-level clustering instructions and then infer the corresponding latent groupings.
  arXiv  Detail & Related papers  (2026-03-06T10:15:54Z)
• GALACTIC: Global and Local Agnostic Counterfactuals for Time-series Clustering [9.33534873952683]
  This paper introduces GALACTIC, the first unified framework to bridge local and global counterfactual explainability for unsupervised time-series clustering.<n>At instance level (local), GALACTIC generates perturbations via a cluster-aware optimization objective.<n>At cluster level (global), to mitigate cognitive load and enhance interpretability, we formulate a representative CE selection problem.
  arXiv  Detail & Related papers  (2026-03-05T15:59:33Z)
• GCC: Generative Calibration Clustering [55.44944397168619]
  We propose a novel Generative Clustering (GCC) method to incorporate feature learning and augmentation into clustering procedure. First, we develop a discrimirative feature alignment mechanism to discover intrinsic relationship across real and generated samples. Second, we design a self-supervised metric learning to generate more reliable cluster assignment.
  arXiv  Detail & Related papers  (2024-04-14T01:51:11Z)
• Causal Unsupervised Semantic Segmentation [60.178274138753174]
  Unsupervised semantic segmentation aims to achieve high-quality semantic grouping without human-labeled annotations. We propose a novel framework, CAusal Unsupervised Semantic sEgmentation (CAUSE), which leverages insights from causal inference.
  arXiv  Detail & Related papers  (2023-10-11T10:54:44Z)
• Robust Hierarchical Clustering for Directed Networks: An Axiomatic Approach [13.406858660972551]
  We provide a complete taxonomic characterization of robust hierarchical clustering methods for directed networks. We introduce three practical properties associated with robustness in hierarchical clustering: linear scale preservation, stability, and excisiveness. We also address the implementation of our methods and describe an application to real data.
  arXiv  Detail & Related papers  (2021-08-16T17:28:21Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)
• Target-Embedding Autoencoders for Supervised Representation Learning [111.07204912245841]
  This paper analyzes a framework for improving generalization in a purely supervised setting, where the target space is high-dimensional. We motivate and formalize the general framework of target-embedding autoencoders (TEA) for supervised prediction, learning intermediate latent representations jointly optimized to be both predictable from features as well as predictive of targets.
  arXiv  Detail & Related papers  (2020-01-23T02:37:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.