Unsupervised Features Mining via Activation Geometry

• URL: http://arxiv.org/abs/2607.04222v1
• Date: Sun, 05 Jul 2026 10:28:04 GMT
• Title: Unsupervised Features Mining via Activation Geometry
• Abstract summary: Interpretability methods aim to reveal the features represented inside large language models (LLMs)<n>We introduce emphMining via Activation Geometry (MAG), a simple unsupervised framework for extracting reasoning features from model activations.<n>We measure how the instruction changes the model's internal representation using $m(Q mid p) - m(p)$ at a single readout point.
• Score: 1.3636639880574484
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Interpretability methods aim to reveal the features represented inside large language models (LLMs). Many existing methods begin with labeled examples of a human-defined concept that may reflect human biases, and then identify how that concept is represented within the model, for example in its activation space or through other decomposition methods. We introduce \emph{Mining via Activation Geometry} (MAG), a simple unsupervised framework for extracting reasoning features from model activations by prepending the same natural-language instruction $Q$ to every input $p$, where $Q$ defines the reasoning feature of interest, such as ``Can this object be found in the desert?'' or ``Is this prompt malicious?'' We measure how the instruction changes the model's internal representation using $m(Q \mid p) - m(p)$ at a single readout point. We explore eight different MAGs. The extracted reasoning features predict the models' own world understanding and judgment, can be approximated into a single activation direction, we found that some features are more linearly represented and some less, this linear representation, which is vector steering, can change the LLMs' decisions through activation steering by injecting reasoning features. Finally, we use the same method to select the best training datasets for prompt-injection classifier probes: while similarity between ordinary activations is almost unrelated to downstream performance, RFD-based similarity achieves $94.7\%$ Top-1 and $100\%$ Top-2 accuracy.

Related papers

• Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretability [51.56484100374058]
  We prove the first identifiability theorem for a mechanistic-interpretability primitive.<n>The spectrum is an identifiable, model-intrinsic fingerprint with a stated error bar, not a legible decomposition.
  arXiv  Detail & Related papers  (2026-08-10T19:42:01Z)
• Latent Fact-Checking: Detecting Misinformation through Activation Engineering [1.3299946892361476]
  We introduce a misinformation detection framework grounded in activation engineering.<n>Our approach elicits a misinformation direction in the residual stream by contrasting activations from paired truthful and false statements.<n>We evaluate the method across 11 models on three fact-checking benchmarks: AVeriTeC, LIAR, and FACTors.
  arXiv  Detail & Related papers  (2026-08-05T18:00:14Z)
• ResAD++: Towards Class Agnostic Anomaly Detection via Residual Feature Learning [52.11294707895649]
  This paper explores the problem of class-agnostic anomaly detection (AD)<n>The objective is to train one class-agnostic AD model that can generalize to detect anomalies in diverse new classes from different domains without any retraining or fine-tuning on the target data.<n> Comprehensive experiments on eight real-world AD datasets demonstrate that our ResAD++ can achieve remarkable AD results when directly used in new classes.
  arXiv  Detail & Related papers  (2025-09-28T08:41:05Z)
• Features that Make a Difference: Leveraging Gradients for Improved Dictionary Learning [4.051777802443125]
  Sparse Autoencoders (SAEs) are a promising approach for extracting neural network representations.<n>We introduce Gradient SAEs, which modify the $k$-sparse autoencoder architecture by augmenting the TopK activation function.<n>We find evidence that g-SAEs learn latents that are on average more effective at steering models in arbitrary contexts.
  arXiv  Detail & Related papers  (2024-11-15T18:03:52Z)
• Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation [15.77263269398368]
  Large Language Models (LLMs) drive current AI breakthroughs. We shed the light on LLMs inner mechanisms through the lens of geometry. We derive interpretable geometrical features that can be extracted from any (pre-trained) LLM.
  arXiv  Detail & Related papers  (2023-12-04T06:01:32Z)
• ALSO: Automotive Lidar Self-supervision by Occupancy estimation [70.70557577874155]
  We propose a new self-supervised method for pre-training the backbone of deep perception models operating on point clouds. The core idea is to train the model on a pretext task which is the reconstruction of the surface on which the 3D points are sampled. The intuition is that if the network is able to reconstruct the scene surface, given only sparse input points, then it probably also captures some fragments of semantic information.
  arXiv  Detail & Related papers  (2022-12-12T13:10:19Z)
• Self-Supervised Learning Disentangled Group Representation as Feature [82.07737719232972]
  We show that existing Self-Supervised Learning (SSL) only disentangles simple augmentation features such as rotation and colorization. We propose an iterative SSL algorithm: Iterative Partition-based Invariant Risk Minimization (IP-IRM) We prove that IP-IRM converges to a fully disentangled representation and show its effectiveness on various benchmarks.
  arXiv  Detail & Related papers  (2021-10-28T16:12:33Z)
• For Manifold Learning, Deep Neural Networks can be Locality Sensitive Hash Functions [14.347610075713412]
  We show that neural representations can be viewed as LSH-like functions that map each input to an embedding. An important consequence of this behavior is one-shot learning to unseen classes.
  arXiv  Detail & Related papers  (2021-03-11T18:57:47Z)
• Is Plug-in Solver Sample-Efficient for Feature-based Reinforcement Learning? [30.065091907118827]
  This work considers sample complexity of finding an $epsilon$-optimal policy in a Markov decision process (MDP) We solve this problem via a plug-in solver approach, which builds an empirical model and plans in this empirical model via an arbitrary plug-in solver. We show that a plug-in approach can be sample efficient as well, providing a flexible approach to design model-based algorithms for reinforcement learning.
  arXiv  Detail & Related papers  (2020-10-12T13:13:01Z)
• Improving Robustness and Generality of NLP Models Using Disentangled Representations [62.08794500431367]
  Supervised neural networks first map an input $x$ to a single representation $z$, and then map $z$ to the output label $y$. We present methods to improve robustness and generality of NLP models from the standpoint of disentangled representation learning. We show that models trained with the proposed criteria provide better robustness and domain adaptation ability in a wide range of supervised learning tasks.
  arXiv  Detail & Related papers  (2020-09-21T02:48:46Z)
• Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexity [67.02490430380415]
  We show that model-based MARL achieves a sample complexity of $tilde O(|S||B|(gamma)-3epsilon-2)$ for finding the Nash equilibrium (NE) value up to some $epsilon$ error. We also show that such a sample bound is minimax-optimal (up to logarithmic factors) if the algorithm is reward-agnostic, where the algorithm queries state transition samples without reward knowledge.
  arXiv  Detail & Related papers  (2020-07-15T03:25:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.