Analyzing Visual Aircraft Representations with Sparse Autoencoders

• URL: http://arxiv.org/abs/2606.15468v1
• Date: Sat, 13 Jun 2026 20:54:42 GMT
• Title: Analyzing Visual Aircraft Representations with Sparse Autoencoders
• Abstract summary: This work investigates whether sparse autoencoders can decompose intermediate representations of a vision model into interpretable features.<n>We train a ConvNeXt classifier on the FGVC-Aircraft dataset, extract spatial activations from its final feature stage, and train a sparse autoencoder on these activations.<n>The learned sparse features are analyzed using top-activating image patches, activation strength, and class selectivity.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision models can achieve strong performance on classification tasks, but the internal representations supporting their predictions are often difficult to interpret. This work investigates whether sparse autoencoders can decompose intermediate representations of a vision model into interpretable features. We train a ConvNeXt classifier on the FGVC-Aircraft dataset, extract spatial activations from its final feature stage, and train a sparse autoencoder on these activations. The learned sparse features are analyzed using top-activating image patches, activation strength, and class selectivity. Qualitative visual inspection reveals that several features correspond to recognizable aircraft structures and visual patterns. We evaluate a subset of selected features using input-space and feature-space ablations, measuring how blurring image patches and suppressing sparse features affect class logits, classification margins, and prediction confidence. The results suggest that sparse autoencoders can reveal partially interpretable, class-relevant visual features associated with aircraft recognition, while also exposing limitations such as polysemanticity and coarse spatial localization.

Related papers

• Transcoders Trace Visual Grounding and Hallucinations in Vision-Language Models [7.425292797106273]
  Generative Vision-Language Models (VLMs) perform well on multimodal reasoning, but how visual inputs are transformed to text remains poorly understood.<n>We adopt a function-centric framework based on Transcoders, sparse approximations of sublayers that act as a causal proxy for layer-wise.<n>We show that function-centric circuit decomposition yields interpretable and predictive accounts of multimodal computation inVLMs.
  arXiv  Detail & Related papers  (2026-05-21T17:34:39Z)
• Compositional Caching for Training-free Open-vocabulary Attribute Detection [65.46250297408974]
  We present Compositional Caching (ComCa), a training-free method for open-vocabulary attribute detection.<n>ComCa requires only the list of target attributes and objects as input, using them to populate an auxiliary cache of images.<n>Experiments on public datasets demonstrate that ComCa significantly outperforms zero-shot and cache-based baselines.
  arXiv  Detail & Related papers  (2025-03-24T21:00:37Z)
• Sparse Autoencoder Insights on Voice Embeddings [3.2377830280631468]
  This study applies sparse autoencoders to speaker embeddings generated from a Titanet model.<n>The extracted features exhibit characteristics similar to those found in Large Language Model embeddings, including feature splitting and steering.<n>The analysis reveals that the autoencoder can identify and manipulate features such as language and music, which are not evident in the original embedding.
  arXiv  Detail & Related papers  (2025-01-31T19:21:43Z)
• Spatial Action Unit Cues for Interpretable Deep Facial Expression Recognition [55.97779732051921]
  State-of-the-art classifiers for facial expression recognition (FER) lack interpretability, an important feature for end-users. A new learning strategy is proposed to explicitly incorporate AU cues into classifier training, allowing to train deep interpretable models. Our new strategy is generic, and can be applied to any deep CNN- or transformer-based classifier without requiring any architectural change or significant additional training time.
  arXiv  Detail & Related papers  (2024-10-01T10:42:55Z)
• Neural Clustering based Visual Representation Learning [61.72646814537163]
  Clustering is one of the most classic approaches in machine learning and data analysis. We propose feature extraction with clustering (FEC), which views feature extraction as a process of selecting representatives from data. FEC alternates between grouping pixels into individual clusters to abstract representatives and updating the deep features of pixels with current representatives.
  arXiv  Detail & Related papers  (2024-03-26T06:04:50Z)
• Guided Interpretable Facial Expression Recognition via Spatial Action Unit Cues [55.97779732051921]
  A new learning strategy is proposed to explicitly incorporate au cues into classifier training. We show that our strategy can improve layer-wise interpretability without degrading classification performance.
  arXiv  Detail & Related papers  (2024-02-01T02:13:49Z)
• An Integral Projection-based Semantic Autoencoder for Zero-Shot Learning [0.46644955105516456]
  Zero-shot Learning (ZSL) classification categorizes or predicts classes (labels) that are not included in the training set (unseen classes) Recent works proposed different semantic autoencoder (SAE) models where the encoder embeds a visual feature space into the semantic space and the decoder reconstructs the original visual feature space. We propose an integral projection-based semantic autoencoder (IP-SAE) where an encoder projects a visual feature space vectord with the semantic space into a latent representation space.
  arXiv  Detail & Related papers  (2023-06-26T12:06:20Z)
• Feature Calibration Network for Occluded Pedestrian Detection [137.37275165635882]
  We propose a novel feature learning method in the deep learning framework, referred to as Feature Network (FC-Net) FC-Net is based on the observation that the visible parts of pedestrians are selective and decisive for detection. Experiments on CityPersons and Caltech datasets demonstrate that FC-Net improves detection performance on occluded pedestrians up to 10%.
  arXiv  Detail & Related papers  (2022-12-12T05:48:34Z)
• Spurious Features Everywhere -- Large-Scale Detection of Harmful Spurious Features in ImageNet [36.48282338829549]
  In this paper, we develop a framework that allows us to systematically identify spurious features in large datasets like ImageNet. We validate our results by showing that presence of the harmful spurious feature of a class alone is sufficient to trigger the prediction of that class. We introduce SpuFix as a simple mitigation method to reduce the dependence of any ImageNet classifier on previously identified harmful spurious features.
  arXiv  Detail & Related papers  (2022-12-09T14:23:25Z)
• Fine-Grained Visual Classification using Self Assessment Classifier [12.596520707449027]
  Extracting discriminative features plays a crucial role in the fine-grained visual classification task. In this paper, we introduce a Self Assessment, which simultaneously leverages the representation of the image and top-k prediction classes. We show that our method achieves new state-of-the-art results on CUB200-2011, Stanford Dog, and FGVC Aircraft datasets.
  arXiv  Detail & Related papers  (2022-05-21T07:41:27Z)
• Self-Supervised Point Cloud Representation Learning with Occlusion Auto-Encoder [63.77257588569852]
  We present 3D Occlusion Auto-Encoder (3D-OAE) for learning representations for point clouds. Our key idea is to randomly occlude some local patches of the input point cloud and establish the supervision via recovering the occluded patches. In contrast with previous methods, our 3D-OAE can remove a large proportion of patches and predict them only with a small number of visible patches.
  arXiv  Detail & Related papers  (2022-03-26T14:06:29Z)
• Synthesizing the Unseen for Zero-shot Object Detection [72.38031440014463]
  We propose to synthesize visual features for unseen classes, so that the model learns both seen and unseen objects in the visual domain. We use a novel generative model that uses class-semantics to not only generate the features but also to discriminatively separate them.
  arXiv  Detail & Related papers  (2020-10-19T12:36:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.