How Far Do Auto-Interpretation Labels Generalize: A Controlled Study Across Languages, Scripts, and Rewordings

• URL: http://arxiv.org/abs/2606.00356v2
• Date: Wed, 03 Jun 2026 03:59:07 GMT
• Title: How Far Do Auto-Interpretation Labels Generalize: A Controlled Study Across Languages, Scripts, and Rewordings
• Abstract summary: We show that SAE feature sets activated by the same content in different languages, scripts, and wordings share substantial overlap.<n>We then test whether auto-interpretation labels keep pace. They often do not.<n>These results suggest that auto-interpretation labels may reflect a feature's behavior on well-represented inputs rather than the concept itself.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sparse autoencoder (SAE) features are increasingly used to interpret language models, with auto-generated natural-language labels serving as the primary interface for understanding what each feature represents. We ask whether these labels generalize: does a feature labeled for a concept actually track that concept across languages and scripts? Using Serbian digraphia as a controlled testbed--the same language written in both Latin and Cyrillic via deterministic transliteration--we first find that SAE feature sets activated by the same content in different languages, scripts, and wordings share substantial overlap (mean Jaccard 0.39 vs. 0.13 random baseline, peaking at 0.57), suggesting genuine cross-lingual semantic features. We then test whether auto-interpretation labels keep pace. They often do not: features whose labels describe semantic content miss the same meaning in Serbian up to 4x more often thanwithin English, and miss Serbian Cyrillic more than Serbian Latin--two scripts that are deterministic transliterations of each other--suggesting the failures align with how well each form is represented in training. The gap grows with network depth, yet the labels give no indication that they fail. These results suggest that auto-interpretation labels may reflect a feature's behavior on well-represented inputs rather than the concept itself.

Related papers

• Tokenizing Crosslingual Homographs [8.254230288283258]
  Multilingual language models rely on shared subword vocabularies to represent multiple languages within a limited number of token units.<n>We investigate this limitation through cross-lingual homographs and false friends.<n>We propose a simple tokenizer-level intervention based on language cues: language-specific characters replacing initial characters of shared-vocabulary words.
  arXiv  Detail & Related papers  (2026-07-20T08:37:31Z)
• A Universal Vibe? Finding and Controlling Language-Agnostic Informal Register with SAEs [0.858070544154173]
  We study whether multilingual language models process culture-specific pragmatic registers, such as slang, as isolated language-specific memorizations or as unified, abstract concepts.<n>We introduce a novel dataset in which every target term is polysemous, appearing in both literal and informal contexts.<n>We find that while much of the informal-register signal is distributed across language-specific features, a small but highly robust cross-linguistic core consistently emerges.
  arXiv  Detail & Related papers  (2026-03-27T09:58:31Z)
• One Language, Two Scripts: Probing Script-Invariance in LLM Concept Representations [0.0]
  We investigate whether features learned by Sparse Autoencoders (SAEs) represent abstract meaning, or are tied to how text is written.<n>We find that identical sentences in different Serbian scripts activate highly overlapping features, far exceeding random baselines.<n>Our findings suggest SAE features can capture semantics at a level of abstraction above surface tokenization.
  arXiv  Detail & Related papers  (2026-03-09T19:31:20Z)
• Label Forensics: Interpreting Hard Labels in Black-Box Text Classifier [27.11428257480993]
  We introduce label forensics, a black-box framework that reconstructs a label's semantic meaning.<n> Experiments on multiple black-box classifiers achieve an average label consistency of around 92.24 percent.
  arXiv  Detail & Related papers  (2025-12-01T10:39:51Z)
• Tomato, Tomahto, Tomate: Measuring the Role of Shared Semantics among Subwords in Multilingual Language Models [88.07940818022468]
  We take an initial step on measuring the role of shared semantics among subwords in the encoder-only multilingual language models (mLMs) We form "semantic tokens" by merging the semantically similar subwords and their embeddings. inspections on the grouped subwords show that they exhibit a wide range of semantic similarities.
  arXiv  Detail & Related papers  (2024-11-07T08:38:32Z)
• Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions [50.92702206798324]
  We propose a generative prompting framework for zero-shot text classification. GEN-Z measures the LM likelihood of input text conditioned on natural language descriptions of labels. We show that zero-shot classification with simple contextualization of the data source consistently outperforms both zero-shot and few-shot baselines.
  arXiv  Detail & Related papers  (2023-11-13T07:12:57Z)
• Contextual Label Projection for Cross-Lingual Structured Prediction [103.55999471155104]
  CLaP translates text to the target language and performs contextual translation on the labels using the translated text as the context. We benchmark CLaP with other label projection techniques on zero-shot cross-lingual transfer across 39 languages.
  arXiv  Detail & Related papers  (2023-09-16T10:27:28Z)
• Description-Enhanced Label Embedding Contrastive Learning for Text Classification [65.01077813330559]
  Self-Supervised Learning (SSL) in model learning process and design a novel self-supervised Relation of Relation (R2) classification task. Relation of Relation Learning Network (R2-Net) for text classification, in which text classification and R2 classification are treated as optimization targets. external knowledge from WordNet to obtain multi-aspect descriptions for label semantic learning.
  arXiv  Detail & Related papers  (2023-06-15T02:19:34Z)
• Statistical and Neural Methods for Cross-lingual Entity Label Mapping in Knowledge Graphs [4.466084612775998]
  We investigate the application of word and sentence alignment techniques to align cross-lingual entity labels extracted from Wikidata in 10 languages. Our results indicate that mapping between Wikidata's main labels stands to be considerably improved (up to $20$ points in F1-score) by any of the employed methods.
  arXiv  Detail & Related papers  (2022-06-17T11:57:08Z)
• Neural Label Search for Zero-Shot Multi-Lingual Extractive Summarization [80.94424037751243]
  In zero-shot multilingual extractive text summarization, a model is typically trained on English dataset and then applied on summarization datasets of other languages. We propose NLS (Neural Label Search for Summarization), which jointly learns hierarchical weights for different sets of labels together with our summarization model. We conduct multilingual zero-shot summarization experiments on MLSUM and WikiLingua datasets, and we achieve state-of-the-art results using both human and automatic evaluations.
  arXiv  Detail & Related papers  (2022-04-28T14:02:16Z)
• On the Language Neutrality of Pre-trained Multilingual Representations [70.93503607755055]
  We investigate the language-neutrality of multilingual contextual embeddings directly and with respect to lexical semantics. Our results show that contextual embeddings are more language-neutral and, in general, more informative than aligned static word-type embeddings. We show how to reach state-of-the-art accuracy on language identification and match the performance of statistical methods for word alignment of parallel sentences.
  arXiv  Detail & Related papers  (2020-04-09T19:50:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.