A Unifying Framework for Concept-Based Representational Similarity

• URL: http://arxiv.org/abs/2606.09653v1
• Date: Mon, 08 Jun 2026 15:42:39 GMT
• Title: A Unifying Framework for Concept-Based Representational Similarity
• Abstract summary: Concept alignment is fundamentally multi-objective: it must be defined, measured, and optimized as such.<n>We propose a unifying framework that decomposes alignment along two axes: what is aligned (representations vs. concepts) and at what level (instance-wise vs. distributional)
• Score: 26.502371676259113
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learned representations across models and modalities often exhibit striking structural similarities, suggesting shared underlying concept decompositions. However, concept alignment remains poorly defined: existing approaches optimize different objectives under the same terminology, obscuring what is actually aligned. We propose a unifying framework that decomposes alignment along two axes: what is aligned (representations vs. concepts) and at what level (instance-wise vs. distributional). This induces four corresponding properties -- instance-wise and distributional variants of translation and concept consistency -- and reveals precisely which of these guarantees existing methods provide. We further introduce \InterVenchA, an intervention-based benchmark that separately measures extraction quality, translation quality, and concept consistency. Through theory and experiments, we show that commonly assumed equivalences between alignment objectives fail in practice: optimizing one property does not reliably recover the others, and purely unsupervised objectives fail to recover meaningful instance-level alignment. We then propose the Coupled Sparse Autoencoder (CoSAE), which jointly enforces complementary alignment objectives. Strong alignment emerges only in this regime. Surprisingly, as little as 0.1\% paired data is sufficient to recover instance-level alignment when anchoring distributional objectives. Overall, our results show that concept alignment is fundamentally multi-objective: it must be defined, measured, and optimized as such.

Related papers

• Interpreting Object-Dependent Concept Brittleness in Text-to-Image Diffusion Models [105.53190946111586]
  We term this phenomenon object-dependent concept brittleness.<n>We present an interpretability-oriented framework to audit and minimally correct these failures.<n>We evaluate the proposed framework on style and attribute failure cases across multiple diffusion backbones.
  arXiv  Detail & Related papers  (2026-09-09T09:07:40Z)
• Not All Objectives Are Born Equal: Priority-Constrained Descent for Hierarchical Multi-Objective Optimization [0.6875312133832079]
  We introduce Priority-Constrained Descent (PCD) to explicitly exploit hierarchical objective structures.<n>PCD preserves the direction of primary descent whilst allowing for the minimal distortion necessary to guarantee progress on secondary objectives.<n>We evaluate PCD within structured network compression settings, unstructured sparsity and low-rankness, and across a variety of synthetic experiments.
  arXiv  Detail & Related papers  (2026-06-28T17:31:47Z)
• Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception [50.278200968044665]
  Camouflaged object detection (COD) aims to localize targets that exhibit minimal perceptual differences from backgrounds through physical attributes.<n>Existing methods, constrained by the static train-then-freeze paradigm, suffer from domain rigidity and dependency annotation.<n>We propose the hierarchical consistency learning framework, which integrates test-time adaptation for dynamic representation recalibration.
  arXiv  Detail & Related papers  (2026-05-25T09:57:46Z)
• Rethinking Semantic Collaborative Integration: Why Alignment Is Not Enough [23.86079699614653]
  Large language models (LLMs) have become an important semantic infrastructure for recommender systems.<n>We formalize the global low-complexity alignment hypothesis and argue that it is stronger than necessary and often structurally mismatched with real-world recommendation settings.<n>We propose a complementary perspective in which semantic and collaborative representations are treated as partially shared yet fundamentally heterogeneous views.
  arXiv  Detail & Related papers  (2026-04-24T03:47:12Z)
• Position: General Alignment Has Hit a Ceiling; Edge Alignment Must Be Taken Seriously [51.03213216886717]
  We take the position that the dominant paradigm of General Alignment reaches a structural ceiling in settings with conflicting values.<n>We introduce Edge Alignment as a distinct approach in which systems preserve multi dimensional value structure.
  arXiv  Detail & Related papers  (2026-02-23T16:51:43Z)
• Escaping Plato's Cave: JAM for Aligning Independently Trained Vision and Language Models [30.07172193932125]
  We show that the Joint Autoencoder Modulator (JAM) induces alignment even across independently trained representations.<n>Our findings offer theoretical insight into the structure of shared semantics and practical guidance for transforming generalist unimodal foundations into specialist multimodal models.
  arXiv  Detail & Related papers  (2025-07-01T21:43:50Z)
• Beyond Scalars: Concept-Based Alignment Analysis in Vision Transformers [10.400355814467401]
  Vision transformers (ViTs) can be trained using various learning paradigms, from fully supervised to self-supervised.<n>We propose a concept-based alignment analysis of representations from four different ViTs.<n>The concept-based alignment analysis of representations from four different ViTs reveals that increased supervision correlates with a reduction in the semantic structure of learned representations.
  arXiv  Detail & Related papers  (2024-12-09T16:33:28Z)
• Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence [51.54175067684008]
  This paper introduces a Transformer-based integrative feature and cost aggregation network designed for dense matching tasks. We first show that feature aggregation and cost aggregation exhibit distinct characteristics and reveal the potential for substantial benefits stemming from the judicious use of both aggregation processes. Our framework is evaluated on standard benchmarks for semantic matching, and also applied to geometric matching, where we show that our approach achieves significant improvements compared to existing methods.
  arXiv  Detail & Related papers  (2024-03-17T07:02:55Z)
• A Probabilistic Relaxation of the Two-Stage Object Pose Estimation Paradigm [0.0]
  We propose a matching-free probabilistic formulation for object pose estimation. It enables unified and concurrent optimization of both visual correspondence and geometric alignment. It can represent different plausible modes of the entire distribution of likely poses.
  arXiv  Detail & Related papers  (2023-06-01T16:50:40Z)
• Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning [53.68371566336254]
  We argue that the key to better performance lies in meaningful latent modality structures instead of perfect modality alignment. Specifically, we design 1) a deep feature separation loss for intra-modality regularization; 2) a Brownian-bridge loss for inter-modality regularization; and 3) a geometric consistency loss for both intra- and inter-modality regularization.
  arXiv  Detail & Related papers  (2023-03-10T14:38:49Z)
• Reinforcement Learning based Collective Entity Alignment with Adaptive Features [35.04861875266298]
  We propose a reinforcement learning (RL) based model to align entities collectively. Under the RL framework, we devise the coherence and exclusiveness constraints to characterize the interdependence and collective alignment. Our proposal is evaluated on both cross-lingual and mono-lingual EA benchmarks and compared against state-of-the-art solutions.
  arXiv  Detail & Related papers  (2021-01-05T05:04:09Z)
• Mixup-CAM: Weakly-supervised Semantic Segmentation via Uncertainty Regularization [73.03956876752868]
  We propose a principled and end-to-end train-able framework to allow the network to pay attention to other parts of the object. Specifically, we introduce the mixup data augmentation scheme into the classification network and design two uncertainty regularization terms to better interact with the mixup strategy.
  arXiv  Detail & Related papers  (2020-08-03T21:19:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.