Directional Confusions Reveal Divergent Inductive Biases Through Rate-Distortion Geometry in Human and Machine Vision

• URL: http://arxiv.org/abs/2604.21909v1
• Date: Thu, 23 Apr 2026 17:52:16 GMT
• Title: Directional Confusions Reveal Divergent Inductive Biases Through Rate-Distortion Geometry in Human and Machine Vision
• Abstract summary: Humans and modern vision models can reach similar classification accuracy while making systematically different kinds of mistakes.<n>We show that these directional confusions reveal distinct inductive biases that are invisible to accuracy alone.<n>We find that humans exhibit broad but weak asymmetries, whereas deep vision models show sparser, stronger directional collapses.
• Score: 13.021443536696497
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humans and modern vision models can reach similar classification accuracy while making systematically different kinds of mistakes - differing not in how often they err, but in who gets mistaken for whom, and in which direction. We show that these directional confusions reveal distinct inductive biases that are invisible to accuracy alone. Using matched human and deep vision model responses on a natural-image categorization task under 12 perturbation types, we quantify asymmetry in confusion matrices and link it to generalization geometry through a Rate-Distortion (RD) framework, summarized by three geometric signatures (slope (beta), curvature (kappa)) and efficiency (AUC). We find that humans exhibit broad but weak asymmetries, whereas deep vision models show sparser, stronger directional collapses. Robustness training reduces global asymmetry but fails to recover the human-like breadth-strength profile of graded similarity. Mechanistic simulations further show that different asymmetry organizations shift the RD frontier in opposite directions, even when matched for performance. Together, these results position directional confusions and RD geometry as compact, interpretable signatures of inductive bias under distribution shift.

Related papers

• Structural Loss Metrics for Tensor Approximation via Matrix Low-Rank Approximation [0.0]
  We prove that squared relative reconstruction error decomposes into interaction loss and out-of-subspace energy.<n>Experiments on synthetic and hyperspectral datasets demonstrate that nearly identical reconstruction errors can yield markedly different structural-loss profiles.
  arXiv  Detail & Related papers  (2026-07-27T05:12:27Z)
• Aligning Network Equivariance with Data Symmetry: A Theoretical Framework and Adaptive Approach for Image Restoration [71.51826063983434]
  Image restoration is an inherently ill posed inverse problem.<n>We use non strict symmetry as a constraint to formulate the restoration inverse problem.<n>We show that the equivariance error of the optimal restoration operator is strictly bounded by the data symmetry error and the discretization mesh size.
  arXiv  Detail & Related papers  (2026-05-13T16:22:19Z)
• The Geometric Alignment Tax: Tokenization vs. Continuous Geometry in Scientific Foundation Models [0.0]
  We identify the root cause: the Geometric Alignment Tax.<n>We evaluate 14 biological foundation models with rate-distortion theory and MINE.<n>No model achieves simultaneously low distortion, high mutual information, and global coherence.
  arXiv  Detail & Related papers  (2026-04-05T15:45:49Z)
• Rate-Distortion Signatures of Generalization and Information Trade-offs [13.021443536696497]
  Generalization to novel visual conditions remains a central challenge for both human and machine vision.<n>Standard robustness metrics offer limited insight into how systems trade accuracy for robustness.<n>We introduce a rate-distortion-theoretic framework that treats stimulus-response behavior as an effective communication channel.
  arXiv  Detail & Related papers  (2026-03-02T07:48:39Z)
• Counterfactual Explanations on Robust Perceptual Geodesics [13.054357482525505]
  We introduce Perceptual Counterfactual Geodesics (PCG), a method that constructs counterfactuals by tracing geodesics under a metric induced from robust vision features.<n>This geometry aligns with human perception and penalizes brittle directions, enabling smooth, on-manifold, semantically valid transitions.<n>Experiments on three vision datasets show that PCG outperforms baselines and reveals failure modes hidden under standard metrics.
  arXiv  Detail & Related papers  (2026-01-26T16:52:54Z)
• Generative Model Inversion Through the Lens of the Manifold Hypothesis [98.37040155914595]
  Model inversion attacks (MIAs) aim to reconstruct class-representative samples from trained models.<n>Recent generative MIAs utilize generative adversarial networks to learn image priors that guide the inversion process.
  arXiv  Detail & Related papers  (2025-09-24T14:39:25Z)
• Generalized Linear Mode Connectivity for Transformers [87.32299363530996]
  A striking phenomenon is linear mode connectivity (LMC), where independently trained models can be connected by low- or zero-loss paths.<n>Prior work has predominantly focused on neuron re-ordering through permutations, but such approaches are limited in scope.<n>We introduce a unified framework that captures four symmetry classes: permutations, semi-permutations, transformations, and general invertible maps.<n>This generalization enables, for the first time, the discovery of low- and zero-barrier linear paths between independently trained Vision Transformers and GPT-2 models.
  arXiv  Detail & Related papers  (2025-06-28T01:46:36Z)
• BiasConnect: Investigating Bias Interactions in Text-to-Image Models [73.76853483463836]
  We introduce BiasConnect, a novel tool designed to analyze and quantify bias interactions in Text-to-Image models.<n>Our method provides empirical estimates that indicate how other bias dimensions shift toward or away from an ideal distribution when a given bias is modified.<n>We demonstrate the utility of BiasConnect for selecting optimal bias mitigation axes, comparing different TTI models on the dependencies they learn, and understanding the amplification of intersectional societal biases in TTI models.
  arXiv  Detail & Related papers  (2025-03-12T19:01:41Z)
• Latent Representation Matters: Human-like Sketches in One-shot Drawing Tasks [15.328499301244708]
  We study how different inductive biases shape the latent space of Latent Diffusion Models (LDMs) We demonstrate that LDMs with redundancy reduction and prototype-based regularizations produce near-human-like drawings.
  arXiv  Detail & Related papers  (2024-06-10T07:52:29Z)
• A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learning [68.76846801719095]
  We show that double descent appears exactly when and where it occurs, and that its location is not inherently tied to the threshold p=n. This provides a resolution to tensions between double descent and statistical intuition.
  arXiv  Detail & Related papers  (2023-10-29T12:05:39Z)
• It's an Alignment, Not a Trade-off: Revisiting Bias and Variance in Deep Models [51.66015254740692]
  We show that for an ensemble of deep learning based classification models, bias and variance are emphaligned at a sample level. We study this phenomenon from two theoretical perspectives: calibration and neural collapse.
  arXiv  Detail & Related papers  (2023-10-13T17:06:34Z)
• Predicting and Enhancing the Fairness of DNNs with the Curvature of Perceptual Manifolds [44.79535333220044]
  Recent studies have shown that tail classes are not always hard to learn, and model bias has been observed on sample-balanced datasets.<n>In this work, we first establish a geometric perspective for analyzing model fairness and then systematically propose a series of geometric measurements.
  arXiv  Detail & Related papers  (2023-03-22T04:49:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.