Calibrating WEAT Against Anisotropy: ZCA Whitening as a Geometric Pre-Processing Step for Embedding Association Tests

• URL: http://arxiv.org/abs/2608.06908v1
• Date: Fri, 07 Aug 2026 07:42:26 GMT
• Title: Calibrating WEAT Against Anisotropy: ZCA Whitening as a Geometric Pre-Processing Step for Embedding Association Tests
• Abstract summary: We propose Zero-phase Component Analysis (ZCA) whitening as a geometric pre-processing step for the Word Embedding Association Test (WEAT)<n>ZCA whitening transforms the covariance of the embedding space into the identity matrix while minimizing perturbation to the original vectors.<n>We evaluate our approach on ten standard WEAT test suites and seven models spanning three architectural families, yielding 70 model-task combinations.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose Zero-phase Component Analysis (ZCA) whitening as a geometric pre-processing step for the Word Embedding Association Test (WEAT). WEAT is a bias measurement method widely used in both computational social science and AI fairness research. It relies on cosine similarity as a measure of semantic association, which assumes that the embedding space is approximately isotropic. However, prior work has reported that many widely used language models do not satisfy this assumption, raising concerns about the reliability of bias measurements. ZCA whitening transforms the covariance of the embedding space into the identity matrix while minimizing perturbation to the original vectors. This transformation restores the isotropy condition on which WEAT relies. We evaluate our approach on ten standard WEAT test suites and seven models spanning three architectural families, yielding 70 model-task combinations. The results show that ZCA whitening substantially reduces the anisotropy of the embedding spaces across all models. Particularly for highly anisotropic models, we further observe improvements on standard semantic similarity benchmarks, indicating that the calibrated space better captures semantic associations. After calibration, over 30% of WEAT results change significance status, and effect sizes shift in both directions depending on bias category. These shifts suggest that uncalibrated measurements may both overestimate and underestimate the associations encoded in the embedding space. These findings indicate that previously reported bias measurements in anisotropic embedding spaces should be interpreted with caution and may benefit from re-evaluation with calibrated methods. Our approach contributes to restoring the measurement foundation of WEAT across both computational social science and AI fairness research.

Related papers

• Aligning Network Equivariance with Data Symmetry: A Theoretical Framework and Adaptive Approach for Image Restoration [71.51826063983434]
  Image restoration is an inherently ill posed inverse problem.<n>We use non strict symmetry as a constraint to formulate the restoration inverse problem.<n>We show that the equivariance error of the optimal restoration operator is strictly bounded by the data symmetry error and the discretization mesh size.
  arXiv  Detail & Related papers  (2026-05-13T16:22:19Z)
• On Procrustes Contamination in Machine Learning Applications of Geometric Morphometrics [0.0]
  Geometric morphometrics (GMM) is widely used to quantify shape variation, more recently serving as input for machine learning (ML) analyses.<n>Standard practice aligns all specimens via Generalized Procrustes Analysis (GPA) prior to splitting data into training and test sets.<n>Here, the effects of GPA-induced contamination are formally characterised using controlled 2D and 3D simulations.
  arXiv  Detail & Related papers  (2026-01-26T12:56:23Z)
• Calibrated Similarity for Reliable Geometric Analysis of Embedding Spaces [0.0]
  We construct a isotonic transformation that achieves near-perfect calibration while preserving rank correlation and local stability.<n>Our contribution is not to replace cosine similarity, but to restore interpretability of its absolute values through monotone calibration.
  arXiv  Detail & Related papers  (2026-01-23T17:14:44Z)
• Diversity by Design: Addressing Mode Collapse Improves scRNA-seq Perturbation Modeling on Well-Calibrated Metrics [1.756284691863048]
  Recent benchmarks reveal that models for single-cell perturbation response are often outperformed by simply predicting the dataset mean.<n>We trace this anomaly to a metric artifact: control-referenced deltas and unweighted error metrics reward mode collapse whenever the control is biased or the biological signal is sparse.<n>We introduce differentially expressed gene (DEG)-aware metrics, weighted mean-squared error (WMSE) and weighted delta $R2$ ($R2_w(Delta)$) with respect to all perturbations, that measure error in niche signals with high sensitivity.
  arXiv  Detail & Related papers  (2025-06-27T21:12:46Z)
• ML-EAT: A Multilevel Embedding Association Test for Interpretable and Transparent Social Science [18.682235650233743]
  This research introduces the Multilevel Embedding Association Test (ML-EAT) The ML-EAT addresses issues of ambiguity and difficulty in interpreting the traditional EAT measurement. Using the ML-EAT, this research defines a taxonomy of EAT patterns describing the nine possible outcomes of an embedding association test.
  arXiv  Detail & Related papers  (2024-08-04T09:04:44Z)
• Improving Bias Correction Standards by Quantifying its Effects on Treatment Outcomes [54.18828236350544]
  Propensity score matching (PSM) addresses selection biases by selecting comparable populations for analysis. Different matching methods can produce significantly different Average Treatment Effects (ATE) for the same task, even when meeting all validation criteria. To address this issue, we introduce a novel metric, A2A, to reduce the number of valid matches.
  arXiv  Detail & Related papers  (2024-07-20T12:42:24Z)
• Orthogonal Causal Calibration [55.28164682911196]
  We develop general algorithms for reducing the task of causal calibration to that of calibrating a standard (non-causal) predictive model.<n>Our results are exceedingly general, showing that essentially any existing calibration algorithm can be used in causal settings.
  arXiv  Detail & Related papers  (2024-06-04T03:35:25Z)
• STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection [80.04000067312428]
  We propose a Self-adapTive Ambiguity Reduction (STAR) loss by exploiting the properties of semantic ambiguity. We find that semantic ambiguity results in the anisotropic predicted distribution, which inspires us to use predicted distribution to represent semantic ambiguity. We also propose two kinds of eigenvalue restriction methods that could avoid both distribution's abnormal change and the model's premature convergence.
  arXiv  Detail & Related papers  (2023-06-05T10:33:25Z)
• Stable Anisotropic Regularization [18.52015282224059]
  We propose I-STAR: IsoScore*-based STable Anisotropic Regularization, a novel regularization method that can be used to increase or decrease levels of isotropy in embedding space during training. I-STAR uses IsoScore*, the first accurate measure of isotropy that is both differentiable and stable on mini-batch computations.
  arXiv  Detail & Related papers  (2023-05-30T18:57:45Z)
• On Calibrating Semantic Segmentation Models: Analyses and An Algorithm [51.85289816613351]
  We study the problem of semantic segmentation calibration. Model capacity, crop size, multi-scale testing, and prediction correctness have impact on calibration. We propose a simple, unifying, and effective approach, namely selective scaling.
  arXiv  Detail & Related papers  (2022-12-22T22:05:16Z)
• Learning from Aggregate Observations [82.44304647051243]
  We study the problem of learning from aggregate observations where supervision signals are given to sets of instances. We present a general probabilistic framework that accommodates a variety of aggregate observations. Simple maximum likelihood solutions can be applied to various differentiable models.
  arXiv  Detail & Related papers  (2020-04-14T06:18:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.