GGSS: Geodesic-Gated Spherical Steering for Inference-Time Debiasing of Generative Vision-Language Models

• URL: http://arxiv.org/abs/2608.25375v1
• Date: Wed, 26 Aug 2026 04:58:40 GMT
• Title: GGSS: Geodesic-Gated Spherical Steering for Inference-Time Debiasing of Generative Vision-Language Models
• Abstract summary: Generative vision-language models (VLMs) are increasingly used in human-centered settings.<n>They can produce demographically biased outputs even when images differ only in controlled attributes such as perceived race or gender.<n>We propose GGSS---Geodesic-Gated Spherical Steering---a norm-preserving intervention that steers visual tokens along geodesic arcs.
• Score: 19.055058751178056
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative vision-language models (VLMs) are increasingly used in human-centered settings, yet they can produce demographically biased outputs even when images differ only in controlled attributes such as perceived race or gender. However, existing inference-time debiasers were largely designed for static embeddings or CLIP-like models rather than generative VLMs. We propose GGSS---Geodesic-Gated Spherical Steering---a norm-preserving intervention that discovers a counterfactual bias subspace on the unit hypersphere, steers visual tokens along geodesic arcs, and uses an adaptive gate to focus correction on tokens that carry stronger demographic signal. We evaluate four generative VLMs against ten adapted inference-time debiasing baselines and prompt-based mitigation under a single operating-point protocol across categorical, pairwise, and occupation-gender bias tests, while also measuring general visual-language capability. GGSS achieves the lowest average bias on all four models, significant on three of four backbones under paired permutation tests, while preserving MMStar accuracy within +/- 0.6 p.p. of the unsteered baseline. Code is available at https://github.com/dukesun99/GGSS.

Related papers

• MBD: A Model-Based Debiasing Framework Across User, Content, and Model Dimensions [50.00784452900918]
  We propose a general, model-based debiasing (MBD) framework that addresses this challenge.<n>We explicitly estimate the contextual mean and variance of the engagement distribution for arbitrary cohorts.<n>This integration allows the framework to convert biased raw signals into unbiased representations.
  arXiv  Detail & Related papers  (2026-03-15T15:07:01Z)
• Bi-directional Bias Attribution: Debiasing Large Language Models without Modifying Prompts [29.864293711943038]
  We propose a framework for detecting stereotype-inducing words and attributing neuron-level bias in large language models.<n>Our framework first identifies stereotype-inducing adjectives and nouns via comparative analysis across demographic groups.<n> Experiments on three widely used LLMs demonstrate that our method effectively reduces bias while preserving overall model performance.
  arXiv  Detail & Related papers  (2026-02-04T10:27:36Z)
• Prototypicality Bias Reveals Blindspots in Multimodal Evaluation Metrics [25.374192139098284]
  We study prototypicality bias as a systematic failure mode in multimodal evaluation.<n>We introduce a controlled contrastive benchmark ProtoBias, spanning Animals, Objects, and Demography images.<n>Our results show that widely used metrics, including CLIPScore, PickScore, and VQA-based scores, frequently misrank these pairs.<n>We propose ProtoScore, a robust 7B- parameter metric that substantially reduces failure rates and suppresses misranking.
  arXiv  Detail & Related papers  (2026-01-08T13:49:14Z)
• Bias Is a Subspace, Not a Coordinate: A Geometric Rethinking of Post-hoc Debiasing in Vision-Language Models [13.46898179372249]
  Vision-Language Models (VLMs) have become indispensable for multimodal reasoning, yet their representations often encode and amplify demographic biases.<n>We propose a geometrically principled framework that identifies and removes the entire subspace of linearly decodable bias.<n>Our method achieves more robust debiasing with an average improvement of $18.5%$ across four fairness metrics.
  arXiv  Detail & Related papers  (2025-11-22T17:04:30Z)
• SegDebias: Test-Time Bias Mitigation for ViT-Based CLIP via Segmentation [23.565073829744676]
  We propose a test-time debiasing method for ViT based CLIP models that requires no additional training or assumptions of bias annotations.<n>Our approach uses a pretrained segmentation model to isolate the target visual attribute, then adjusts the non target regions so that their embeddings are uniformly similar to all class specific text prompts.<n>Experiments on Waterbirds and CelebA show that our method outperforms existing test-time debiasing approaches in both group robustness metrics and Attention IoU.
  arXiv  Detail & Related papers  (2025-11-01T11:57:57Z)
• debiaSAE: Benchmarking and Mitigating Vision-Language Model Bias [1.3995965887921709]
  We analyze demographic biases across five models and six datasets.<n>Portrait datasets like UTKFace and CelebA are the best tools for bias detection.<n>Our debiasing method improves fairness, gaining 5-15 points in performance over the baseline.
  arXiv  Detail & Related papers  (2024-10-17T02:03:27Z)
• Debiasing Vision-Language Models via Biased Prompts [79.04467131711775]
  We propose a general approach for debiasing vision-language foundation models by projecting out biased directions in the text embedding. We show that debiasing only the text embedding with a calibrated projection matrix suffices to yield robust classifiers and fair generative models.
  arXiv  Detail & Related papers  (2023-01-31T20:09:33Z)
• Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning [122.62311703151215]
  Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations. DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
  arXiv  Detail & Related papers  (2022-11-12T09:21:49Z)
• General Greedy De-bias Learning [163.65789778416172]
  We propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model like gradient descent in functional space. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
  arXiv  Detail & Related papers  (2021-12-20T14:47:32Z)
• Balancing Biases and Preserving Privacy on Balanced Faces in the Wild [50.915684171879036]
  There are demographic biases present in current facial recognition (FR) models. We introduce our Balanced Faces in the Wild dataset to measure these biases across different ethnic and gender subgroups. We find that relying on a single score threshold to differentiate between genuine and imposters sample pairs leads to suboptimal results. We propose a novel domain adaptation learning scheme that uses facial features extracted from state-of-the-art neural networks.
  arXiv  Detail & Related papers  (2021-03-16T15:05:49Z)
• LOGAN: Local Group Bias Detection by Clustering [86.38331353310114]
  We argue that evaluating bias at the corpus level is not enough for understanding how biases are embedded in a model. We propose LOGAN, a new bias detection technique based on clustering. Experiments on toxicity classification and object classification tasks show that LOGAN identifies bias in a local region.
  arXiv  Detail & Related papers  (2020-10-06T16:42:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.