UGID: Unified Graph Isomorphism for Debiasing Large Language Models

• URL: http://arxiv.org/abs/2603.19144v1
• Date: Thu, 19 Mar 2026 16:59:37 GMT
• Title: UGID: Unified Graph Isomorphism for Debiasing Large Language Models
• Abstract summary: We propose underlineUnified underlineGraph underlineIsomorphism for underlineDebiasing large language models.<n>textittextbfUGID is an internal-representation-level debiasing framework for large language models.
• Score: 13.549042680933306
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) exhibit pronounced social biases. Output-level or data-optimization--based debiasing methods cannot fully resolve these biases, and many prior works have shown that biases are embedded in internal representations. We propose \underline{U}nified \underline{G}raph \underline{I}somorphism for \underline{D}ebiasing large language models (\textit{\textbf{UGID}}), an internal-representation--level debiasing framework for large language models that models the Transformer as a structured computational graph, where attention mechanisms define the routing edges of the graph and hidden states define the graph nodes. Specifically, debiasing is formulated as enforcing invariance of the graph structure across counterfactual inputs, with differences allowed only on sensitive attributes. \textit{\textbf{UGID}} jointly constrains attention routing and hidden representations in bias-sensitive regions, effectively preventing bias migration across architectural components. To achieve effective behavioral alignment without degrading general capabilities, we introduce a log-space constraint on sensitive logits and a selective anchor-based objective to preserve definitional semantics. Extensive experiments on large language models demonstrate that \textit{\textbf{UGID}} effectively reduces bias under both in-distribution and out-of-distribution settings, significantly reduces internal structural discrepancies, and preserves model safety and utility.

Related papers

• Exposing the Unsaid: Visualizing Hidden LLM Bias through Stochastic Path Aggregation [16.63373250752545]
  Large Language Models (LLMs) exhibit representational and syntactic biases that are difficult to evaluate due to the marginalization of text generation.<n>This paper introduces TreeTracer, a visual analytics tool designed to evaluate bias through aggregated comparison.
  arXiv  Detail & Related papers  (2026-04-24T13:37:51Z)
• Graph Negative Feedback Bias Correction Framework for Adaptive Heterophily Modeling [15.157616444432563]
  Graph Neural Networks (GNNs) have emerged as a powerful framework for processing graph-structured data.<n>GNNs are inherently limited by the homophily assumption, leading to degradation in performance on heterophilic graphs.<n>We propose Graph Negative Feedback Bias Correction (GNFBC), a simple yet effective framework that is independent of any specific aggregation strategy.
  arXiv  Detail & Related papers  (2026-03-04T02:34:48Z)
• Generative Data Transformation: From Mixed to Unified Data [57.84692191369066]
  textscTaesar is a emphdata-centric framework for textbftarget-textbfal textbfregeneration.<n>It encodes cross-domain context into target sequences, enabling standard models to learn intricate dependencies without complex fusion architectures.
  arXiv  Detail & Related papers  (2026-02-26T08:30:09Z)
• NAG: A Unified Native Architecture for Encoder-free Text-Graph Modeling in Language Models [33.49410203951687]
  We argue this approach is suboptimal for text-graphs.<n>NAG (Native Architecture for Graphs) is a unified framework that internalizes graph processing within the Language Models.<n>NAG achieves robust graph comprehension without the overhead of external encoders.
  arXiv  Detail & Related papers  (2026-01-30T07:22:11Z)
• Semantic Refinement with LLMs for Graph Representations [37.72134125261354]
  We propose a Data-Adaptive Semantic Refinement framework DAS for graph representation learning.<n>We evaluate our approach on both text-rich and text-free graphs.<n>Results show consistent improvements on structure-dominated graphs while remaining competitive on semantics-rich graphs.
  arXiv  Detail & Related papers  (2025-12-24T11:10:28Z)
• Bias Is a Subspace, Not a Coordinate: A Geometric Rethinking of Post-hoc Debiasing in Vision-Language Models [13.46898179372249]
  Vision-Language Models (VLMs) have become indispensable for multimodal reasoning, yet their representations often encode and amplify demographic biases.<n>We propose a geometrically principled framework that identifies and removes the entire subspace of linearly decodable bias.<n>Our method achieves more robust debiasing with an average improvement of $18.5%$ across four fairness metrics.
  arXiv  Detail & Related papers  (2025-11-22T17:04:30Z)
• SCALEX: Scalable Concept and Latent Exploration for Diffusion Models [59.86284983662119]
  Image generation models frequently encode social biases, including stereotypes tied to gender, race, and profession.<n>We introduce SCALEX, a framework for scalable and automated exploration of diffusion model latent spaces.<n>It extracts semantically meaningful directions from H-space using only natural language prompts, enabling zero-shot interpretation without retraining or labelling.
  arXiv  Detail & Related papers  (2025-11-13T22:02:44Z)
• Semantic and Structural Analysis of Implicit Biases in Large Language Models: An Interpretable Approach [1.5749416770494704]
  It proposes an interpretable bias detection method aimed at identifying hidden social biases in model outputs.<n>The method combines nested semantic representation with a contextual contrast mechanism.<n>The evaluation focuses on several key metrics, such as bias detection accuracy, semantic consistency, and contextual sensitivity.
  arXiv  Detail & Related papers  (2025-08-08T09:21:10Z)
• Graph-KV: Breaking Sequence via Injecting Structural Biases into Large Language Models [63.64507678113921]
  We introduce Graph-KV, which governs interaction through structural inductive biases.<n>In this framework, 'target' segments selectively attend only to the KV-cache of their designated'source' segments.<n>We evaluate Graph-KV across three scenarios: (1) seven RAG benchmarks spanning direct inference, multi-hop reasoning, and long-document understanding; (2) Arxiv-QA, a novel academic paper QA task with full-text scientific papers structured as citation ego-graphs; and (3) paper topic classification within a citation network.
  arXiv  Detail & Related papers  (2025-06-09T00:30:08Z)
• Scalable Graph Generative Modeling via Substructure Sequences [50.32639806800683]
  We introduce Generative Graph Pattern Machine (G$2$PM), a generative Transformer pre-training framework for graphs.<n>G$2$PM represents graph instances (nodes, edges, or entire graphs) as sequences of substructures.<n>It employs generative pre-training over the sequences to learn generalizable and transferable representations.
  arXiv  Detail & Related papers  (2025-05-22T02:16:34Z)
• Federated Deconfounding and Debiasing Learning for Out-of-Distribution Generalization [25.181305392387802]
  Attribute bias in federated learning (FL) typically leads local models to optimize inconsistently due to the learning of non-causal associations.<n>We propose the underlineFederated underlineDeconfounding and underlineDebiasing underlineLearning (FedDDL) method.<n>It constructs a structured causal graph to analyze the model inference process, and performs backdoor adjustment to eliminate confounding paths.
  arXiv  Detail & Related papers  (2025-05-08T06:32:59Z)
• Causality and Independence Enhancement for Biased Node Classification [56.38828085943763]
  We propose a novel Causality and Independence Enhancement (CIE) framework, applicable to various graph neural networks (GNNs) Our approach estimates causal and spurious features at the node representation level and mitigates the influence of spurious correlations. Our approach CIE not only significantly enhances the performance of GNNs but outperforms state-of-the-art debiased node classification methods.
  arXiv  Detail & Related papers  (2023-10-14T13:56:24Z)
• Debiasing Vision-Language Models via Biased Prompts [79.04467131711775]
  We propose a general approach for debiasing vision-language foundation models by projecting out biased directions in the text embedding. We show that debiasing only the text embedding with a calibrated projection matrix suffices to yield robust classifiers and fair generative models.
  arXiv  Detail & Related papers  (2023-01-31T20:09:33Z)
• General Greedy De-bias Learning [163.65789778416172]
  We propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model like gradient descent in functional space. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
  arXiv  Detail & Related papers  (2021-12-20T14:47:32Z)
• Bayesian Attention Modules [65.52970388117923]
  We propose a scalable version of attention that is easy to implement and optimize. Our experiments show the proposed method brings consistent improvements over the corresponding baselines.
  arXiv  Detail & Related papers  (2020-10-20T20:30:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.