CatRAG: Functor-Guided Structural Debiasing with Retrieval Augmentation for Fair LLMs

• URL: http://arxiv.org/abs/2603.21524v1
• Date: Mon, 23 Mar 2026 03:33:13 GMT
• Title: CatRAG: Functor-Guided Structural Debiasing with Retrieval Augmentation for Fair LLMs
• Abstract summary: We propose CatRAG Debiasing, a dual-pronged framework that integrates functor with Retrieval-Augmented Generation (RAG) guided structural debiasing.<n>On the Bias Benchmark for Question Answering (BBQ) across three open-source LLMs (Meta Llama-3, OpenAI GPT-OSS, and Google Gemma-3), CatRAG achieves accuracy by up to 40% over the corresponding base models and by more than 10% over prior debiasing methods.
• Score: 1.7697598440512727
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) are deployed in high-stakes settings but can show demographic, gender, and geographic biases that undermine fairness and trust. Prior debiasing methods, including embedding-space projections, prompt-based steering, and causal interventions, often act at a single stage of the pipeline, resulting in incomplete mitigation and brittle utility trade-offs under distribution shifts. We propose CatRAG Debiasing, a dual-pronged framework that integrates functor with Retrieval-Augmented Generation (RAG) guided structural debiasing. The functor component leverages category-theoretic structure to induce a principled, structure-preserving projection that suppresses bias-associated directions in the embedding space while retaining task-relevant semantics. On the Bias Benchmark for Question Answering (BBQ) across three open-source LLMs (Meta Llama-3, OpenAI GPT-OSS, and Google Gemma-3), CatRAG achieves state-of-the-art results, improving accuracy by up to 40% over the corresponding base models and by more than 10% over prior debiasing methods, while reducing bias scores to near zero (from 60% for the base models) across gender, nationality, race, and intersectional subgroups.

Related papers

• Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias [28.271298519059034]
  We argue that the same biases admit a representation-level account in the judge's hidden state.<n>We report three findings, across seven judges, seven bias types, and nine benchmarks.
  arXiv  Detail & Related papers  (2026-07-13T17:55:19Z)
• Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglement [66.67463557274358]
  Foundation models trained on biased datasets often rely on spurious correlations between target labels and non-causal attributes.<n>We introduce Confidence-guided Concept Mining (CBCM) to identify reliable spurious attributes without attribute annotations.<n>We propose Dual-branch Cross-projection Debiasing (DCD), a prompt-tuning framework that separates target and spurious representations into two branches.
  arXiv  Detail & Related papers  (2026-06-23T05:28:47Z)
• DebiasRAG: A Tuning-Free Path to Fair Generation in Large Language Models through Retrieval-Augmented Generation [25.854582760304726]
  Large language models (LLMs) have achieved unprecedented success due to their exceptional generative capabilities.<n>DebiasRAG is a tuning-free and dynamic query-specific debiasing framework based on retrieval-augmented generation (RAG)
  arXiv  Detail & Related papers  (2026-05-15T15:58:10Z)
• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• Position: LLMs Must Use Functor-Based and RAG-Driven Bias Mitigation for Fairness [0.09558392439655011]
  This paper advocates addressing demographic and gender biases in large language models (LLMs)<n>By combining structural debiasing through functor-based mappings and contextual grounding via RAG, we outline a comprehensive framework capable of delivering equitable and fair model outputs.
  arXiv  Detail & Related papers  (2026-03-07T22:40:44Z)
• SimGR: Escaping the Pitfalls of Generative Decoding in LLM-based Recommendation [68.00727783181289]
  A core objective in recommender systems is to accurately model the distribution of user preferences over items to enable personalized recommendations.<n>We observe that existing methods inevitably introduce systematic bias when estimating item-level preference distributions.<n>We propose textbfSimply textbfGenerative textbfRecommendation (textbfSimGR), a framework that directly models item-level preference distributions in a shared latent space.
  arXiv  Detail & Related papers  (2026-02-08T07:26:52Z)
• Normalized Attention Guidance: Universal Negative Guidance for Diffusion Models [57.20761595019967]
  We present Normalized Attention Guidance (NAG), an efficient, training-free mechanism that applies extrapolation in attention space with L1-based normalization and refinement.<n>NAG restores effective negative guidance where CFG collapses while maintaining fidelity.<n>NAG generalizes across architectures (UNet, DiT), sampling regimes (few-step, multi-step), and modalities (image, video)
  arXiv  Detail & Related papers  (2025-05-27T13:30:46Z)
• Evaluating and Mitigating Social Bias for Large Language Models in Open-ended Settings [18.318232025355524]
  We extend an existing dataset BBQ to Open-BBQ to evaluate the social bias of LLMs in open-ended settings.<n>We develop an evaluation process to detect biases from open-ended content by labeling sentences and paragraphs.<n>In order to solve this issue, we propose Composite Prompting, an In-context Learning (ICL) method combining structured examples with explicit chain-of-thought reasoning.
  arXiv  Detail & Related papers  (2024-12-09T01:29:47Z)
• Identifying and Mitigating Social Bias Knowledge in Language Models [52.52955281662332]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases.<n>FAST surpasses state-of-the-art baselines with superior debiasing performance.<n>This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Conceptor-Aided Debiasing of Large Language Models [1.0435741631709405]
  Pre-trained large language models (LLMs) reflect the inherent social biases of their training corpus. We use conceptors--a soft projection method--to identify and remove the bias subspace in LLMs such as BERT and GPT. We propose two methods of applying conceptors (1) bias subspace projection by post-processing by the conceptor NOT operation; and (2) a new architecture, conceptor-intervened BERT (CI-BERT)
  arXiv  Detail & Related papers  (2022-11-20T21:24:48Z)
• Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning [122.62311703151215]
  Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations. DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
  arXiv  Detail & Related papers  (2022-11-12T09:21:49Z)
• General Greedy De-bias Learning [163.65789778416172]
  We propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model like gradient descent in functional space. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
  arXiv  Detail & Related papers  (2021-12-20T14:47:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.