Unified Multi-Task Learning & Model Fusion for Efficient Language Model Guardrailing

• URL: http://arxiv.org/abs/2504.19333v2
• Date: Tue, 29 Apr 2025 02:42:00 GMT
• Title: Unified Multi-Task Learning & Model Fusion for Efficient Language Model Guardrailing
• Abstract summary: We show that task-specific data generation can lead to fine-tuned classifiers.<n>Using a single model, textttMultiTaskGuard, that is pretrained on a large synthetically generated dataset improves generalization.<n>Our most performant models, textttUniGuard, are found using our proposed search-based model merging approach.
• Score: 7.959705237659548
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The trend towards large language models (LLMs) for guardrailing against undesired behaviors is increasing and has shown promise for censoring user inputs. However, increased latency, memory consumption, hosting expenses and non-structured outputs can make their use prohibitive. In this work, we show that task-specific data generation can lead to fine-tuned classifiers that significantly outperform current state of the art (SoTA) while being orders of magnitude smaller. Secondly, we show that using a single model, \texttt{MultiTaskGuard}, that is pretrained on a large synthetically generated dataset with unique task instructions further improves generalization. Thirdly, our most performant models, \texttt{UniGuard}, are found using our proposed search-based model merging approach that finds an optimal set of parameters to combine single-policy models and multi-policy guardrail models. % On 7 public datasets and 4 guardrail benchmarks we created, our efficient guardrail classifiers improve over the best performing SoTA publicly available LLMs and 3$^{\text{rd}}$ party guardrail APIs in detecting unsafe and safe behaviors by an average F1 score improvement of \textbf{29.92} points over Aegis-LlamaGuard and \textbf{21.62} over \texttt{gpt-4o}, respectively. Lastly, our guardrail synthetic data generation process that uses custom task-specific guardrail poli

Related papers

• CoMerge: Conflict-Driven Preference Optimization for Multi-Task Model Merging [59.836482609414816]
  We propose a conflict-driven preference optimization framework for model merging (CoMerge)<n>CoMerge enables the model to mitigate parameter-space conflicts while preserving task-specific capabilities.<n>Experiments show that CoMerge achieves an average normalized performance of 0.9968 on MergeBench.
  arXiv  Detail & Related papers  (2026-09-02T08:21:28Z)
• Robust LLM-based Column Type Annotation via Prompt Augmentation with LoRA Tuning [0.749377967268953]
  Column Type.<n>CTA is a fundamental step towards enabling schema alignment and semantic understanding.<n>Existing encoder-only language models achieve high accuracy when fine-tuned on labeled columns.<n>We present a parameter-efficient framework for CTA that trains models over prompt-augmented data.
  arXiv  Detail & Related papers  (2025-12-28T02:04:17Z)
• An Efficient Training Pipeline for Reasoning Graphical User Interface Agents [2.553589584067239]
  This work introduces an efficient training pipeline that combines model-based data filtering with parameter-efficient fine-tuning.<n>From 4.8M synthetic examples, 12K clean and diverse instances are curated by first identifying challenging cases, removing misaligned and then selecting a diverse set of multimodal instances.<n>Models trained with the filtered data and lightweight training strategies match or surpass larger baselines on benchmarks such as ScreenSpot, Multimodal-Mind2Web, and AndroidControl.
  arXiv  Detail & Related papers  (2025-11-11T12:36:04Z)
• The Surprising Effectiveness of Membership Inference with Simple N-Gram Coverage [71.8564105095189]
  We introduce N-Gram Coverage Attack, a membership inference attack that relies solely on text outputs from the target model.<n>We first demonstrate on a diverse set of existing benchmarks that N-Gram Coverage Attack outperforms other black-box methods.<n>We find that more recent models, such as GPT-4o, exhibit increased robustness to membership inference.
  arXiv  Detail & Related papers  (2025-08-13T08:35:16Z)
• Secret Breach Detection in Source Code with Large Language Models [2.5484785866796833]
  Leaking sensitive information, such as API keys, tokens, and credentials, in source code remains a persistent security threat.<n>This work aims to enhance secret detection in source code using large language models (LLMs), reducing false positives while maintaining high recall.
  arXiv  Detail & Related papers  (2025-04-26T03:33:14Z)
• SafeRoute: Adaptive Model Selection for Efficient and Accurate Safety Guardrails in Large Language Models [63.63254955809224]
  We propose a binary router that distinguishes hard examples from easy ones.<n>Our method selectively applies the larger safety guard model to the data that the router considers hard, improving efficiency while maintaining accuracy.<n> Experimental results on multiple benchmark datasets demonstrate that our adaptive model selection significantly enhances the trade-off between computational cost and safety performance.
  arXiv  Detail & Related papers  (2025-02-18T02:51:17Z)
• Streamlining the Collaborative Chain of Models into A Single Forward Pass in Generation-Based Tasks [13.254837575157786]
  "Chain of Models" approach is widely used in Retrieval-Augmented Generation (RAG) and agent-based frameworks.<n>Recent advancements attempt to address this by applying prompt tuning, which allows a shared base model to adapt to multiple tasks.<n>In this paper, we introduce FTHSS, a novel prompt-tuning method that enables models to share hidden states.
  arXiv  Detail & Related papers  (2025-02-16T11:37:14Z)
• DuoGuard: A Two-Player RL-Driven Framework for Multilingual LLM Guardrails [12.621656255109546]
  We propose a novel two-player Reinforcement Learning framework, where a generator and a guardrail model co-evolve adversarially to produce high-quality synthetic data for multilingual guardrail training.<n> Empirical evaluations show that our model ours outperforms state-of-the-art models, achieving nearly 10% improvement over LlamaGuard3 on English benchmarks.
  arXiv  Detail & Related papers  (2025-02-07T18:45:03Z)
• Practical Secure Inference Algorithm for Fine-tuned Large Language Model Based on Fully Homomorphic Encryption [0.0]
  We combine Fully Homomorphic Encryption(FHE) and provable security theory with Fine-Tuning(PEFT) to propose an efficient and secure inference scheme for large language models.<n>In this paper, we use the open-source model ChatGLM2-6B as the base model which is fine-tuned by LoRA.<n> Experimental results show the inference efficiency of our scheme reaches 1.61s/ which displays that the scheme has good practicality.
  arXiv  Detail & Related papers  (2025-01-03T07:19:23Z)
• Label Privacy in Split Learning for Large Models with Parameter-Efficient Training [51.28799334394279]
  We search for a way to fine-tune models over an API while keeping the labels private.<n>We propose P$3$EFT, a multi-party split learning algorithm that takes advantage of existing PEFT properties to maintain privacy at a lower performance overhead.
  arXiv  Detail & Related papers  (2024-12-21T15:32:03Z)
• Zeroth-Order Adaptive Neuron Alignment Based Pruning without Re-Training [3.195234044113248]
  We propose textscNeuroAL, a emphtop-up algorithm for network pruning.<n>It modifies the block-wise and row-wise sparsity exploiting information from both the dense model and its sparse version.<n>It consistently outperforms the latest state-of-the-art methods in terms of performance-runtime trade-off.
  arXiv  Detail & Related papers  (2024-11-11T15:30:16Z)
• Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration [90.41908331897639]
  Large language models (LLMs) have significantly benefited from training on diverse, high-quality task-specific data. We present a novel approach, ReverseGen, designed to automatically generate effective training samples.
  arXiv  Detail & Related papers  (2024-10-22T06:43:28Z)
• When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models [61.363259848264725]
  Inheritune is a simple and effective training recipe for building smaller, more efficient language models.<n>We show that Inheritune trained models, despite having significantly fewer layers, can match or even outperform their larger counterparts.
  arXiv  Detail & Related papers  (2024-04-12T17:53:34Z)
• Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors [44.5740422079]
  We show that pretraining with standard denoising objectives leads to dramatic gains across multiple architectures. In stark contrast to prior works, we find vanilla Transformers to match the performance of S4 on Long Range Arena when properly pretrained.
  arXiv  Detail & Related papers  (2023-10-04T17:17:06Z)
• Cross-Modal Adapter for Text-Video Retrieval [91.9575196703281]
  We present a novel $textbfCross-Modal Adapter$ for parameter-efficient fine-tuning. Inspired by adapter-based methods, we adjust the pre-trained model with a few parameterization layers. It achieves superior or comparable performance compared to fully fine-tuned methods on MSR-VTT, MSVD, VATEX, ActivityNet, and DiDeMo datasets.
  arXiv  Detail & Related papers  (2022-11-17T16:15:30Z)
• Just Fine-tune Twice: Selective Differential Privacy for Large Language Models [69.66654761324702]
  We propose a simple yet effective just-fine-tune-twice privacy mechanism to achieve SDP for large Transformer-based language models. Experiments show that our models achieve strong performance while staying robust to the canary insertion attack.
  arXiv  Detail & Related papers  (2022-04-15T22:36:55Z)
• Improving Contrastive Learning with Model Augmentation [123.05700988581806]
  The sequential recommendation aims at predicting the next items in user behaviors, which can be solved by characterizing item relationships in sequences. Due to the data sparsity and noise issues in sequences, a new self-supervised learning (SSL) paradigm is proposed to improve the performance.
  arXiv  Detail & Related papers  (2022-03-25T06:12:58Z)
• DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models [152.29364079385635]
  As pre-trained models grow bigger, the fine-tuning process can be time-consuming and computationally expensive. We propose a framework for resource- and parameter-efficient fine-tuning by leveraging the sparsity prior in both weight updates and the final model weights. Our proposed framework, dubbed Dually Sparsity-Embedded Efficient Tuning (DSEE), aims to achieve two key objectives: (i) parameter efficient fine-tuning and (ii) resource-efficient inference.
  arXiv  Detail & Related papers  (2021-10-30T03:29:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.