ZKLoRA: Efficient Zero-Knowledge Proofs for LoRA Verification

• URL: http://arxiv.org/abs/2501.13965v1
• Date: Tue, 21 Jan 2025 23:20:33 GMT
• Title: ZKLoRA: Efficient Zero-Knowledge Proofs for LoRA Verification
• Authors: Bidhan Roy, Peter Potash, Marcos Villagra,
• Abstract summary: Low-Rank Adaptation (LoRA) is a widely adopted method for customizing large-scale language models.<n>In distributed, untrusted training environments, an open source base model user may want to use LoRA weights created by an external contributor.<n>We present ZKLoRA, a zero-knowledge verification protocol that relies on succinct proofs and our novel Multi-Party Inference procedure.
• Score: 0.20482269513546458
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Low-Rank Adaptation (LoRA) is a widely adopted method for customizing large-scale language models. In distributed, untrusted training environments, an open source base model user may want to use LoRA weights created by an external contributor, leading to two requirements: (1) the base model user must confirm that the LoRA weights are effective when paired with the intended base model, and (2) the LoRA contributor must keep their proprietary weights private until compensation is assured. We present ZKLoRA, a zero-knowledge verification protocol that relies on succinct proofs and our novel Multi-Party Inference procedure to verify LoRA-base model compatibility without exposing LoRA weights. ZKLoRA produces deterministic correctness guarantees and validates each LoRA module in only 1-2 seconds on state-of-the-art large language models. This low-latency approach enables nearly real-time verification and promotes secure collaboration among geographically decentralized teams and contract-based training pipelines. The protocol ensures that the delivered LoRA module works as claimed, safeguarding the contributor's intellectual property while providing the base model user with verification of compatibility and lineage.

Related papers

• Cross-LoRA: A Data-Free LoRA Transfer Framework across Heterogeneous LLMs [10.218401136555064]
  Cross-LoRA is a framework for transferring LoRA modules between diverse base models.<n>Experiments show that Cross-LoRA achieves relative gains of up to 5.26% over base models.
  arXiv  Detail & Related papers  (2025-08-07T10:21:08Z)
• LoRAShield: Data-Free Editing Alignment for Secure Personalized LoRA Sharing [43.88211522311429]
  Low-Rank Adaptation (LoRA) models can be shared on platforms like Civitai and Liblib.<n>LoRAShield is the first data-free editing framework for securing LoRA models against misuse.
  arXiv  Detail & Related papers  (2025-07-05T02:53:17Z)
• LoRA-Gen: Specializing Large Language Model via Online LoRA Generation [68.01864057372067]
  We propose the LoRA-Gen framework to generate LoRA parameters for edge-side models based on task descriptions.<n>We merge the LoRA parameters into the edge-side model to achieve flexible specialization.<n>Our method facilitates knowledge transfer between models while significantly improving the inference efficiency of the specialized model.
  arXiv  Detail & Related papers  (2025-06-13T10:11:01Z)
• AC-LoRA: (Almost) Training-Free Access Control-Aware Multi-Modal LLMs [1.8633238548765558]
  AC-LoRA is an end-to-end system for access control-aware corporate LLMs.<n>AC-LoRA maintains separate LoRA adapters for permissioned datasets, along with the document embedding they are finetuned on.<n>AC-LoRA retrieves a precise set of LoRA adapters based on the similarity score with the user query and their permission.
  arXiv  Detail & Related papers  (2025-05-15T23:19:35Z)
• Activated LoRA: Fine-tuned LLMs for Intrinsics [9.503174205896533]
  Low-Rank Adaptation (LoRA) has emerged as a highly efficient framework for finetuning the weights of large foundation models. We propose Activated LoRA (aLoRA), which modifies the LoRA framework to only adapt weights for the tokens in the sequence emphafter the aLoRA is invoked. This change crucially allows aLoRA to accept the base model's KV cache of the input string, meaning that aLoRA can be instantly activated whenever needed in a chain.
  arXiv  Detail & Related papers  (2025-04-16T18:03:21Z)
• SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning [73.93639228235622]
  Continual Learning with foundation models has emerged as a promising paradigm to exploit abundant knowledge acquired during pre-training for tackling sequential tasks. Existing prompt-based and Low-Rank Adaptation-based (LoRA-based) methods often require expanding a prompt/LoRA pool or retaining samples of previous tasks. We propose Scalable Decoupled LoRA (SD-LoRA) for class incremental learning, which continually separates the learning of the magnitude and direction of LoRA components without rehearsal.
  arXiv  Detail & Related papers  (2025-01-22T20:00:41Z)
• LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models [15.900125475191958]
  Guardrails have emerged as an alternative to safety alignment for content moderation of large language models (LLMs)<n>We introduce LoRA-Guard, a parameter-efficient guardrail adaptation method that relies on knowledge sharing between LLMs and guardrail models.<n>We show that LoRA-Guard outperforms existing approaches with 100-1000x lower parameter overhead while maintaining accuracy, enabling on-device content moderation.
  arXiv  Detail & Related papers  (2024-07-03T10:38:40Z)
• Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning [57.36978335727009]
  Low-Rank Adaptation (LoRA) offers an efficient way to fine-tune large language models (LLMs) In this paper, we propose a framework that adaptively retrieves and composes multiple LoRAs based on input prompts.
  arXiv  Detail & Related papers  (2024-06-24T05:24:41Z)
• Unlocking the Global Synergies in Low-Rank Adapters [20.32980343066711]
  Low-rank Adaption (LoRA) has been the de-facto parameter-efficient fine-tuning technique for large language models. We present HeteroLoRA, a light-weight search algorithm that leverages zero-cost proxies to allocate the limited LoRA trainable parameters. Experiments show that HeteroLoRA enables improvements in model performance given the same parameter budge.
  arXiv  Detail & Related papers  (2024-06-21T08:10:03Z)
• Mixture of LoRA Experts [87.50120181861362]
  This paper introduces the Mixture of LoRA Experts (MoLE) approach, which harnesses hierarchical control and unfettered branch selection. The MoLE approach achieves superior LoRA fusion performance in comparison to direct arithmetic merging.
  arXiv  Detail & Related papers  (2024-04-21T11:59:53Z)
• Continual Forgetting for Pre-trained Vision Models [70.51165239179052]
  In real-world scenarios, selective information is expected to be continuously removed from a pre-trained model. We propose Group Sparse LoRA (GS-LoRA) for efficient and effective deleting. We conduct extensive experiments on face recognition, object detection and image classification and demonstrate that GS-LoRA manages to forget specific classes with minimal impact on other classes.
  arXiv  Detail & Related papers  (2024-03-18T07:33:56Z)
• LoRA-as-an-Attack! Piercing LLM Safety Under The Share-and-Play Scenario [61.99243609126672]
  We study how to inject backdoor into the LoRA module and dive deeper into LoRA's infection mechanisms. Our aim is to raise awareness of the potential risks under the emerging share-and-play scenario, so as to proactively prevent potential consequences caused by LoRA-as-an-Attack.
  arXiv  Detail & Related papers  (2024-02-29T20:25:16Z)
• LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild [76.67343971195267]
  Low-Rank Adaptation (LoRA) provides an efficient solution for fine-tuning large language models (LLM) LoraRetriever is a retrieve-then-compose framework that adaptively retrieves and composes multiple LoRAs according to the input prompts. Experimental results indicate that LoraRetriever consistently outperforms the baselines.
  arXiv  Detail & Related papers  (2024-02-15T15:02:46Z)
• CA-LoRA: Adapting Existing LoRA for Compressed LLMs to Enable Efficient Multi-Tasking on Personal Devices [78.16679232748196]
  We introduce a Compression-Aware LoRA (CA-LoRA) framework to transfer Large Language Models (LLMs) to other tasks. Experiment results demonstrate that CA-LoRA outperforms the vanilla LoRA methods applied to a compressed LLM. The source code of CA-LoRA is available at https://github.com/thunlp/CA-LoRA.
  arXiv  Detail & Related papers  (2023-07-15T04:37:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.