LoRAX: LoRA eXpandable Networks for Continual Synthetic Image Attribution

• URL: http://arxiv.org/abs/2504.08149v1
• Date: Thu, 10 Apr 2025 22:20:00 GMT
• Title: LoRAX: LoRA eXpandable Networks for Continual Synthetic Image Attribution
• Authors: Danielle Sullivan-Pao, Nicole Tian, Pooya Khorrami,
• Abstract summary: We propose LoRAX, a class incremental algorithm that adapts to novel generative image models without the need for full retraining.<n>Our approach trains an extremely parameter-efficient feature extractor per continual learning task via Low Rank Adaptation.<n>LoRAX outperforms or remains competitive with state-of-the-art class incremental learning algorithms on the Continual Deepfake Detection benchmark.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As generative AI image technologies become more widespread and advanced, there is a growing need for strong attribution models. These models are crucial for verifying the authenticity of images and identifying the architecture of their originating generative models-key to maintaining media integrity. However, attribution models struggle to generalize to unseen models, and traditional fine-tuning methods for updating these models have shown to be impractical in real-world settings. To address these challenges, we propose LoRA eXpandable Networks (LoRAX), a parameter-efficient class incremental algorithm that adapts to novel generative image models without the need for full retraining. Our approach trains an extremely parameter-efficient feature extractor per continual learning task via Low Rank Adaptation. Each task-specific feature extractor learns distinct features while only requiring a small fraction of the parameters present in the underlying feature extractor's backbone model. Our extensive experimentation shows LoRAX outperforms or remains competitive with state-of-the-art class incremental learning algorithms on the Continual Deepfake Detection benchmark across all training scenarios and memory settings, while requiring less than 3% of the number of trainable parameters per feature extractor compared to the full-rank implementation. LoRAX code is available at: https://github.com/mit-ll/lorax_cil.

Related papers

• PointLoRA: Low-Rank Adaptation with Token Selection for Point Cloud Learning [54.99373314906667]
  Self-supervised representation learning for point cloud has demonstrated effectiveness in improving pre-trained model performance across diverse tasks. As pre-trained models grow in complexity, fully fine-tuning them for downstream applications demands substantial computational and storage resources. We propose PointLoRA, a simple yet effective method that combines low-rank adaptation (LoRA) with multi-scale token selection to efficiently fine-tune point cloud models.
  arXiv  Detail & Related papers  (2025-04-22T16:41:21Z)
• Any Image Restoration via Efficient Spatial-Frequency Degradation Adaptation [158.37640586809187]
  Restoring any degraded image efficiently via just one model has become increasingly significant. Our approach, termed AnyIR, takes a unified path that leverages inherent similarity across various degradations. To fuse the degradation awareness and the contextualized attention, a spatial-frequency parallel fusion strategy is proposed.
  arXiv  Detail & Related papers  (2025-04-19T09:54:46Z)
• Few-Shot Class-Incremental Model Attribution Using Learnable Representation From CLIP-ViT Features [1.534667887016089]
  This work proposes a new strategy to deal with persistently emerging generative models. We adapt few-shot class-incremental learning (FSCIL) mechanisms for MA problem to uncover novel generative AI models. To learn an effective representation, we propose Adaptive Integration Module (AIM) to calculate a weighted sum of CLIP-ViT block features for each image.
  arXiv  Detail & Related papers  (2025-03-11T08:05:26Z)
• Recurrent Diffusion for Large-Scale Parameter Generation [52.98888368644455]
  We introduce Recurrent Diffusion for Large Scale Generation (RPG), a novel framework that generates full neural network parameters up to hundreds of millions on a single GPU.<n>RPG serves as a critical advance in AI generating AI, potentially enabling efficient weight generation at scales previously deemed infeasible.
  arXiv  Detail & Related papers  (2025-01-20T16:46:26Z)
• VELoRA: A Low-Rank Adaptation Approach for Efficient RGB-Event based Recognition [54.27379947727035]
  This paper proposes a novel PEFT strategy to adapt the pre-trained foundation vision models for the RGB-Event-based classification.<n>The frame difference of the dual modalities is also considered to capture the motion cues via the frame difference backbone network.<n>The source code and pre-trained models will be released on urlhttps://github.com/Event-AHU/VELoRA.
  arXiv  Detail & Related papers  (2024-12-28T07:38:23Z)
• LoRA Unlearns More and Retains More (Student Abstract) [0.0]
  PruneLoRA reduces the need for large-scale parameter updates by applying low-rank updates to the model. We leverage LoRA to selectively modify a subset of the pruned model's parameters, thereby reducing the computational cost, memory requirements and improving the model's ability to retain performance on the remaining classes.
  arXiv  Detail & Related papers  (2024-11-16T16:47:57Z)
• In-Context LoRA for Diffusion Transformers [49.288489286276146]
  We show that text-to-image DiTs can effectively perform in-context generation without any tuning. We name our models In-Context LoRA (IC-LoRA) Our pipeline generates high-fidelity image sets that better adhere to prompts.
  arXiv  Detail & Related papers  (2024-10-31T09:45:00Z)
• Drama: Mamba-Enabled Model-Based Reinforcement Learning Is Sample and Parameter Efficient [9.519619751861333]
  We propose a state space model (SSM)-based world model, Drama, that achieves $O(n)$ memory and computational complexity.<n>We also introduce a novel sampling method to mitigate the suboptimality caused by an incorrect world model in the early training stages.<n>Drama is accessible and trainable on off-the-shelf hardware, such as a standard laptop.
  arXiv  Detail & Related papers  (2024-10-11T15:10:40Z)
• HyperDet: Generalizable Detection of Synthesized Images by Generating and Merging A Mixture of Hyper LoRAs [17.88153857572688]
  We introduce a novel and generalizable detection framework termed HyperDet. In this work, we propose a novel objective function that balances the pixel and semantic artifacts effectively. Our work paves a new way to establish generalizable domain-specific fake image detectors based on pretrained large vision models.
  arXiv  Detail & Related papers  (2024-10-08T13:43:01Z)
• Restore Anything Model via Efficient Degradation Adaptation [129.38475243424563]
  RAM takes a unified path that leverages inherent similarities across various degradations to enable efficient and comprehensive restoration.<n> RAM's SOTA performance confirms RAM's SOTA performance, reducing model complexity by approximately 82% in trainable parameters and 85% in FLOPs.
  arXiv  Detail & Related papers  (2024-07-18T10:26:53Z)
• Fine-tuning a Multiple Instance Learning Feature Extractor with Masked Context Modelling and Knowledge Distillation [0.21756081703275998]
  We propose to increase downstream MIL classification by fine-tuning the feature extractor model using itMasked Context Modelling with Knowledge Distillation. A single epoch of the proposed task suffices to increase the downstream performance of the feature-extractor model when used in a MIL scenario, while being considerably smaller and requiring a fraction of its compute.
  arXiv  Detail & Related papers  (2024-03-08T14:04:30Z)
• Training Neural Networks from Scratch with Parallel Low-Rank Adapters [46.764982726136054]
  We introduce LoRA-the-Explorer (LTE), a novel bi-level optimization algorithm designed to enable parallel training of multiple low-rank heads across computing nodes. Our approach includes extensive experimentation on vision transformers using various vision datasets, demonstrating that LTE is competitive with standard pre-training.
  arXiv  Detail & Related papers  (2024-02-26T18:55:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.