Quantization with Unified Adaptive Distillation to enable multi-LoRA based one-for-all Generative Vision Models on edge

• URL: http://arxiv.org/abs/2603.29535v1
• Date: Tue, 31 Mar 2026 10:17:07 GMT
• Title: Quantization with Unified Adaptive Distillation to enable multi-LoRA based one-for-all Generative Vision Models on edge
• Abstract summary: GenAI features such as image editing, object removal, and prompt-guided image transformation are increasingly integrated into mobile applications.<n>Existing Mobile deployment pipelines typically compile separate model binaries for each Low-Rank Adapters.<n>We present a unified framework for enabling multi-task GenAI inference on edge devices using a single shared model.
• Score: 4.632054706878866
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative Artificial Intelligence (GenAI) features such as image editing, object removal, and prompt-guided image transformation are increasingly integrated into mobile applications. However, deploying Large Vision Models (LVMs) for such tasks on resource-constrained devices remains challenging due to their high memory and compute requirements. While Low-Rank Adapters (LoRAs) enable parameter-efficient task adaptation, existing Mobile deployment pipelines typically compile separate model binaries for each LoRA + a copy of the foundation model, resulting in redundant storage and increased runtime overhead. In this work, we present a unified framework for enabling multi-task GenAI inference on edge devices using a single shared model. Our key idea is to treat LoRA weights as runtime inputs rather than embedding them into the compiled model graph, allowing dynamic task switching at runtime without recompilation. Then, to support efficient on-device execution, we introduce QUAD (Quantization with Unified Adaptive Distillation), a quantizationaware training strategy that aligns multiple LoRA adapters under a shared quantization profile. We implement the proposed system with a lightweight runtime stack compatible with mobile NPUs and evaluate it across multiple chipsets. Experimental results demonstrate up to 6x and 4x reduction in memory footprint and latency improvements, respectively, while maintaining high visual quality across multiple GenAI tasks.

Related papers

• UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling [57.126059929740926]
  We study Unified Visual Generation (UVG), where a single model produces diverse image-valued outputs through a unified multimodal interface.<n>We propose UniGen-AR, a framework that pairs a general-purpose multi-modal language model (MLLM) with an efficient visual auto-regressive decoder.<n>In our framework, the MLLM encodes free-form instructions and control signals into a unified sequence, which guides the VAR decoder to generate image-valued outputs for over 15 tasks spanning four families.
  arXiv  Detail & Related papers  (2026-07-27T08:35:09Z)
• Towards Multi-Model LLM Schedulers: Empirical Insights into Offloading and Preemption [3.6129975381123107]
  We show how different Large Language Models behave across hardware platforms, focusing on the performance implications of layer offloading and preemption.<n>We identify a set of key features that future schedulers must consider, including model-specific offloading sensitivity, workload characteristics, and the cost structure of preemption and data transfer.
  arXiv  Detail & Related papers  (2026-05-19T09:39:16Z)
• Co-Training Vision Language Models for Remote Sensing Multi-task Learning [68.15604397741753]
  Vision language models (VLMs) have achieved promising results in RS image understanding, grounding, and ultra-high-resolution (UHR) image reasoning.<n>We present RSCoVLM, a simple yet flexible VLM baseline for RS MTL.<n>We propose a unified dynamic-resolution strategy to address the diverse image scales inherent in RS imagery.
  arXiv  Detail & Related papers  (2025-11-26T10:55:07Z)
• Visual Perception Engine: Fast and Flexible Multi-Head Inference for Robotic Vision Tasks [6.943057640797408]
  Visual Perception Engine (VPEngine) is a modular framework designed to enable efficient GPU usage for visual multitasking while maintaining developer accessibility.<n>Our framework architecture leverages a shared foundation model backbone that extracts image representations, which are efficiently shared, across multiple specialized task-specific model heads running in parallel.<n>Our example implementation demonstrates end-to-end real-time performance at $geq$50 Hz on NVIDIA Jetson Orin AGX forRT optimized models.
  arXiv  Detail & Related papers  (2025-08-15T16:42:23Z)
• AutoLoRA: Automatic LoRA Retrieval and Fine-Grained Gated Fusion for Text-to-Image Generation [32.46570968627392]
  Low-rank adaptation (LoRA) have demonstrated efficacy in enabling model customization with minimal parameter overhead.<n>We introduce a novel framework that enables semantic-driven LoRA retrieval and dynamic aggregation.<n>Our approach achieves significant improvement in image generation perfermance.
  arXiv  Detail & Related papers  (2025-08-04T06:36:00Z)
• Clip4Retrofit: Enabling Real-Time Image Labeling on Edge Devices via Cross-Architecture CLIP Distillation [1.5181915291533354]
  We propose Clip4Retrofit, an efficient model distillation framework that enables real-time image labeling on edge devices.<n>The framework is deployed on the Retrofit camera, a cost-effective edge device retrofitted into thousands of vehicles.<n>This work bridges the gap between state-of-the-art vision-language models and their deployment in resource-constrained environments.
  arXiv  Detail & Related papers  (2025-05-23T15:42:52Z)
• LoRAX: LoRA eXpandable Networks for Continual Synthetic Image Attribution [0.0]
  We propose LoRAX, a class incremental algorithm that adapts to novel generative image models without the need for full retraining.<n>Our approach trains an extremely parameter-efficient feature extractor per continual learning task via Low Rank Adaptation.<n>LoRAX outperforms or remains competitive with state-of-the-art class incremental learning algorithms on the Continual Deepfake Detection benchmark.
  arXiv  Detail & Related papers  (2025-04-10T22:20:00Z)
• Marmot: Object-Level Self-Correction via Multi-Agent Reasoning [55.74860093731475]
  Marmot is a novel and generalizable framework that leverages Multi-Agent Reasoning for Multi-Object Self-Correcting.<n>Marmot significantly improves accuracy in object counting, attribute assignment, and spatial relationships for image generation tasks.
  arXiv  Detail & Related papers  (2025-04-10T16:54:28Z)
• Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model [83.85856356798531]
  VistaLLM is a visual system that addresses coarse- and fine-grained vision-language tasks. It employs a gradient-aware adaptive sampling technique to represent binary segmentation masks as sequences. We also introduce a novel task, AttCoSeg, which boosts the model's reasoning and grounding capability over multiple input images.
  arXiv  Detail & Related papers  (2023-12-19T18:53:01Z)
• ViR: Towards Efficient Vision Retention Backbones [97.93707844681893]
  We propose a new class of computer vision models, dubbed Vision Retention Networks (ViR) ViR has dual parallel and recurrent formulations, which strike an optimal balance between fast inference and parallel training with competitive performance. We have validated the effectiveness of ViR through extensive experiments with different dataset sizes and various image resolutions.
  arXiv  Detail & Related papers  (2023-10-30T16:55:50Z)
• An Efficient General-Purpose Modular Vision Model via Multi-Task Heterogeneous Training [79.78201886156513]
  We present a model that can perform multiple vision tasks and can be adapted to other downstream tasks efficiently. Our approach achieves comparable results to single-task state-of-the-art models and demonstrates strong generalization on downstream tasks.
  arXiv  Detail & Related papers  (2023-06-29T17:59:57Z)
• Energy-efficient Task Adaptation for NLP Edge Inference Leveraging Heterogeneous Memory Architectures [68.91874045918112]
  adapter-ALBERT is an efficient model optimization for maximal data reuse across different tasks. We demonstrate the advantage of mapping the model to a heterogeneous on-chip memory architecture by performing simulations on a validated NLP edge accelerator.
  arXiv  Detail & Related papers  (2023-03-25T14:40:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.