LoRA-based Parameter-Efficient LLMs for Continuous Learning in Edge-based Malware Detection

• URL: http://arxiv.org/abs/2602.11655v1
• Date: Thu, 12 Feb 2026 07:20:26 GMT
• Title: LoRA-based Parameter-Efficient LLMs for Continuous Learning in Edge-based Malware Detection
• Abstract summary: We present a continuous learning architecture for malware detection on edge devices.<n>Lightweight transformer models (DistilBERT, DistilGPT-2, TinyT5) run on edge nodes and are incrementally fine-tuned on device-specific traffic.<n>We evaluate on two public IoT security datasets, Edge-IIoTset and TON-IoT, under multi-round learning to simulate evolving threats.
• Score: 3.676378009156675
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The proliferation of edge devices has created an urgent need for security solutions capable of detecting malware in real time while operating under strict computational and memory constraints. Recently, Large Language Models (LLMs) have demonstrated remarkable capabilities in recognizing complex patterns, yet their deployment on edge devices remains impractical due to their resource demands. However, in edge malware detection, static or centrally retrained models degrade under evolving threats and heterogeneous traffic; locally trained models become siloed and fail to transfer across domains. To overcome these limitations, in this paper, we present a continuous learning architecture for edge-based malware detection that combines local adaptation on each device with global knowledge sharing through parameter-efficient LoRA adapters. Lightweight transformer models (DistilBERT, DistilGPT-2, TinyT5) run on edge nodes and are incrementally fine-tuned on device-specific traffic; only the resulting LoRA modules are aggregated by a lightweight coordinator and redistributed, enabling cross-device generalization without exchanging raw data. We evaluate on two public IoT security datasets, Edge-IIoTset and TON-IoT, under multi-round learning to simulate evolving threats. Compared to isolated fine-tuning, the LoRA-based exchange yields up to 20-25% accuracy gains when models encounter previously unseen attacks from another domain, while maintaining stable loss and F1 across rounds. LoRA adds less than 1% to model size (~0.6-1.8 MB), making updates practical for constrained edge hardware.

Related papers

• Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challenges [58.887152635127165]
  Diffusion language models (DLMs) offer a non-autoregressive alternative for mobile edge agentic artificial intelligence (AI)<n>DLMs can update multiple uncertain tokens in parallel and exploit bidirectional context throughout the generation process.<n>This survey reviews DLM foundations and analyzes their suitability for edge settings under latency, memory, energy, bandwidth, privacy, and reliability constraints.
  arXiv  Detail & Related papers  (2026-09-04T06:16:49Z)
• Localized LoRA-MoE: Block-wise Low-Rank Experts With Adaptive Routing [12.174849960744211]
  We introduce Localized LoRA-MoE, a unified framework that fuses localized spatial blocking with dynamic, context-efficient routing.<n>Our empirical results establish that decentralized cell-level gating achieves complete statistical parity with an omniscient global coordinator.
  arXiv  Detail & Related papers  (2026-07-06T14:06:18Z)
• FLARE: Diffusion for Hybrid Language Model [72.60770374799634]
  FLARE is a systematic conversion framework for hybrid-attention large language models.<n>It combines a token-equal AR-and-diffusion objective, hardware-aware kernels, and unified inference, enabling one checkpoint to support both AR-style verified decoding and diffusion-style parallel denoising.<n>Our results suggest practical dLLMs are limited not only by decoding algorithms, but also by transfer data quality and the training inefficiency of current block-diffusion objectives.
  arXiv  Detail & Related papers  (2026-06-01T06:58:15Z)
• A Comparative Analysis of Machine Learning Models for Intrusion Detection in Intelligent Transport Systems [0.0]
  AI at the edge computing nodes plays a crucial role in protecting against sophisticated threats.<n>This research applies edge computing to improve latency, bandwidth efficiency, and service responsiveness.<n>It also expands the cyberattack surface because edge nodes are distributed, heterogeneous, and often resource-constrained.
  arXiv  Detail & Related papers  (2026-04-30T22:41:34Z)
• Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA [61.12136997430116]
  Decentralized federated learning (DFL) based on low-rank adaptation (LoRA) enables mobile devices with multi-task datasets to collaboratively fine-tune a large language model (LLM) by exchanging locally updated parameters with a subset of neighboring devices via wireless connections for knowledge integration.<n> directly aggregating parameters fine-tuned on heterogeneous datasets induces three primary issues across the DFL life-cycle: (i) catastrophic knowledge forgetting during fine-tuning process, arising from conflicting update directions caused by data heterogeneity; (ii) textitinefficient communication and convergence during model aggregation process,
  arXiv  Detail & Related papers  (2026-02-24T02:45:32Z)
• Decomposing and Composing: Towards Efficient Vision-Language Continual Learning via Rank-1 Expert Pool in a Single LoRA [50.97792275353563]
  We introduce a novel framework that restructures a single Low-Rank Adaptation (LoRA) module as a decomposable Rank-1 Expert Pool.<n>Our method learns to dynamically compose a sparse, task-specific update by selecting from this expert pool, guided by the semantics of the [Guided] token.
  arXiv  Detail & Related papers  (2026-01-30T10:54:51Z)
• Lightweight LLMs for Network Attack Detection in IoT Networks [0.7879756662633696]
  Internet of Things (IoT) devices have increased the scale and diversity of cyberattacks, exposing limitations in traditional intrusion detection systems.<n>This study investigates lightweight decoder-only Large Language Models (LLMs) for IoT attack detection by integrating structured-to-text conversion, Quantized Low-Rank Adaptation (QLoRA) fine-tuning, and Retrieval-Augmented Generation (RAG)
  arXiv  Detail & Related papers  (2026-01-21T18:52:26Z)
• OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMs [36.57820295876294]
  We introduce OpenRT, a unified, modular, and high- throughput red-teaming framework for MLLM safety evaluation.<n>At its core, OpenRT architects a paradigm shift in automated red-teaming by introducing an adversarial kernel that enables modular separation across five dimensions.<n>Our framework integrates 37 diverse attack methodologies, spanning white-box gradients, multi-modal perturbations, and sophisticated multi-agent evolutionary strategies.
  arXiv  Detail & Related papers  (2026-01-04T16:41:33Z)
• Large Language Models for Real-World IoT Device Identification [5.841950328636518]
  We introduce a semantic inference pipeline that reframes device identification as a language modeling task over heterogeneous network metadata.<n>To construct reliable supervision, we generate high-fidelity vendor labels for the IoT Inspector dataset.<n>We then instruction-tune a quantized LLaMA3.18B model with curriculum learning to support generalization under sparsity and long-tail vendor distributions.
  arXiv  Detail & Related papers  (2025-09-24T05:33:48Z)
• Securing Transformer-based AI Execution via Unified TEEs and Crypto-protected Accelerators [19.93096649006403]
  Machine learning runs on untrusted cloud infrastructure, exposing data and models to potential breaches.<n>Running model inference entirely within trusted execution environments (TEEs) is subject to non-trivial slowdown.<n>We propose TwinShield, a framework enabling secure Transformer inference in heterogeneous TEE and accelerator systems.
  arXiv  Detail & Related papers  (2025-07-04T03:52:53Z)
• LoRA-Gen: Specializing Large Language Model via Online LoRA Generation [68.01864057372067]
  We propose the LoRA-Gen framework to generate LoRA parameters for edge-side models based on task descriptions.<n>We merge the LoRA parameters into the edge-side model to achieve flexible specialization.<n>Our method facilitates knowledge transfer between models while significantly improving the inference efficiency of the specialized model.
  arXiv  Detail & Related papers  (2025-06-13T10:11:01Z)
• PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing [48.30406812516552]
  We introduce the PLM, a Peripheral Language Model, developed through a co-design process that jointly optimize model architecture and edge system constraints.<n>PLM employs a Multi-head Latent Attention mechanism and employs the squared ReLU activation function to encourage sparsity, thereby reducing peak memory footprint.<n> evaluation results demonstrate that PLM outperforms existing small language models trained on publicly available data.
  arXiv  Detail & Related papers  (2025-03-15T15:11:17Z)
• Unlocking Tuning-Free Few-Shot Adaptability in Visual Foundation Models by Recycling Pre-Tuned LoRAs [76.40876036912537]
  Large Language Models (LLMs) demonstrate strong few-shot adaptability without requiring fine-tuning.<n>Current Visual Foundation Models (VFMs) require explicit fine-tuning with sufficient tuning data.<n>We propose a framework, LoRA Recycle, that distills a meta-LoRA from diverse pre-tuned LoRAs with a meta-learning objective.
  arXiv  Detail & Related papers  (2024-12-03T07:25:30Z)
• Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities [50.980446687774645]
  We introduce ADV-LLM, an iterative self-tuning process that crafts adversarial LLMs with enhanced jailbreak ability.<n>Our framework significantly reduces the computational cost of generating adversarial suffixes while achieving nearly 100% ASR on various open-source LLMs.<n>It exhibits strong attack transferability to closed-source models, achieving 99% ASR on GPT-3.5 and 49% ASR on GPT-4, despite being optimized solely on Llama3.
  arXiv  Detail & Related papers  (2024-10-24T06:36:12Z)
• SHFL: Secure Hierarchical Federated Learning Framework for Edge Networks [26.482930943380918]
  Federated Learning (FL) is a distributed machine learning paradigm designed for privacy-sensitive applications that run on resource-constrained devices with non-Identically and Independently Distributed (IID) data. Traditional FL frameworks adopt the client-server model with a single-level aggregation process, where the server builds the global model by aggregating all trained local models received from client devices.
  arXiv  Detail & Related papers  (2024-09-23T14:38:20Z)
• Distributed Threat Intelligence at the Edge Devices: A Large Language Model-Driven Approach [0.0]
  Decentralized threat intelligence on edge devices represents a promising paradigm for enhancing cybersecurity on resource-constrained edge devices. This approach involves the deployment of lightweight machine learning models directly onto edge devices to analyze local data streams, such as network traffic and system logs, in real-time. Our proposed framework can improve edge computing security by providing better security in cyber threat detection and mitigation by isolating the edge devices from the network.
  arXiv  Detail & Related papers  (2024-05-14T16:40:37Z)
• Training Neural Networks from Scratch with Parallel Low-Rank Adapters [46.764982726136054]
  We introduce LoRA-the-Explorer (LTE), a novel bi-level optimization algorithm designed to enable parallel training of multiple low-rank heads across computing nodes. Our approach includes extensive experimentation on vision transformers using various vision datasets, demonstrating that LTE is competitive with standard pre-training.
  arXiv  Detail & Related papers  (2024-02-26T18:55:13Z)
• Towards Accurate Knowledge Transfer via Target-awareness Representation Disentanglement [56.40587594647692]
  We propose a novel transfer learning algorithm, introducing the idea of Target-awareness REpresentation Disentanglement (TRED) TRED disentangles the relevant knowledge with respect to the target task from the original source model and used as a regularizer during fine-tuning the target model. Experiments on various real world datasets show that our method stably improves the standard fine-tuning by more than 2% in average.
  arXiv  Detail & Related papers  (2020-10-16T17:45:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.