AgriKD: Cross-Architecture Knowledge Distillation for Efficient Leaf Disease Classification

• URL: http://arxiv.org/abs/2605.01355v2
• Date: Thu, 07 May 2026 04:46:48 GMT
• Title: AgriKD: Cross-Architecture Knowledge Distillation for Efficient Leaf Disease Classification
• Abstract summary: AgriKD is a cross-architecture knowledge distillation framework for efficient edge deployment.<n>It transfers knowledge from a Vision Transformer (ViT) teacher to a compact convolutional student model.<n>It achieves consistent predictive performance with negligible accuracy.
• Score: 0.05599792629509228
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated leaf disease classification is critical for early disease detection in resource-constrained field environments. Vision Transformers (ViTs) provide strong representation capability by modeling long-range dependencies and inter-class relationships; however, their high computational cost makes them impractical for deployment on edge devices. As a result, existing approaches struggle to effectively transfer these rich representations to lightweight models. This paper introduces AgriKD, a cross-architecture knowledge distillation framework for efficient edge deployment, which transfers knowledge from a Vision Transformer (ViT) teacher to a compact convolutional student model. To bridge the representational gap between Transformer and CNN architectures, the proposed approach integrates multiple distillation objectives at the output, feature, and relational levels, where each objective captures a different aspect of the teacher knowledge. This enables the student model to better preserve and utilize transformer-derived global representations. Experiments on multiple leaf disease datasets show that the distilled student achieves performance comparable to the teacher while significantly improving efficiency, reducing model parameters by approximately 172 times, computational cost by 47.57 times, and inference latency by 18-22 times. Furthermore, the optimized model is deployed across multiple runtime formats, including ONNX, TFLite Float16, and TensorRT FP16, achieving consistent predictive performance with negligible accuracy degradation. Real-world deployment on NVIDIA Jetson edge devices and a mobile application demonstrates reliable real-time inference, highlighting the practicality of AgriKD for AI-powered agricultural applications in resource-constrained environments.

Related papers

• DAIT: Distillation from Vision-Language Models to Lightweight Classifiers with Adaptive Intermediate Teacher Transfer [10.030998208145807]
  Large-scale Vision-Language Models (VLMs) encode rich multimodal semantics that are highly beneficial for fine-grained visual categorization.<n>Their prohibitive computational cost hinders practical deployment in resource-constrained environments.<n>We propose Distillation with Adaptive Intermediate Teacher transfer (DAIT) to facilitate adaptive knowledge transfer from VLMs to lightweight students.
  arXiv  Detail & Related papers  (2026-03-16T12:00:31Z)
• SnapGen++: Unleashing Diffusion Transformers for Efficient High-Fidelity Image Generation on Edge Devices [72.0937240883345]
  Recent advances in diffusion transformers (DiTs) have set new standards in image generation, yet remain impractical for on-device deployment.<n>We present an efficient DiT framework tailored for mobile and edge devices that achieves transformer-level generation quality under strict resource constraints.
  arXiv  Detail & Related papers  (2026-01-13T07:46:46Z)
• Learning from Stochastic Teacher Representations Using Student-Guided Knowledge Distillation [64.15918654558816]
  Self-distillation (SSD) training strategy is introduced for filtering and weighting teacher representation to distill from task-relevant representations only.<n> Experimental results on real-world affective computing, wearable/biosignal datasets from the UCR Archive, the HAR dataset, and image classification datasets show that the proposed SSD method can outperform state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-04-19T14:08:56Z)
• Underlying Semantic Diffusion for Effective and Efficient In-Context Learning [113.4003355229632]
  Underlying Semantic Diffusion (US-Diffusion) is an enhanced diffusion model that boosts underlying semantics learning, computational efficiency, and in-context learning capabilities.<n>We present a Feedback-Aided Learning (FAL) framework, which leverages feedback signals to guide the model in capturing semantic details.<n>We also propose a plug-and-play Efficient Sampling Strategy (ESS) for dense sampling at time steps with high-noise levels.
  arXiv  Detail & Related papers  (2025-03-06T03:06:22Z)
• Edge-AI for Agriculture: Lightweight Vision Models for Disease Detection in Resource-Limited Settings [0.0]
  The proposed system integrates advanced object detection, classification, and segmentation models, optimized for deployment on edge devices.<n>The study evaluates the performance of various state-of-the-art models, focusing on their accuracy, computational efficiency, and generalization capabilities.
  arXiv  Detail & Related papers  (2024-12-23T06:48:50Z)
• Efficient Partitioning Vision Transformer on Edge Devices for Distributed Inference [13.533267828812455]
  We propose a novel framework, ED-ViT, which is designed to efficiently split and execute complex Vision Transformers across multiple edge devices.<n>Our approach involves partitioning Vision Transformer models into several sub-models, while each dedicated to handling a specific subset of data classes.<n>We demonstrate that our method significantly cuts down inference latency on edge devices and achieves a reduction in model size by up to 28.9 times and 34.1 times, respectively.
  arXiv  Detail & Related papers  (2024-10-15T14:38:14Z)
• TSCM: A Teacher-Student Model for Vision Place Recognition Using Cross-Metric Knowledge Distillation [6.856317526681759]
  Visual place recognition plays a pivotal role in autonomous exploration and navigation of mobile robots.<n>Existing methods overcome this by exploiting powerful yet large networks.<n>We propose a high-performance teacher and lightweight student distillation framework called TSCM.
  arXiv  Detail & Related papers  (2024-04-02T02:29:41Z)
• Distilling Knowledge from CNN-Transformer Models for Enhanced Human Action Recognition [1.8722948221596285]
  The research aims to enhance the performance and efficiency of smaller student models by transferring knowledge from larger teacher models. The proposed method employs a Transformer vision network as the student model, while a convolutional network serves as the teacher model. The Vision Transformer (ViT) architecture is introduced as a robust framework for capturing global dependencies in images.
  arXiv  Detail & Related papers  (2023-11-02T14:57:58Z)
• Energy-efficient Task Adaptation for NLP Edge Inference Leveraging Heterogeneous Memory Architectures [68.91874045918112]
  adapter-ALBERT is an efficient model optimization for maximal data reuse across different tasks. We demonstrate the advantage of mapping the model to a heterogeneous on-chip memory architecture by performing simulations on a validated NLP edge accelerator.
  arXiv  Detail & Related papers  (2023-03-25T14:40:59Z)
• Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge Distillation [65.62538699160085]
  We propose a Directed Acyclic Graph Factorization Machine (KD-DAGFM) to learn the high-order feature interactions from existing complex interaction models for CTR prediction via Knowledge Distillation. KD-DAGFM achieves the best performance with less than 21.5% FLOPs of the state-of-the-art method on both online and offline experiments.
  arXiv  Detail & Related papers  (2022-11-21T03:09:42Z)
• Efficient Vision Transformers via Fine-Grained Manifold Distillation [96.50513363752836]
  Vision transformer architectures have shown extraordinary performance on many computer vision tasks. Although the network performance is boosted, transformers are often required more computational resources. We propose to excavate useful information from the teacher transformer through the relationship between images and the divided patches.
  arXiv  Detail & Related papers  (2021-07-03T08:28:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.