Lightweight Vision Transformer Compression for On-Device Plant Disease Detection in Resource-Constrained Agricultural Field Conditions

• URL: http://arxiv.org/abs/2609.05334v1
• Date: Fri, 04 Sep 2026 16:40:06 GMT
• Title: Lightweight Vision Transformer Compression for On-Device Plant Disease Detection in Resource-Constrained Agricultural Field Conditions
• Abstract summary: Vision Transformers (ViTs) have achieved high classification accuracy, but their large computational footprint makes deployment on resource constrained devices challenging.<n>We propose a unified Vision Transformer compression framework that combines H-BAC, guided by second-order sensitivity estimation, with quantization and attention-based knowledge distillation.
• Score: 1.5530227558314913
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Chilli (Capsicum annuum) is one of India's most economically significant crops, yet its productivity is persistently threatened by diseases that are difficult to identify without expert intervention. While Vision Transformers (ViTs) have achieved high classification accuracy, their large computational footprint makes deployment on resource constrained devices challenging. Existing compression approaches typically address pruning, quantization, and knowledge distillation in isolation, leaving the potential benefits and interactions of their combined application insufficiently explored. We propose a unified Vision Transformer compression framework that combines Hessian-Balanced Adaptive Block Pruning (H-BAC), guided by second-order sensitivity estimation, with quantization and attention-based knowledge distillation. To systematically identify the most effective configuration within each compression family, each technique is first evaluated independently through controlled ablation studies, after which the best-performing components are integrated into a sequential deployment pipeline tailored to real-world agricultural constraints. On a chilli 3-class village-split dataset with a genuine cross-village, cross-device out-of-distribution test split, the resulting compressed models match or exceed the 95.13% FP32 baseline's accuracy, alongside 74-98% model size reduction, and the fully integrated compression pipeline achieves a 54.5x size reduction (327.42 MB to 6.01 MB) at 95.13 +/- 2.32% accuracy across four tested configurations. A direct comparison further reveals that, on this dataset, a directly-trained student of the same final size, without pruning or distillation, reaches comparable accuracy of 94.87%, at the same 6.01 MB INT8 size, indicating where H-BAC and knowledge distillation are, and are not yet shown to be, worth their computational cost.

Related papers

• AgroVisNet: A lightweight Convolutional Network and the BD-PlantDX Expert-Validated Benchmark for Radish, Potato and Pointed Gourd Disease Classification [0.3686530147760243]
  We propose AgroVisNet, a compact convolutional network trained from scratch, together with BD-PlantDX.<n>On BD-PlantDX the model attains 99.52% test accuracy and 99.52% weighted F1, exceeding all six ImageNet-pretrained lightweight backbones evaluated.<n>Exported for deployment, the model quantises to a 0.46 MB full-integer network at a 0.22 percentage-point accuracy cost and classifies an image in 8.40 ms on a single CPU.
  arXiv  Detail & Related papers  (2026-09-09T17:12:17Z)
• A Multidimensional Data-Driven Hybrid Transformer Framework for Non-invasive Continuous Blood Pressure Prediction [28.58502100288247]
  We propose a hybrid Transformer framework to estimate diastolic and systolic BP from ECG/III feature sequences.<n>Rather than raw waveforms, the framework models 10-step sequences of physiological descriptors and two demographic cos.
  arXiv  Detail & Related papers  (2026-08-24T14:02:10Z)
• Towards Compact Unified Multimodal Tracking: Synergizing Knowledge Distillation with Structural Pruning [80.73561460934809]
  multimodal object tracking has achieved remarkable robustness by leveraging complementary sensor data.<n>We identify the prediction head as a critical but often overlooked efficiency bottleneck.<n>By strategically streamlining the decoder architecture, we unlock the potential for real-time inference.
  arXiv  Detail & Related papers  (2026-08-02T20:41:33Z)
• Pixel-Precise Explainable Stress Indexing: A Semantic Segmentation Framework for Disease Severity Quantification in Field Crops [3.466303200807006]
  Plant diseases, resulting from both biotic and abiotic stresses, cause an estimated 20-40% loss in global agricultural yield annually.<n>This paper proposes a unified deep learning pipeline integrating semantic segmentation, regression-based severity estimation, and disease classification.
  arXiv  Detail & Related papers  (2026-07-05T06:04:20Z)
• Condensing Large-Scale Datasets Directly with Minimal Information Loss [34.43216741538152]
  CIM is a novel, metric-driven framework that abandons the flawed dual-compression paradigm.<n> CIM explicitly quantifies and minimizes the information gap between the original and synthetic datasets.
  arXiv  Detail & Related papers  (2026-07-01T13:21:44Z)
• Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization [36.286359836094086]
  We propose TASA (Task-Aware Sensitivity Analysis) to jointly optimize calibration-data composition and mixed-precision bit allocation.<n>Experiments on LLaMA-3-8B and Qwen2.5-7B reveal a precision inversion: appropriately allocated 3.5-bit models can match or surpass less task-aware 4-bit baselines.
  arXiv  Detail & Related papers  (2026-07-01T13:12:21Z)
• Activation Outliers in Transformer Quantization: Reproduction, Statistical Analysis, and Deployment Tradeoffs [0.0]
  Post-training quantization (PTQ) of transformers is known to suffer from severe accuracy degradation due to structured activation outliers.<n>This paper provides a reproducible empirical reproduction and systems-level extension of that phenomenon in BERT-base fine-tuned on QNLI.
  arXiv  Detail & Related papers  (2026-03-04T17:26:29Z)
• An Efficient Additive Kolmogorov-Arnold Transformer for Point-Level Maize Localization in Unmanned Aerial Vehicle Imagery [9.080987184733456]
  High-resolution UAV photogrammetry has become a key technology for precision agriculture.<n>Point-level maize localization in UAV imagery remains challenging due to extremely small object-to-pixel ratios.<n>We propose the Additive Kolmogorov-Arnold Transformer (AKT) to address these challenges.
  arXiv  Detail & Related papers  (2026-01-12T20:16:10Z)
• A Multimodal Approach to Heritage Preservation in the Context of Climate Change [0.0]
  We propose a lightweight multimodal architecture that fuses sensor data (temperature, humidity) with visual imagery to predict severity at heritage sites.<n>On data from Strasbourg Cathedral, our model achieves 76.9% accu- racy, a 43% improvement over standard multimodal architectures.
  arXiv  Detail & Related papers  (2025-10-15T22:07:57Z)
• Hybrid Knowledge Transfer through Attention and Logit Distillation for On-Device Vision Systems in Agricultural IoT [0.0]
  This work advances real-time, energy-efficient crop monitoring in precision agriculture.<n>It demonstrates how we can attain ViT-level diagnostic precision on edge devices.
  arXiv  Detail & Related papers  (2025-04-21T06:56:41Z)
• Extreme Compression for Pre-trained Transformers Made Simple and Efficient [31.719905773863566]
  Extreme compression, particularly ultra-low bit precision (binary/ternary) quantization, has been proposed to fit large NLP models on resource-constraint devices. We propose a simple yet effective compression pipeline for extreme compression, named XTC.
  arXiv  Detail & Related papers  (2022-06-04T00:19:45Z)
• Compressive Visual Representations [30.857175484570732]
  Two families of self-supervised methods, contrastive learning and latent bootstrapping, have made significant progress. We develop SimCLR and BYOL formulations compatible with the Conditional Entropy Bottleneck objective. Experiments confirm that adding compression to SimCLR and BYOL significantly improves linear evaluation accuracies and robustness model across a wide range of domain shifts.
  arXiv  Detail & Related papers  (2021-09-27T09:53:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.