Lightweight Vision Transformer Compression for On-Device Plant Disease Detection in Resource-Constrained Agricultural Field Conditions

The researchers proposed a unified lightweight visual Transformer compression framework aimed at solving the problem of deploying in resource-constrained agricultural environments. This framework integrates adaptive block pruning based on second-order sensitivity estimation (H-BAC), quantization techniques, and attention knowledge distillation methods. Experiments verified the effectiveness of this approach on the pepper three-class classification dataset: the integrated model maintained an accuracy comparable to that of the FP32 baseline (95.13%), while reducing the size of the INT8 version model to 6.01 MB, with a reduction of 74%-98%. The fully integrated solution achieved a compression ratio of up to 54.5 times. Comparative experiments further confirmed that compared to directly trained student models of the same size, introducing H-BAC and knowledge distillation techniques provided additional performance benefits under specific cost constraints.