FAVE: Foveated Adaptive Visual Encoding for Efficient Fine-Grained Visual Understanding
2026-09-07 12:00Models🔥 42.2 heat score
1sources
1days unfolding
42.2heat score
5mentions
SummaryAI generated
FAVE proposes a lightweight variable-resolution Vision Transformer (ViT) that retains the native geometric structure while encoding the selected local regions with high clarity. In the ImageNet 96-pixel small object detection task, this method improved the Top-1 accuracy by 9.4 points compared to fixed-resolution ViT, while reducing the computational cost by 12.7 times. Subsequently, FAVE was integrated into the FastVLM model as a complementary local branch. By adding only up to 16 local tokens, significant performance and efficiency improvements were achieved: in the TextVQA task, the score increased by 1.60 points, and the inference first-word delay (TTFT) was 3.3 times faster than SmolVLM2-2.2B; in the GQA attribute-questioning problem, the performance of the FastVLM-1.5B model improved by 1.31 points.
FAVE proposes a lightweight variable-resolution ViT that encodes external regions with high clarity while preserving the native geometry. On ImageNet with 96-pixel small objects, this method improves the Top-1 accuracy by 9.4 points compared to fixed-resolution ViT, and reduces the computational cost by 12.7 times. Subsequently, FAVE is integrated into FastVLM as a complementary local branch, adding only up to 16 local tokens, resulting in a 1.60-point improvement in TextVQA scores and a 3.3-times faster TTFT speed compared to SmolVLM2-2.2B; in the GQA attribute problem, FastVLM-1.5B improves performance by 1.31 points.