MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision\n Transformer

🤖 Plain-English Summary

Light-weight convolutional neural networks (CNNs) are the de-facto for mobile\nvision tasks. On the ImageNet-1k dataset,\nMobileViT achieves top-1 accuracy of 78.4% with about 6 million parameters,\nwhich is 3.2% and 6.2% more accurate than MobileNetv3 (CNN-based) and DeIT\n(ViT-based) for a similar number of parameters.

🔑 Key Findings

• Their spatial inductive biases allow them to learn\nrepresentations with fewer parameters across different vision tasks.
• However,\nthese networks are spatially local.
• To learn global representations,\nself-attention-based vision trans-formers (ViTs) have been adopted.

🔗 Related Resources