Vision Transformer with Deformable Attention

🤖 Plain-English Summary

Transformers have recently shown superior performances on various vision tasks. Extensive experi-ments show that our models achieve consistently improved results on comprehensive benchmarks.

🔑 Key Findings

• The large, sometimes even global, receptive field endows Transformer models with higher representation power over their CNN counterparts.
• Nevertheless, simply enlarging receptive field also gives rise to several concerns.
• On the one hand, using dense attention e.g., in ViT, leads to excessive memory and computational cost, and features can be influenced by irrelevant parts which are beyond the region of interests.

🔗 Related Resources