DINOv2: Learning Robust Visual Features without Supervision

🤖 Plain-English Summary

The recent breakthroughs in natural language processing for model pretraining on large quantities of data have opened the way for similar foundation models in computer vision. In terms of data, we propose an automatic pipeline to build a dedicated, diverse, and curated image dataset instead of uncurated data, as typically done in the self-supervised literature.

🔑 Key Findings

• These models could greatly simplify the use of images in any system by producing all-purpose visual features, i.e., features that work across image distributions and tasks without finetuning.
• This work shows that existing pretraining methods, especially self-supervised methods, can produce such features if trained on enough curated data from diverse sources.
• We revisit existing approaches and combine different techniques to scale our pretraining in terms of data and model size.

🔗 Related Resources