BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

🤖 Plain-English Summary

Vision-Language Pre-training (VLP) has advanced the performance for many vision-language tasks. BLIP also demonstrates strong generalization ability when directly transferred to video-language tasks in a zero-shot manner.

🔑 Key Findings

• However, most existing pre-trained models only excel in either understanding-based tasks or generation-based tasks.
• Additionally, performance improvement has been largely achieved by scaling up the dataset with noisy image-text pairs collected from the web, which is a suboptimal source of supervision.
• In this paper, we propose BLIP, a new VLP framework which transfers flexibly to both vision-language understanding and generation tasks.

🔗 Related Resources