RePaint: Inpainting using Denoising Diffusion Probabilistic Models

🤖 Plain-English Summary

Free-form inpainting is the task of adding new content to an image in the regions specified by an arbitrary binary mask. Re-Paint outperforms advanced Autoregressive, and GAN approaches for at least five out of six mask distributions.

🔑 Key Findings

• Most existing approaches train for a certain distribution of masks, which limits their generalization capabilities to unseen mask types.
• Additionally, training with pixel-wise and perceptual losses often leads to simple textural extensions towards the missing areas instead of semantically meaningful generation.
• In this work, we propose RePaint: A Denoising Diffusion Probabilistic Model (DDPM) based inpainting approach that is applicable to even extreme masks.

🔗 Related Resources