DN-DETR: Accelerate DETR Training by Introducing Query DeNoising

🤖 Plain-English Summary

We present in this paper a novel denoising training method to speedup DETR (DEtection TRansformer) training and offer a deepened understanding of the slow convergence issue of DETR-like methods. Compared with the baseline under the same setting, DN-DETR achieves comparable performance with 50% training epochs.

🔑 Key Findings

• We show that the slow convergence results from the instability of bipartite graph matching which causes inconsistent optimization goals in early training stages.
• To address this issue, except for the Hungarian loss, our method additionally feeds ground-truth bounding boxes with noises into Transformer decoder and trains the model to reconstruct the original boxes, which effectively reduces the bipartite graph matching difficulty and leads to a faster convergence.
• Our method is universal and can be easily plugged into any DETR-like methods by adding dozens of lines of code to achieve a remarkable improvement.

🔗 Related Resources