Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection

🤖 Plain-English Summary

This study addresses the issue of fusing infrared and visible images that appear differently for object detection. Extensive experiments on several public datasets and our benchmark demonstrate that our method outputs not only visually appealing fusion but also higher detection mAP than the advanced approaches.

🔑 Key Findings

• Aiming at generating an image of high visual quality, previous approaches discover commons underlying the two modalities and fuse upon the common space either by iterative optimization or deep networks.
• These approaches neglect that modality differences implying the complementary information are extremely important for both fusion and subsequent detection task.
• This paper proposes a bilevel optimization formulation for the joint problem of fusion and detection, and then unrolls to a target-aware Dual Adversarial Learning (TarDAL) network for fusion and a commonly used detection network.

🔗 Related Resources