JCa2Co: A joint cascade convolution coding network based on fuzzy regional characteristics for infrared and visible image fusion

• Main Authors: Zhao Xu, Gang Liu, Gang Xiao, Lili Tang, Yanhui Li
• Format: Article
• Language: English
• Published: Wiley 2021-10-01
• Series: IET Computer Vision
• Online Access: https://doi.org/10.1049/cvi2.12046
• ISSN: 1751-9632; 1751-9640

Abstract To facilitate the extraction of source image information, and preserve the consistency of heterogeneous regional features, a multiscale image fusion method based on a joint cascaded convolutional coding network (JCa2Co) is proposed. The JCa2Co network can retain vast quantities of information from source images in a multiscale perspective. The approach includes an encoder, a fusion layer and decoder. In the fusion layer, the Fuzzy Regional Characteristics (FRC) scheme is considered, and the multiscale feature maps are extracted from image subregions to ensure regional image consistency. Firstly, a joint cascaded encoder is used to extract multiscale features of the source image, in which the output of each layer is connected to every other layer. The fusion layer based on FRC is then performed to fuse each scale feature. Finally, the fused image is reconstructed by the decoder. In addition, to verify the regional consistency of the fused image, a regional consistency measure is proposed. Experiments are performed on the TNO Image Fusion Database. The experimental results exhibit that the proposed JCa2Co method has better comprehensive performance than the eight state‐of‐the‐art fusion methods. Moreover, it can effectively integrate meaningful information in infrared and visible images and has excellent performance in objective evaluation and visual quality, which is beneficial to target recognition and tracking.