JMLNet: Joint Multi-Label Learning Network for Weakly Supervised Semantic Segmentation in Aerial Images

• Main Authors: Rongxin Guo, Xian Sun, Kaiqiang Chen, Xiao Zhou, Zhiyuan Yan, Wenhui Diao, Menglong Yan
• Format: Article
• Language: English
• Published: MDPI AG 2020-09-01
• Series: Remote Sensing
• Subjects: deep learning; image segmentation; weak supervision; aerial image; multi-label learning
• Online Access: https://www.mdpi.com/2072-4292/12/19/3169

Weakly supervised semantic segmentation in aerial images has attracted growing research attention due to the significant saving in annotation cost. Most of the current approaches are based on one specific pseudo label. These methods easily overfit the wrongly labeled pixels from noisy label and limit the performance and generalization of the segmentation model. To tackle these problems, we propose a novel joint multi-label learning network (JMLNet) to help the model learn common knowledge from multiple noisy labels and prevent the model from overfitting one specific label. Our combination strategy of multiple proposals is that we regard them all as ground truth and propose three new multi-label losses to use the multi-label guide segmentation model in the training process. JMLNet also contains two methods to generate high-quality proposals, which further improve the performance of the segmentation task. First we propose a detection-based GradCAM (GradCAM<inline-formula><math display="inline"><semantics><msup><mrow></mrow><mi>D</mi></msup></semantics></math></inline-formula>) to generate segmentation proposals from object detectors. Then we use GradCAM<inline-formula><math display="inline"><semantics><msup><mrow></mrow><mi>D</mi></msup></semantics></math></inline-formula> to adjust the GrabCut algorithm and generate segmentation proposals (GrabCut<inline-formula><math display="inline"><semantics><msup><mrow></mrow><mi>C</mi></msup></semantics></math></inline-formula>). We report the state-of-the-art results on the semantic segmentation task of iSAID and mapping challenge dataset when training with bounding boxes annotations.