Hierarchical Multi-Label Object Detection Framework for Remote Sensing Images

• Main Authors: Su-Jin Shin, Seyeob Kim, Youngjung Kim, Sungho Kim
• Format: Article
• Language: English
• Published: MDPI AG 2020-08-01
• Series: Remote Sensing
• Subjects: object detection; remote sensing images; convolutional neural network (CNN); hierarchical multi-label classification
• Online Access: https://www.mdpi.com/2072-4292/12/17/2734

Detecting objects such as aircraft and ships is a fundamental research area in remote sensing analytics. Owing to the prosperity and development of CNNs, many previous methodologies have been proposed for object detection within remote sensing images. Despite the advance, using the object detection datasets with a more complex structure, i.e., datasets with hierarchically multi-labeled objects, is limited to the existing detection models. Especially in remote sensing images, since objects are obtained from bird’s-eye view, the objects are captured with restricted visual features and not always guaranteed to be labeled up to fine categories. We propose a hierarchical multi-label object detection framework applicable to hierarchically partial-annotated datasets. In the framework, an object detection pipeline called <i>Decoupled Hierarchical Classification Refinement</i> (DHCR) fuses the results of two networks: (1) an object detection network with multiple classifiers, and (2) a hierarchical sibling classification network for supporting hierarchical multi-label classification. Our framework additionally introduces a region proposal method for efficient detection on vain areas of the remote sensing images, called <i>clustering-guided cropping</i> strategy. Thorough experiments validate the effectiveness of our framework on our own object detection datasets constructed with remote sensing images from WorldView-3 and SkySat satellites. Under our proposed framework, DHCR-based detections significantly improve the performance of respective baseline models and we achieve state-of-the-art results on the datasets.