Semantic Relation Model and Dataset for Remote Sensing Scene Understanding

• Main Authors: Peng Li, Dezheng Zhang, Aziguli Wulamu, Xin Liu, Peng Chen
• Format: Article
• Language: English
• Published: MDPI AG 2021-07-01
• Series: ISPRS International Journal of Geo-Information
• Subjects: remote sensing scene understanding; semantic relation cognition; scene graph generation; multi-scale semantic fusion; attentional mechanism; graph convolutional network
• Online Access: https://www.mdpi.com/2220-9964/10/7/488

A deep understanding of our visual world is more than an isolated perception on a series of objects, and the relationships between them also contain rich semantic information. Especially for those satellite remote sensing images, the span is so large that the various objects are always of different sizes and complex spatial compositions. Therefore, the recognition of semantic relations is conducive to strengthen the understanding of remote sensing scenes. In this paper, we propose a novel multi-scale semantic fusion network (MSFN). In this framework, dilated convolution is introduced into a graph convolutional network (GCN) based on an attentional mechanism to fuse and refine multi-scale semantic context, which is crucial to strengthen the cognitive ability of our model Besides, based on the mapping between visual features and semantic embeddings, we design a sparse relationship extraction module to remove meaningless connections among entities and improve the efficiency of scene graph generation. Meanwhile, to further promote the research of scene understanding in remote sensing field, this paper also proposes a remote sensing scene graph dataset (RSSGD). We carry out extensive experiments and the results show that our model significantly outperforms previous methods on scene graph generation. In addition, RSSGD effectively bridges the huge semantic gap between low-level perception and high-level cognition of remote sensing images.